managing a large-ish dataset of questions

Godot Version

latest (4.3)

Question

I have a large-ish dataset of trivia questions (around 70 thousand questions). Each question is tagged with a difficult level, set of categories and a few other traits. My app needs to start the user off on easy questions and as they progress, increases the difficulty level. I also would like to avoid showing a user duplicate questions.

I tried two approaches:

  1. Use a csv file
  2. Use a database (in this case some sqlite package for godot from assetlib

The csv file had issues mainly around performance - I needed to avoid asing the user questions they have already seen and I needed to pull questions at a certain level of difficulty (easy/medium/hard) and filter down by category (math, physics etc) in some scenarios. It seemed like a relational database would solve this better.

The sqlite lite approach works to solve the filtering issue but the package itself was not compatible with some old cheap android device (whereas a csv would have worked and been compatible across the board). Also - it seems that in order to delete rows in the database, I would have to copy the db to the user data directory so it is writeable (caused some noticeable lag when first copying the ~25mb db file)

My question is - is there an obvious pattern for this kind of thing that i am overlooking?