improve questions

[jeremymanning]

the current question set's questions are too long and difficult: it takes a long time to read, think about, and answer each one. further, many questions can be answered by logic alone, as opposed to actual deep conceptual or content knowledge.

i think we need to overhaul the question generation approach. first, we should use a very high quality model-- claude opus-4.6, not gpt-5.2-nano. second, i suggest generating questions as follows:

1. for each domain, generate 50 unique concepts that are central to that area
2. revise the list by removing duplicates. cull down to the 50 most central and important concepts.
3. for each concept:

• look up (and read) the wikipedia article (via web fetch). use the article for context.
• generate a short (50-100ish word) question that gauges one or more of the following, depending on the level:
  • level 1: knowledge of the relevant high-level vocabulary
  • level 2: knowledge of the relevant low-level vocabulary
  • level 3: basic working knowledge of the given concept (cannot answer through logic along, nor by rote memorization alone)
  • level 4: deep knowledge of the given concept (that cannot be answered through logic alone, nor by rote memorization alone)
• then generate the correct response to the question. it should be short (25-50ish words)
• now generate 3 "distractor" (incorrect) responses by prompting agents to repeatedly asking it to “change something about this statement to make it an incorrect answer to the question” (providing the question and the correct response)
• keep track of the core concept being tested, for the json file

then during display, to ensure random responses, ALWAYS randomize the response option order every time a new question is displayed.

also: make sure to properly format math expressions as latex, and use the escape character to denote dollar signs to ensure easy parsing.