Skip to main content
Version: v0 (current)

Instructor repository internals

When instructor_repo_token is set in a Classroom 50 assignment repository, GrillMyCode writes a full copy of each assessment, with questions and answers, to a private repository for the assignment. It works only in conjunction with Classroom 50, because the assignment and student are read from Classroom 50's repository naming. This page covers how that works. To set it up, see Keeping a private answer key.

How the assignment and student are identified​

Classroom 50 names every student repository <classroom>-<assignment>-<username> (lowercased), and gh student accept adds the accepting student to it as a direct collaborator. The action reads both names from those two facts and nothing else:

  • Student folder: the repository's direct collaborator whose login ends the repository name. For cs-principles-lab-3-jsmith with direct collaborator jsmith, the folder is jsmith/.
  • Instructor repository: everything before that login, plus -grillmycode-instructor, so cs-principles-lab-3-grillmycode-instructor. The classroom slug stays attached, which keeps two classrooms running the same assignment in separate instructor repositories.
  • Team assignments: a team repository ends in -group-<n> rather than a login, and its assessment is filed under group-<n>/.

The repository name cs-principles-lab-3-jsmith split in two: cs-principles-lab-3 (the classroom and assignment) and -jsmith (the login), which matches the repository's one direct collaborator, jsmith. The first part, plus -grillmycode-instructor, names the instructor repository cs-principles-lab-3-grillmycode-instructor; the login names the student folder jsmith/. A team repository, cs-principles-lab-3-group-2, is filed under group-2/. Anything else, with no match or more than one, gets no answer key: a workflow warning, and the issue and PDF as normal. A footnote says who pushed, who started the run, commit authors and the template are never used.

Who pushed, who started the run, who authored the commits and which template the repository came from play no part. Pushing a workflow file into a student's repository, or running the workflow by hand, never changes where an assessment is filed. Each run logs what it resolved; look for Assignment: cs-principles-lab-3 · Submitter: jsmith in the Actions log.

If a repository doesn't fit, the action doesn't guess. That covers a repository not created by Classroom 50, a student who is no longer a direct collaborator, or more than one collaborator whose login ends the name. It skips instructor delivery, raises a workflow warning giving the reason, and repeats the reason in the run summary. The student's issue and PDF are produced as normal.

Renaming an assignment with gh teacher assignment rename renames its student repositories too, so assessments made after a rename go to a new instructor repository named for the new slug.

What each run does​

Five steps in order, under the token each uses. With GITHUB_TOKEN: 1, posts the issue and PDF, always first, even with no instructor token. With INSTRUCTOR_REPO_TOKEN: 2, creates the private repository, only if it doesn't exist yet; 3, syncs the action-owned files, the README and quiz workflow; 4, writes the student's files, replacing the last run's; 5, writes questions.json last, because its commit builds the quiz. A note says a failure in steps 2 to 5 shows as an error on the run but never fails the job, since the student already has their questions, and conflicting writes and rate limits are retried.

  1. All student-facing work (the issue, the PDF) uses the workflow's GITHUB_TOKEN. Only the steps below use INSTRUCTOR_REPO_TOKEN.
  2. If the instructor repository doesn't exist, it is created as a private repository in the same organization.
  3. The action-owned files are synced (see below).
  4. The student's files are written to their folder, replacing any from an earlier run.
  5. questions.json is written last, and writing it triggers the Generate LMS Quiz workflow, which builds the student's quiz files.

Creating the repository needs the student repositories to belong to an organization. Under a personal account the create call fails; see Troubleshooting for the workaround.

A delivery failure is reported as an error annotation on the run but doesn't fail the job, so the student is never shown a failed assessment because of an instructor-side problem. Conflicting writes and rate limits, which are common when a whole class pushes at once, are retried up to five times. See Troubleshooting.

Repository layout​

README.md ← describes the repository (action-owned)
.github/workflows/
generate-lms-quiz.yml ← builds the quiz files (action-owned)
{student-login}/
questions.md ← the assessment: questions and answers
{assignment}_{student-login}_quiz_{count}.imscc ← quiz package for any LMS
{assignment}_{student-login}_brightspace_quiz_{count}.csv ← alternative, Brightspace only
data/ ← files you don't need to read
questions.json ← the same questions as data, for the quiz
raw-ai-output.md ← the AI's unprocessed reply, for diagnosis

The top of each student's folder holds only what you read or import. The data/ subfolder holds what GrillMyCode itself uses, and what you only need when something looks wrong.

For the classroom cs-principles, assignment lab-3 and student jsmith:

  • Instructor repository: https://github.com/my-school/cs-principles-lab-3-grillmycode-instructor
  • Assessment: jsmith/questions.md
  • Quiz package: jsmith/cs-principles-lab-3_jsmith_quiz_20.imscc
  • Brightspace alternative: jsmith/cs-principles-lab-3_jsmith_brightspace_quiz_20.csv
  • Raw AI output: jsmith/data/raw-ai-output.md

Each run replaces the student's files, so there is always exactly one up-to-date assessment per student (per delivery group). If the question count changes, the new count appears in the quiz file names and the files carrying the old count are removed.

Action-owned files​

README.md and generate-lms-quiz.yml belong to the action. Every run compares them with the copies shipped in the action and rewrites any that differ, so each repository picks up fixes by itself. Local edits to them are replaced on the next run.

Writing under .github/workflows/ needs the token's workflow scope. Without it the sync logs a warning, and the assessment is still delivered; see Troubleshooting.

questions.md​

The instructor's copy of the report. It has the same header as the student's issue (see The assessment issue and PDF), plus:

  • the answer to every question, whatever include_answers is set to
  • three multiple-choice distractors per question, used to build the quiz
  • the full set of questions, including any withheld from the student's copy
  • on tag runs, a Submission line flagging a resubmission (see below)

Distractors depend on the token​

The three wrong options per question are only needed for the quiz, and every student-facing report strips them. Runs without instructor_repo_token therefore ask the model for the correct answer only, which is cheaper and quicker. With the token in place, every run generates distractors. A class that adds the token mid-semester needs no other change, but assessments produced before it was added have no options to build a quiz from; re-running the workflow on those repositories regenerates them.

questions.json​

In the data/ subfolder. The same questions as questions.md, as data: each question's code snippets with the file and line numbers they come from, question, answer, distractors, whether it is a broader question, and whether it is about the starter code alone (about_starter_code, only ever true with starter_code: ask). The quiz is built from this file alone. You don't need to open it unless you want to correct a question in the quiz: edit it here, and saving the change rebuilds that student's quiz. An edit to questions.md doesn't reach the quiz. The VS Code extension's instructor view reads this file too.

Questions GrillMyCode dropped because they didn't point at the student's code come last, with "dropped": true and no number. The student's report and the quiz leave them out, but you can see what the model asked and the code it showed. For a question dropped because its file or lines couldn't be found, only the file and line numbers are recorded, with no code. To put a dropped question in the quiz anyway, change its dropped to false.

raw-ai-output.md​

In the data/ subfolder. A diagnostic record, not something to read or import; questions.md is the assessment. It holds the model's reply exactly as it arrived: a JSON object, shown in a code block. Its snippets are file names and line numbers, not code. GrillMyCode has not yet checked it, copied in the code, dropped questions that don't point at the student's own code, cut questions beyond num_questions, or numbered and formatted the rest (see What code is assessed). It usually holds more questions than num_questions, because the model is asked for spares; the header gives both counts.

Open it when a student's questions.md looks wrong. Questions cut for being over the count, or dropped for missing question text or an answer, are only visible here; the ones dropped for not pointing at the student's code are also in questions.json. Include its contents in any bug report about generated questions.

The header records how the reply was produced:

  • why the model stopped. A length stop means it hit its output limit and the reply is incomplete.
  • how many tokens went in and out, and how many of those out were reasoning
  • how many attempts the request took
  • the settings used: questions requested, the temperature if one was set, reasoning effort, and a short hash identifying the prompt version

The same facts, with full commit SHAs, are embedded as JSON in a <!-- gmc:provenance … --> comment for tooling. It is invisible in the rendered view.

Quiz files​

The Generate LMS Quiz workflow runs whenever a questions.json is written, and can be started by hand from the Actions tab.

It checks every student but skips any whose questions haven't changed since their quiz was last built, so normally only the student who just pushed gets new files. It decides this by comparing a content hash stored inside each .imscc and .csv. A change to the quiz format, such as an updated workflow, leaves no matching hashes, so the next run rebuilds every student's quiz once; see Upgrade notes.

The .imscc is an IMS Common Cartridge package with a QTI quiz, titled {assignment} - {student-login} and limited to one attempt. The .csv holds the same questions in Brightspace's question-import format, with options pre-shuffled. See Importing quizzes into your LMS.

Each student's quiz is built from their questions.json. A folder without one gets no quiz. A questions.json that isn't valid JSON fails that student's quiz with an error on the run; every other student's quiz is still built.

Each quiz question shows its code snippets, each under the name of the file it comes from and the lines it covers, such as index.php, lines 28–37. The model occasionally writes a question without a snippet, most often under a Broader Questions heading near the end when the requested count is large for the size of the submission. That question appears in the quiz as text only.

A question without distractors, or with a blank option, is left out of the quiz rather than imported with a single option, and the run log names it. A placeholder or duplicate distractor is removed, and the question stays in the quiz with its remaining options. The count in the file name is the number of questions the quiz actually holds, so a short quiz can be spotted without opening it.

Submission tag folders​

With a tag-triggered workflow, each submission_tags entry gets its own subfolder inside the student's folder, so a milestone's assessment is kept when the next one arrives:

{student-login}/
{tag-group}/ ← e.g. phase1 or complete
questions.md
submissions.md ← every run for this tag, with a resubmission count
history/ ← question sets replaced by a resubmission
1-questions.md
{assignment}_{student-login}_{tag-group}_quiz_{count}.imscc
{assignment}_{student-login}_{tag-group}_brightspace_quiz_{count}.csv
data/
questions.json
raw-ai-output.md

The tag group is the submission_tags entry reduced to filename-safe characters; for a wildcard entry such as phase*, that is phase. It appears in the quiz file names and in the quiz title shown in the LMS (cs-principles-lab-3 - jsmith (phase1)). Within one group, the one-up-to-date-assessment rule still applies: re-pushing a tag replaces that group's questions.md and rebuilds its quiz.

Spotting resubmissions​

A student can resubmit under the same tag by moving it and pushing it again. Nothing is blocked, but every resubmission is flagged in three places:

  • The assessment itself. A resubmitted questions.md carries a line in its header, for example Submission: ⚠️ resubmitted — this is the 3rd submission of phase1 (previous: 2026-09-18 14:03 UTC, a1b2c3d).
  • submissions.md in the tag folder lists every run for that tag: the date, the trigger (tag push or manual run), who started it and the commit. It also states how many counted as student submissions.
  • history/ keeps each question set a resubmission replaced, as <#>-questions.md, numbered by the submissions.md row that produced it.

What counts as a submission: every tag push, and every manual run started by the student themselves. A manual run started by anyone else, such as your own re-run from the Actions tab, is listed in submissions.md but not counted, and its header says so. In a team repository there's no single student to compare against, so every run counts.

The record lives only in the instructor repository, so students can't see or edit it, and deleting and re-pushing a tag doesn't reset the count. The student's run summary also mentions a resubmission, but that count is informational; the instructor repository is the record to rely on.

Question set folders​

A workflow with a question set files everything one level down, in a set.{set} subfolder of the student's folder, tag group folders included:

{student-login}/
set.{set}/ ← e.g. set.lab-3, for a workflow named Lab 3
questions.md
{assignment}_{student-login}_set.{set}_quiz_{count}.imscc
{assignment}_{student-login}_set.{set}_brightspace_quiz_{count}.csv
data/
questions.json
raw-ai-output.md
{tag-group}/ ← the set's submission tag folders, as above

The set is in the quiz file names and, without the set. mark, in the quiz title (cs-principles-lab-3 - jsmith (lab-3), or (lab-3, phase1) in a tag group's folder). A workflow named GrillMyCode, or without question_set, files at the top of the student's folder, as in Repository layout.