Skip to main content
Version: Next (unreleased)

The assessment issue and PDF

Every run that generates questions delivers them to the student in two fixed places: a GitHub issue and a PDF attached to a release. Neither can be turned off, and both need the permissions in Tokens, secrets and permissions. For what this looks like to a student, see What your students see.

The issue​

Title and label​

Each delivery group has one issue, labelled assessment:

RunIssue title
Push, or manual run on a branchGrillMyCode Questions (main), with the branch's name
Submission tag phase1GrillMyCode Questions (tag: phase1)
A question set, Lab 3GrillMyCode Questions: Lab 3 (main), or GrillMyCode Questions: Lab 3 (tag: phase1)

Who the issue is assigned to​

The issue is assigned to the student the repository belongs to: the direct collaborator whose login ends the repository name (see how the assignment and student are identified). Who pushed or who started the run makes no difference. A team repository (…-group-<n>) has no single owner, so its issue isn't assigned.

Pinning​

The issue is pinned the first time it is created, so students find it at the top of their issues list. Later runs that update it don't pin it again. GitHub allows at most three pinned issues per repository; if that limit is already reached, pinning is skipped with a warning in the Actions log.

What the body contains​

A GrillMyCode heading with the logo, a Download as PDF button, then a header:

LineShown
GeneratedAlways: date and time in UTC
Student / RepositoryWhen known: the student's login and the owner/repo the questions came from. This makes each copy identifiable in the instructor repository
Commits reviewedAlways: base → head, as short SHAs
BranchOnly on a branch other than main or master
Submission tagOn tag runs, with the tag the diff started from when tag_diff_base is previous-tag or tag:<name>
Code Files AssessedAlways: the files that passed filtering and were sent to the AI
Assignment ContextWhen assignment_context matched any files
Codebase contextWhen starter_code or previous_work sent any files: how many, and that answers may depend on them though they are not assessed
Starter codeWhen starter_code is ask and any question is about starter code alone: how many. See Asking about starter code
Instructor NoteWhen instructor_context is set: a one-sentence summary of the question focus, written by the model

The questions follow, then a footer naming the model, provider and action version.

The student's copy never includes answers or multiple-choice distractors unless include_answers is 'true'. A question whose text would reveal an answer is withheld, and the report says how many. An Instructor Note that would reveal one is left out. See What code is assessed.

Hidden data​

Under the heading, the body holds one HTML comment, which GitHub doesn't display:

<!-- gmc:questions {"version":1,"headSha":"…","questions":[{"number":1,"broader":false,"snippets":[{"file":"src/app.js","start_line":12,"end_line":18}]}]} -->
FieldHolds
versionThe version of the issue's layout. It goes up when the body changes in a way a reader written for the last layout couldn't follow
headShaThe full SHA of the head commit, which Commits reviewed shows the start of
setOnly with a question set: its folder name, such as lab-3
questionsFor each question shown: its number, whether it is a broader question, and the file and lines of each snippet

The comment never holds a question's text, its code, an answer or a distractor, whether include_answers is on or off. A withheld question isn't listed.

It has two uses. The VS Code extension and the JetBrains plugin read it, and the action knows its own issues by it. The PDF and the instructor repository's copy don't carry it.

When the questions are regenerated​

Every run in the same delivery group overwrites the issue body. The issue number, URL and comment history stay the same. A note comment is added each time, recording the commit the new questions came from:

The assessment questions in this issue were regenerated at commit a1b2c3d and the questions have been updated. Any previous questions have been replaced.

Only an open issue with the exact title, whose body GrillMyCode wrote, is reused. If the student closes their issue, the next run opens a new one.

An issue counts as GrillMyCode's when its body has the hidden data, or opens the way every report did before that data existed: the GrillMyCode heading, then a Commits reviewed line. Any other issue with the same title and label, such as one a person opened by hand, is never overwritten or deleted. The questions go in an issue of their own, and the Actions log names the issue that was left alone. A GrillMyCode issue edited until neither mark remains is treated the same way.

If more than one open issue that GrillMyCode wrote exists for the same group, the extras are deleted. Deleting an issue needs admin rights, so if the workflow's token lacks them, the duplicates are left in place with a warning in the Actions log.

Length limit​

GitHub caps an issue body at 65,536 characters. If the report is longer than 65,000 characters, the issue body is cut short and a callout points to the PDF, which always has the full content. A warning is also logged. To avoid it, lower num_questions or shorten instructor_context.

The PDF​

The PDF is attached to a rolling GitHub release tagged gmc-assessments in the student's repository, and linked from the top of the issue.

What the PDF contains​

The same report as the issue body, including the logo beside the heading, with the GrillMyCode name at the top of every page and page numbers at the bottom. The logo is built into the PDF, and any other images or HTML in the report are left out, so nothing in it is loaded from the web.

File name​

The asset is named after the repository, grill-my-code-{repository}.pdf, with any character other than a letter, digit, - or _ replaced by -:

RepositoryPDF asset
cs-principles-lab-3-jsmithgrill-my-code-cs-principles-lab-3-jsmith.pdf
my.projectgrill-my-code-my-project.pdf

A repository has one PDF for all its push and manual runs, so it always holds the most recent assessment, whichever branch produced it.

A tag run adds the tag's group, meaning its submission_tags entry reduced to the same filename-safe characters, so each milestone keeps its own PDF:

Repositorysubmission_tags entryPDF asset
cs-principles-lab-3-jsmithphase1grill-my-code-cs-principles-lab-3-jsmith-phase1.pdf
cs-principles-lab-3-jsmithphase* (wildcard)grill-my-code-cs-principles-lab-3-jsmith-phase.pdf

Re-pushing a tag replaces that group's PDF, and every tag matching one wildcard entry shares a single asset.

A question set adds .set. and its folder name before any tag group, so each workflow keeps its own PDF: grill-my-code-cs-principles-lab-3-jsmith.set.lab-3.pdf, or grill-my-code-cs-principles-lab-3-jsmith.set.lab-3-phase1.pdf for a tag run.

The download URL follows the pattern:

https://github.com/{owner}/{repo}/releases/download/gmc-assessments/{filename}

The release tag and file name don't change between runs, so neither does the URL. Each run replaces the asset, and the link in the issue always gives the latest version. The URL is also available as the pdf_url output; see Inputs and outputs.

On a private repository, the link only works for someone signed in to GitHub with access to the repository. That is normal for release assets on private repositories.

If PDF generation fails, the issue is still created, without the PDF button, and pdf_url is empty.