Architecture

How the action runs

This is a Docker-based GitHub Action — rather than executing JavaScript directly on the runner, GitHub pulls a pre-built Docker image and runs the assessment script inside it. The image is published to the GitHub Container Registry (ghcr.io) and pinned in action.yml.

Consumer workflow
      │
      ▼
action.yml  ──► pulls Docker image from ghcr.io
                      │
                      ▼
              entrypoint.sh  ──► cd $GITHUB_WORKSPACE
                                       │
                                       ▼
                                 node src/main.js

Using Docker means:

• The Node version, git binary, and all dependencies are fixed and identical across every runner — no version drift
• The image is built once and reused; consumer repos pay no build cost at runtime
• The pre-built image reference in action.yml is updated automatically by the release workflow each time a version tag is pushed

---

Execution flow

When main.js runs, it follows this sequence:

readInputs()
    │  Reads all INPUT_* environment variables set by action.yml
    │
resolveTagName() → resolveSubmissionTag()
    │  Only on a run started by a tag (a tag push, or a manual run on a tag)
    │  Matches the tag against submission_tags; no match fails the run
    │  The matched pattern names the delivery group (issue, PDF, instructor folder)
    │
resolveSHAs()
    │  Determines baseSha and headSha from the event context
    │  Handles: push, workflow_dispatch, and tag runs (head peeled to its commit)
    │  Applies include_initial_commit override when enabled
    │  Applies tag_diff_base (previous-tag / tag:<name>) on a tag run
    │
resolveBranch()  — or, on a tag run, assertOnDefaultBranch()
    │  Extracts the branch name from the event payload or GITHUB_REF
    │  A tag run instead fails unless the tagged commit is on the default branch
    │
resolveSubmissionIdentity()
    │  Lists the repository's direct collaborators (one API call)
    │  Student = the collaborator whose login ends the Classroom 50 repo name
    │  (<classroom>-<assignment>-<username>); assignment = everything before it
    │  A repo ending in -group-<n> is a team submission, filed under group-<n>
    │  Anything else is unresolved, and instructor delivery is skipped
    │  Never consults the event sender, run actor, commit authors or template
    │
getChangedFiles() → filterFiles()
    │  Runs `git diff --name-only baseSha headSha`
    │  Applies auto-detected stack patterns, additional_exclude_patterns, and exclude_pattern_overrides via minimatch
    │
    ├─ no files survive → reportEmptyAssessment() and return
    │     Distinguishes an empty commit range from a fully excluded file list,
    │     writes a job summary naming the reason, then warns — or fails when
    │     fail_on_empty_assessment is enabled
    │
getDiff()
    │  Runs `git diff baseSha headSha -- <files>`
    │  Result kept as a fallback only — not sent to the AI directly
    │
collectRawFiles()
    │  Fetches full file content at headSha via `git show`
    │  (deleted files are silently skipped)
    │
stripCommentsFromFiles()
    │  Writes each file to /tmp, runs the rmcm binary on it
    │  Falls back silently to original content for unsupported types
    │  Falls back to raw diff if stripping produces no output at all
    │
buildCodeContent()
    │  Formats stripped files as fenced Markdown code blocks
    │
readAssignmentContextFiles()
    │  Reads files from GITHUB_WORKSPACE that match assignment_context globs
    │  Concatenates contents as headed sections; capped at assignment_context_max_chars input (default 20000)
    │  Returns an empty string when no globs are supplied or no files match
    │
buildPrompt()
    │  Constructs the system + user messages for the AI
    │  Injects assignment context (file contents) then instructor instructions
    │  AI receives comment-stripped file content, not the raw diff
    │  Asks for multiple-choice distractors only when instructor_repo_token is
    │  set — nothing else consumes them, so without it the model is asked for
    │  the correct answer alone
    │
callAI()
    │  POSTs to the provider's chat completions endpoint
    │  Returns the model's response text plus response metadata
    │  (finish reason, token usage, attempts, duration)
    │
formatReport(pdfUrl: null)   ← base report (PDF source)
    │
generatePdf()
    │  Converts base report Markdown → PDF Buffer via md-to-pdf + system Chromium
    │
uploadPdfAsset()
    │  Creates/reuses the gmc-assessments rolling release
    │  Replaces the existing PDF asset for this branch (stable URL)
    │  Returns browser_download_url → pdfUrl
    │
formatReport(pdfUrl)    ← issue body (base + PDF download link)
    │
        sets action outputs
        (pdf_url, questions, code_before_strip, code_after_strip)
               │
          postIssue()
               │  Creates or updates the assessment issue (unconditional)
               │  Returns { number, url }
               │
        sets action outputs (issue_url, issue_number)
               │
     deliverToInstructorRepo()   ← only when instructor_repo_token is set
               │  Runs on a separate Octokit built from the instructor PAT
               │
               ├── ensureInstructorRepo()
               │     Creates {assignment}-grillmycode-instructor (private) if absent,
               │     then polls until auto_init's first commit lands
               │
               ├── syncInstructorRepoFiles()
               │     Rewrites generate-lms-quiz.yml and README.md when they differ
               │     from the copies shipped in src/; warns (never throws) on failure.
               │
               ├── writeFileWithRetry()
               │     Writes {studentLogin}/raw-ai-output.md — the model's reply
               │     before postprocessing, under a provenance header recording
               │     the request settings and response metadata; warns (never
               │     throws) on failure
               │
               └── writeFileWithRetry()
                     Writes {studentLogin}/questions.md, retrying on 409/422
                     conflicts and backing off on rate limits
               │
     applyRepoLabels()   ← only when label_repos is "true"
               │  Marks the STUDENT repository, on the instructor PAT (repository
               │  metadata is unreachable with GITHUB_TOKEN — the permissions key
               │  has no administration scope). Runs last and never throws: the
               │  student already has their questions by this point
               │
               ├── getAllTopics() → replaceAllTopics()
               │     Reads first and writes back the union: the endpoint replaces
               │     the whole topic set, so a blind write would delete the
               │     instructor's own topics. Skipped when already present
               │
               └── repos.get() → repos.update()
                     Strips any label from an earlier run before appending the
                     current one, so repeated pushes leave one accurate label.
                     Leaves an over-long description untouched

---

Key modules

readInputs()

Reads and normalizes every INPUT_* environment variable. Responsible for:

• Parsing comma-separated glob lists into arrays
• Parsing additional_exclude_patterns into an array (stack-based patterns are resolved separately in stack-detection.js at runtime)
• Clamping num_questions to a minimum of 1 and a maximum of 50; a workflow warning is emitted if the supplied value exceeds 50
• Splitting assignment_context into a assignmentContextGlobs array for later file resolution

resolveSHAs(ctx, octokit, inputs)

Determines the base and head SHAs for the diff. Handles two event types:

Event	Base SHA	Head SHA
push	previous SHA (or first commit on new branch)	after SHA
everything else (workflow_dispatch, etc.)	first commit	ctx.sha

After event-specific resolution, include_initial_commit can override the base SHA to pin it to the repository's very first commit — the behaviour needed for Classroom 50 to exclude starter template files.

On a run started by a submission tag ({ tagName } passed as the fourth argument), the head is the tagged commit, peeled with git rev-parse <sha>^{commit} because an annotated tag's push names the tag object. With tag_diff_base: previous-tag, the base then moves to the nearest strict ancestor carrying a tag that matches any submission_tags pattern (pickPreviousSubmissionTag in src/tags.js); with none, the base above stands. With tag_diff_base: tag:<name>, the base moves to that tag instead (resolveTagCommit in src/git.js, looked up under refs/tags/), and the run throws if the tag is missing, is not an ancestor of the head, or is on the head itself; it is skipped when base_sha is set. The chosen tag is returned as previousTag.

Manual base_sha / head_sha inputs always take precedence over all of the above.

reportEmptyAssessment({ reason, baseSha, headSha, allFiles, excludePatterns, inputs })

Reports a run that produced no assessment, and decides whether that ends the run as a success or a failure.

Two distinct conditions reach this point and need different fixes, so each gets its own message and its own "what to check" list rather than a shared one:

reason	Condition	Points at
empty-range	git diff --name-only returned nothing — base and head resolved to the same commit	include_initial_commit, or a base_sha/head_sha override
fully-excluded	Files did change, but every one was removed by the exclude patterns	exclude_pattern_overrides, and the applied pattern list

Both write a core.summary block, because a warning annotation shows on the run page but not in a list of runs — an instructor scanning a cohort would otherwise see an unbroken row of green ticks. Summary writing is wrapped in try/catch: a summary is a convenience, never a reason to lose the diagnosis.

Failing is opt-in via fail_on_empty_assessment (default false). Both conditions occur normally the moment an assignment is accepted — a template repository's only commit is its starter code, and a Classroom 50 setup commit contains only the excluded .classroom50.yaml — so failing by default would mark every student repository red at creation.

Explanations that only hold in specific circumstances are emitted conditionally: the accept-time note appears for empty-range only when the initial commit is being excluded and no SHA override is set, and for fully-excluded only when the excluded set is exactly .classroom50.yaml. The excluded file list is capped at EMPTY_ASSESSMENT_FILE_LIST_LIMIT entries, since an excluded tree can run to hundreds of paths.

sanitiseSha(sha)

Validates that a SHA is 4–64 hex characters before passing it to a git command. This prevents shell injection through crafted base_sha/head_sha inputs.

safeFilePart(str)

Returns a filesystem-safe version of a string for use in filenames. Special characters are replaced with hyphens; consecutive hyphens are collapsed; leading and trailing hyphens are stripped. Used to derive the PDF asset filename from the repository name (e.g. grill-my-code-assignment-1-jsmith.pdf), and — through tagGroupSlug() — a tag group's PDF suffix and instructor-repository folder from its submission_tags pattern (submit/* → submit).

callAI({ provider, model, apiKey, messages, retryMaxAttempts, temperature })

A thin provider abstraction over the OpenAI-compatible chat completions API. Each provider maps to a base URL and authentication header:

Provider	URL	Auth header
openrouter	openrouter.ai/api/v1/chat/completions	Authorization: Bearer <api_key>

openrouter is currently the only supported provider. The switch in src/ai.js is retained as the extension point for adding others — see Contributing. Any provider added there uses the same request body shape (model, messages, temperature, top_p). max_tokens is deliberately omitted: on OpenRouter it restricts routing to providers that support a response of that length, and each model's own output limit is left to apply instead.

Transient failures are retried automatically up to retryMaxAttempts total attempts using exponential backoff with full jitter. The following status codes are retried: 429, 500, 502, 503, 504. Network-level failures (e.g. DNS, socket errors) are also retried. A 429 response that includes a Retry-After header has that delay honoured in preference to the calculated backoff, capped at the same 30-second AI_RETRY_MAX_DELAY_MS as every other wait so a long value cannot stall the run. Once generation has started OpenRouter can no longer change the HTTP status, so an upstream failure arrives as a 200 with an { error: { code, message } } body; that is retried when error.code is one of the same retryable codes, and otherwise fails with the provider's message. A 200 whose body is not valid JSON (a dropped connection or a proxy error page) is retried like a network failure, as is a response whose first choice carries no text content. A core.warning() is logged before each retry, showing the attempt number, status code, and delay.

callAI returns { content, metadata }. content is the trimmed reply; metadata carries the finish_reason (and the upstream provider's native reason), token usage, the number of attempts spent, wall time across all of them, and the generation id, model and host OpenRouter reports serving. None of it affects the run: it is written into the provenance header of raw-ai-output.md, where a finish_reason of length is the only way to tell a reply cut off at the output token limit from one that simply held fewer questions.

postIssue()

Uses an update-first strategy:

1. List open assessment issues whose title exactly matches this branch's (GrillMyCode Questions (<branch>), or GrillMyCode Questions when no branch is known) — or, on a tag run, this tag group's (GrillMyCode Questions (tag: <pattern>))
2. If one exists, update its title and body in-place (preserving issue number, URL, and comment history). Extra duplicates are deleted via the deleteIssue GraphQL mutation (non-fatal — needs admin rights, so a refused delete warns and leaves the duplicate in place).
3. If none exists, create a fresh issue, then pin it via the pinIssue GraphQL mutation (non-fatal — silently warns if the 3-issue pin limit is already reached).

Returns { number, url } for use by action outputs.

deliverToInstructorRepo()

Writes the instructor copy of the assessment — questions and answers, regardless of include_answers — to a private repository shared by the whole class. It runs only when instructor_repo_token is set, and every call in the module uses an Octokit built from that PAT; the student's github_token is never used here.

Three concerns are layered inside it:

1. Repository lifecycle. ensureInstructorRepo() creates the repository private via repos.createInOrg when repos.get 404s — so the owner must be an organization — and then polls for the auto_init commit so the first write has a branch to land on. A 422 from a racing run that created it first is treated as success.
2. File ownership. syncInstructorRepoFiles() brings .github/workflows/generate-lms-quiz.yml and README.md into line with the copies in src/workflows/ and src/templates/ on every delivery, not just at creation, committing only when the bytes differ. This is how a repository created by an earlier release picks up quiz-generation fixes. Each file is guarded independently and failures are warnings, never throws: the workflow write needs the PAT's workflow scope, and a token lacking it must not cost the student their assessment.
3. Concurrent writes. writeFileWithRetry() re-fetches the blob SHA and retries on the 409s and stale-SHA 422s that a class pushing at once produces, and distinguishes genuine rate limits (Retry-After, exhausted x-ratelimit-remaining, GitHub's limit wording) from a 403 raised by a missing scope, which fails fast rather than sitting through the full backoff budget.

On a tag run a fourth concern applies: the submission record. Before the instructor report is built, main.js calls readSubmissionHistory() for {student}/{tagGroup}/, and the pure helpers in src/submission-history.js turn its submissions.md rows into this run's row and a resubmission note for the report header. deliverToInstructorRepo({ submission }) then archives the questions.md being replaced to history/<#>-questions.md, rewrites submissions.md with the new row, and only then writes questions.md — so the quiz workflow's trigger is still the last commit. Like the raw-output copy, a failure in the record warns and never costs the assessment. A tag push always counts as a submission; a manual run counts only when the triggering actor is the student (always, in a team repo).

A failure that escapes all of this is caught in main.js and reported with core.error — an annotation that does not fail the job, since the student-facing assessment has already been delivered by that point.

---

Security considerations

• Shell injection prevention: all git calls use spawnSync with an explicit argument array — no shell string interpolation. SHAs are validated with sanitiseSha() before use.
• Secret masking: the external API key is registered with core.setSecret() before any API call, preventing it from appearing in workflow logs.
• Minimal permissions: the action only requests the permissions it needs for the chosen delivery method.
• Token separation: the instructor PAT is used only by src/delivery/instructor-repo.js and src/repo-labels.js, each through its own Octokit instance. It is never passed to the student-facing delivery paths, and the student's GITHUB_TOKEN is never given access to the instructor repository — which is what keeps the answer key out of reach of anyone who can read the student's repository or its workflow logs. src/repo-labels.js writes only repository metadata (topics, description) on the student's own repository and reads nothing from the instructor repository, so sharing the PAT widens what the token is used for without widening what a student can see.

---

Docker image build

The image uses a single-stage Dockerfile based on node:26-slim:

node:26-slim
      │
      ├── apt-get install git curl ca-certificates chromium
      │   (chromium provides the system browser used by md-to-pdf for PDF generation)
      │
      ├── ENV PUPPETEER_SKIP_DOWNLOAD=true
      │   ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium
      │
      ├── curl ──► download pre-built rmcm binary
      │            from GitHub Releases → /usr/local/bin/rmcm
      │
      ├── npm install -g corepack
      │   corepack enable
      │   ENV COREPACK_ENABLE_DOWNLOAD_PROMPT=0
      │
      ├── COPY package.json pnpm-lock.yaml
      │   pnpm install --frozen-lockfile --prod --ignore-scripts
      │   (--ignore-scripts prevents Puppeteer's postinstall Chromium download;
      │    PUPPETEER_SKIP_DOWNLOAD is belt-and-braces)
      │
      └── COPY src/ entrypoint.sh

rmcm (the comment-stripping binary from NSCC-ITC-Assessment/comment-remover) is downloaded as a pre-built Linux x86_64 binary from the grill-my-code GitHub release. pnpm is bootstrapped via corepack, which resolves the exact version from the packageManager field in package.json when pnpm install runs, so the image, CI and the devcontainer all use the same pnpm.

---

CI/CD workflows

Three workflows build and publish Docker images. They are mutually exclusive by trigger.

Workflow	Trigger	Image tag(s) produced	Intended for
branch-build.yml	Push to any non-main branch (code changes only); workflow_dispatch	branch-<sanitized-branch-name>	Contributors — ephemeral dev image for testing a feature or fix branch before it is merged
staging-build.yml	Push to main (code changes only); workflow_dispatch	next	Maintainers — bleeding-edge integration build; reflects the current state of main but is not recommended for consumers
release.yml	Push of a v* tag	vX.Y.Z, vX.Y, vX, latest	Consumers — stable, versioned release; consumers pin to the major tag (e.g. :v0)

All three workflows ignore documentation-only changes (docs-site/**, README.md, etc.) to avoid unnecessary image rebuilds.

The canonical tag in action.yml is the major tag (e.g. :v0). The action-image-tag PR check enforces this and will fail if the tag has been changed manually on a branch.

See Versioning & Releases for the full release process.