Architecture

How the action runs

This is a Docker-based GitHub Action — rather than executing JavaScript directly on the runner, GitHub pulls a pre-built Docker image and runs the assessment script inside it. The image is published to the GitHub Container Registry (ghcr.io) and pinned in action.yml.

Consumer workflow
      │
      ▼
action.yml  ──► pulls Docker image from ghcr.io
                      │
                      ▼
              entrypoint.sh  ──► cd $GITHUB_WORKSPACE
                                       │
                                       ▼
                                 node src/main.js

Using Docker means:

• The Node version, git binary, and all dependencies are fixed and identical across every runner — no version drift
• The image is built once and reused; consumer repos pay no build cost at runtime
• The pre-built image reference in action.yml is updated automatically by the release workflow each time a version tag is pushed

---

Execution flow

When main.js runs, it follows this sequence:

readInputs()
    │  Reads all INPUT_* environment variables set by action.yml
    │
resolveSHAs()
    │  Determines baseSha and headSha from the event context
    │  Handles: push, workflow_dispatch
    │  Applies include_initial_commit override when enabled
    │
resolveBranch()
    │  Extracts the branch name from the event payload or GITHUB_REF
    │
resolveSubmissionIdentity()
    │  Lists the repository's direct collaborators (one API call)
    │  Student = the collaborator whose login ends the Classroom 50 repo name
    │  (<classroom>-<assignment>-<username>); assignment = everything before it
    │  A repo ending in -group-<n> is a team submission, filed under group-<n>
    │  Anything else is unresolved, and instructor delivery is skipped
    │  Never consults the event sender, run actor, commit authors or template
    │
getChangedFiles() → filterFiles()
    │  Runs `git diff --name-only baseSha headSha`
    │  Applies auto-detected stack patterns, additional_exclude_patterns, and exclude_pattern_overrides via minimatch
    │
    ├─ no files survive → reportEmptyAssessment() and return
    │     Distinguishes an empty commit range from a fully excluded file list,
    │     writes a job summary naming the reason, then warns — or fails when
    │     fail_on_empty_assessment is enabled
    │
getDiff()
    │  Runs `git diff baseSha headSha -- <files>`
    │  Result kept as a fallback only — not sent to the AI directly
    │
collectRawFiles()
    │  Fetches full file content at headSha via `git show`
    │  (deleted files are silently skipped)
    │
stripCommentsFromFiles()
    │  Writes each file to /tmp, runs the rmcm binary on it
    │  Falls back silently to original content for unsupported types
    │  Falls back to raw diff if stripping produces no output at all
    │
buildCodeContent()
    │  Formats stripped files as fenced Markdown code blocks
    │
readAssignmentContextFiles()
    │  Reads files from GITHUB_WORKSPACE that match assignment_context globs
    │  Concatenates contents as headed sections; capped at assignment_context_max_chars input (default 20000)
    │  Returns an empty string when no globs are supplied or no files match
    │
buildPrompt()
    │  Constructs the system + user messages for the AI
    │  Injects assignment context (file contents) then instructor instructions
    │  AI receives comment-stripped file content, not the raw diff
    │
callAI()
    │  POSTs to the provider's chat completions endpoint
    │  Returns the model's response text
    │
formatReport(pdfUrl: null)   ← base report (PDF source)
    │
generatePdf()
    │  Converts base report Markdown → PDF Buffer via md-to-pdf + system Chromium
    │
uploadPdfAsset()
    │  Creates/reuses the gmc-assessments rolling release
    │  Replaces the existing PDF asset for this branch (stable URL)
    │  Returns browser_download_url → pdfUrl
    │
formatReport(pdfUrl)    ← issue body (base + PDF download link)
    │
        sets action outputs
        (pdf_url, questions, code_before_strip, code_after_strip)
               │
          postIssue()
               │  Creates or updates the assessment issue (unconditional)
               │  Returns { number, url }
               │
        sets action outputs (issue_url, issue_number)
               │
     deliverToInstructorRepo()   ← only when instructor_repo_token is set
               │  Runs on a separate Octokit built from the instructor PAT
               │
               ├── ensureInstructorRepo()
               │     Creates {assignment}-grillmycode-instructor (private) if absent,
               │     then polls until auto_init's first commit lands
               │
               ├── syncInstructorRepoFiles()
               │     Rewrites generate-lms-quiz.yml and README.md when they differ
               │     from the copies shipped in src/; warns (never throws) on failure
               │
               └── writeFileWithRetry()
                     Writes {studentLogin}/questions.md, retrying on 409/422
                     conflicts and backing off on rate limits

---

Key modules

readInputs()

Reads and normalises every INPUT_* environment variable. Responsible for:

• Parsing comma-separated glob lists into arrays
• Parsing additional_exclude_patterns into an array (stack-based patterns are resolved separately in stack-detection.js at runtime)
• Clamping num_questions to a minimum of 1 and a maximum of 50; a workflow warning is emitted if the supplied value exceeds 50
• Splitting assignment_context into a assignmentContextGlobs array for later file resolution

resolveSHAs(ctx, octokit, inputs)

Determines the base and head SHAs for the diff. Handles two event types:

Event	Base SHA	Head SHA
push	previous SHA (or first commit on new branch)	after SHA
everything else (workflow_dispatch, etc.)	first commit	ctx.sha

After event-specific resolution, include_initial_commit can override the base SHA to pin it to the repository's very first commit — the behaviour needed for Classroom 50 to exclude starter template files.

Manual base_sha / head_sha inputs always take precedence over all of the above.

reportEmptyAssessment({ reason, baseSha, headSha, allFiles, excludePatterns, inputs })

Reports a run that produced no assessment, and decides whether that ends the run as a success or a failure.

Two distinct conditions reach this point and need different fixes, so each gets its own message and its own "what to check" list rather than a shared one:

reason	Condition	Points at
empty-range	git diff --name-only returned nothing — base and head resolved to the same commit	include_initial_commit, or a base_sha/head_sha override
fully-excluded	Files did change, but every one was removed by the exclude patterns	exclude_pattern_overrides, and the applied pattern list

Both write a core.summary block, because a warning annotation shows on the run page but not in a list of runs — an instructor scanning a cohort would otherwise see an unbroken row of green ticks. Summary writing is wrapped in try/catch: a summary is a convenience, never a reason to lose the diagnosis.

Failing is opt-in via fail_on_empty_assessment (default false). Both conditions occur normally the moment an assignment is accepted — a template repository's only commit is its starter code, and a Classroom 50 setup commit contains only the excluded .classroom50.yaml — so failing by default would mark every student repository red at creation.

Explanations that only hold in specific circumstances are emitted conditionally: the accept-time note appears for empty-range only when the initial commit is being excluded and no SHA override is set, and for fully-excluded only when the excluded set is exactly .classroom50.yaml. The excluded file list is capped at EMPTY_ASSESSMENT_FILE_LIST_LIMIT entries, since an excluded tree can run to hundreds of paths.

sanitiseSha(sha)

Validates that a SHA is 4–64 hex characters before passing it to a git command. This prevents shell injection through crafted base_sha/head_sha inputs.

safeFilePart(str)

Returns a filesystem-safe version of a string for use in filenames. Special characters are replaced with hyphens; consecutive hyphens are collapsed; leading and trailing hyphens are stripped. Used to derive the PDF asset filename from the repository name (e.g. grill-my-code-assignment-1-jsmith.pdf).

callAI({ provider, model, apiKey, messages, retryMaxAttempts })

A thin provider abstraction over the OpenAI-compatible chat completions API. Each provider maps to a base URL and authentication header:

Provider	URL	Auth header
openrouter	openrouter.ai/api/v1/chat/completions	Authorization: Bearer <api_key>

openrouter is currently the only supported provider. The switch in src/ai.js is retained as the extension point for adding others — see Contributing. Any provider added there uses the same request body shape (model, messages, temperature, top_p). max_tokens is deliberately omitted: on OpenRouter it restricts routing to providers that support a response of that length, and each model's own output limit is left to apply instead.

Transient failures are retried automatically up to retryMaxAttempts total attempts using exponential backoff with full jitter. The following status codes are retried: 429, 500, 502, 503, 504. Network-level failures (e.g. DNS, socket errors) are also retried. A 429 response that includes a Retry-After header has that delay honoured in preference to the calculated backoff, capped at the same 30-second AI_RETRY_MAX_DELAY_MS as every other wait so a long value cannot stall the run. Once generation has started OpenRouter can no longer change the HTTP status, so an upstream failure arrives as a 200 with an { error: { code, message } } body; that is retried when error.code is one of the same retryable codes, and otherwise fails with the provider's message. A 200 whose body is not valid JSON (a dropped connection or a proxy error page) is retried like a network failure, as is a response whose first choice carries no text content. A core.warning() is logged before each retry, showing the attempt number, status code, and delay.

postIssue()

Uses an update-first strategy:

1. List open assessment issues whose title exactly matches this branch's (GrillMyCode Questions (<branch>), or GrillMyCode Questions when no branch is known)
2. If one exists, update its title and body in-place (preserving issue number, URL, and comment history). Extra duplicates are deleted via the deleteIssue GraphQL mutation (non-fatal — needs admin rights, so a refused delete warns and leaves the duplicate in place).
3. If none exists, create a fresh issue, then pin it via the pinIssue GraphQL mutation (non-fatal — silently warns if the 3-issue pin limit is already reached).

Returns { number, url } for use by action outputs.

deliverToInstructorRepo()

Writes the instructor copy of the assessment — questions and answers, regardless of include_answers — to a private repository shared by the whole class. It runs only when instructor_repo_token is set, and every call in the module uses an Octokit built from that PAT; the student's github_token is never used here.

Three concerns are layered inside it:

1. Repository lifecycle. ensureInstructorRepo() creates the repository private via repos.createInOrg when repos.get 404s — so the owner must be an organisation — and then polls for the auto_init commit so the first write has a branch to land on. A 422 from a racing run that created it first is treated as success.
2. File ownership. syncInstructorRepoFiles() brings .github/workflows/generate-lms-quiz.yml and README.md into line with the copies in src/workflows/ and src/templates/ on every delivery, not just at creation, committing only when the bytes differ. This is how a repository created by an earlier release picks up quiz-generation fixes. Each file is guarded independently and failures are warnings, never throws: the workflow write needs the PAT's workflow scope, and a token lacking it must not cost the student their assessment.
3. Concurrent writes. writeFileWithRetry() re-fetches the blob SHA and retries on the 409s and stale-SHA 422s that a class pushing at once produces, and distinguishes genuine rate limits (Retry-After, exhausted x-ratelimit-remaining, GitHub's limit wording) from a 403 raised by a missing scope, which fails fast rather than sitting through the full backoff budget.

A failure that escapes all of this is caught in main.js and reported with core.error — an annotation that does not fail the job, since the student-facing assessment has already been delivered by that point.

---

Security considerations

• Shell injection prevention: all git calls use spawnSync with an explicit argument array — no shell string interpolation. SHAs are validated with sanitiseSha() before use.
• Secret masking: the external API key is registered with core.setSecret() before any API call, preventing it from appearing in workflow logs.
• Minimal permissions: the action only requests the permissions it needs for the chosen delivery method.
• Token separation: the instructor PAT is used exclusively by src/delivery/instructor-repo.js, through its own Octokit instance. It is never passed to the student-facing delivery paths, and the student's GITHUB_TOKEN is never given access to the instructor repository — which is what keeps the answer key out of reach of anyone who can read the student's repository or its workflow logs.

---

Docker image build

The image uses a single-stage Dockerfile based on node:26-slim:

node:26-slim
      │
      ├── apt-get install git curl ca-certificates chromium
      │   (chromium provides the system browser used by md-to-pdf for PDF generation)
      │
      ├── ENV PUPPETEER_SKIP_DOWNLOAD=true
      │   ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium
      │
      ├── curl ──► download pre-built rmcm binary
      │            from GitHub Releases → /usr/local/bin/rmcm
      │
      ├── npm install -g corepack
      │   corepack enable
      │   ENV COREPACK_ENABLE_DOWNLOAD_PROMPT=0
      │
      ├── COPY package.json pnpm-lock.yaml
      │   pnpm install --frozen-lockfile --prod --ignore-scripts
      │   (--ignore-scripts prevents Puppeteer's postinstall Chromium download;
      │    PUPPETEER_SKIP_DOWNLOAD is belt-and-braces)
      │
      └── COPY src/ entrypoint.sh

rmcm (the comment-stripping binary from NSCC-ITC-Assessment/comment-remover) is downloaded as a pre-built Linux x86_64 binary from the grill-my-code GitHub release. pnpm is bootstrapped via corepack, which resolves the exact version from the packageManager field in package.json when pnpm install runs, so the image, CI and the devcontainer all use the same pnpm.

---

CI/CD workflows

Three workflows build and publish Docker images. They are mutually exclusive by trigger.

Workflow	Trigger	Image tag(s) produced	Intended for
branch-build.yml	Push to any non-main branch (code changes only); workflow_dispatch	branch-<sanitized-branch-name>	Contributors — ephemeral dev image for testing a feature or fix branch before it is merged
staging-build.yml	Push to main (code changes only); workflow_dispatch	next	Maintainers — bleeding-edge integration build; reflects the current state of main but is not recommended for consumers
release.yml	Push of a v* tag	vX.Y.Z, vX.Y, vX, latest	Consumers — stable, versioned release; consumers pin to the major tag (e.g. :v1)

All three workflows ignore documentation-only changes (docs-site/**, README.md, etc.) to avoid unnecessary image rebuilds.

The canonical tag in action.yml is the major tag (e.g. :v1). The action-image-tag PR check enforces this and will fail if the tag has been changed manually on a branch.

See Versioning & Releases for the full release process.