Skip to main content
Version: Next (unreleased)

FAQ

Short answers, each with a link to the full story. For something that isn't working, see Troubleshooting.

About GrillMyCode​

What is GrillMyCode?​

A tool that runs in each student's GitHub repository and uses AI to write questions about the code that student wrote. You use the questions for a short conversation or written check, often called a code viva. See How GrillMyCode works.

Does it grade the code?​

No, and that's deliberate. It writes questions and leaves the judgment to you. A slightly off-target question is easy to skip; an unfair automatic grade isn't. See Why GrillMyCode?

Do I need to install anything?​

No. It runs on GitHub's servers. You add one workflow file to the assignment's template repository, and the Workflow Wizard writes that file for you.

Do I need to know Git and GitHub?​

Yes, the basics: committing and pushing, the default branch, and finding your way around a repository on GitHub. If you use submission tags, you'll also need to create and push tags. You don't need to know GitHub Actions already. See What you need.

Do my students need to do anything?​

No setup at all. If you use submission tags, they need to know the tag commands. What your students see has wording you can copy into your assignment instructions.

Does it work without Classroom 50?​

Mostly. The questions, issue and PDF work in any GitHub repository. The private answer key, and the LMS quizzes and resubmission tracking that come with it, need Classroom 50's repository naming. See Using GrillMyCode with Classroom 50.

Cost and setup​

What does it cost?​

GrillMyCode itself is free. The AI is paid through OpenRouter from a prepaid balance. An assessment can cost less than one cent, and with a low-cost model a small budget can last a large class a semester. The cost can rise sharply with the model you choose, its reasoning settings and how much code is assessed. Working out your class's cost is your responsibility, so do trial runs before rolling it out.

What secrets do I need?​

One, OPENROUTER_API_KEY, saved once as an organization secret. The private answer key, which works only with Classroom 50, needs a second one, INSTRUCTOR_REPO_TOKEN. See Tokens, secrets and permissions.

Can I change the settings after setting it up?​

Yes. Edit the workflow file, or run the Wizard again and replace the file. To change a setting for one run only, see Running it yourself.

Can I stop GrillMyCode changing partway through an assignment?​

Yes. A workflow normally follows the newest release, but it can name one minor version and stay on it, at the cost of new features and later fixes. See Choosing when to upgrade.

Questions​

How many questions are generated?​

20 by default, and anywhere from 1 to 50. See Tailoring the questions.

How do I focus the questions on my assignment?​

Write a few sentences of instructions, and optionally point GrillMyCode at the assignment brief or rubric. See Tailoring the questions.

Can students see the answers?​

No, unless you turn on Include answers, which defeats the purpose. To see the answers yourself, set up the private answer key (Classroom 50 only).

Why are some questions missing from a student's report?​

GrillMyCode holds back any question it can't be sure doesn't give away its answer, and says how many. See Troubleshooting.

Why doesn't the AI see the code comments?​

They're removed on purpose, so questions are about what the code does. You can keep them; see Tailoring the questions.

Is my starter code assessed?​

Not by default. Anything in the template when the student accepted is left out, and if a student edits one of your files, only their added or changed lines are asked about. You can choose to have a few questions asked about your starter code; see Tailoring the questions.

Can the AI see the rest of the project for background?​

Yes. Your starter code is sent if you choose to, and the student's earlier work is sent by default when only their latest submission is assessed. The AI doesn't ask about that code on its own unless you let it ask about your starter code. See Tailoring the questions.

When it runs​

Which trigger should I use?​

Every push suits short assignments and practice. A submission tag suits finished or staged work. Manual only suits you choosing the timing. See Choosing a trigger.

Can students trigger the assessment only when they're done?​

Yes, with a submission tag: the student pushes a tag you named, such as complete. See Choosing a trigger.

Does gh student submit run GrillMyCode?​

With the every-push trigger, yes, like any push. With a submission tag, no: students must also push your tag. See Using GrillMyCode with Classroom 50.

What happens if a student pushes again while a run is in progress?​

The older run is cancelled and a new one starts on the latest commit, so only the latest code is assessed. See Triggers in depth.

Can I tell when a student resubmits under the same tag?​

Yes, with the private answer key set up (Classroom 50 only). Every resubmission is counted, and the replaced questions are kept. See Tracking assessed repositories.

Results​

Where are the questions?​

In a GitHub issue in the student's repository, called GrillMyCode Questions, with a PDF linked from the top. See What your students see.

Can students read the questions in their editor?​

Yes, in VS Code and JetBrains IDEs, with the optional GrillMyCode Companion extension or plugin. It shows each question beside the lines it asks about. See Showing questions in VS Code and Showing questions in JetBrains IDEs.

Can I get the questions into my LMS?​

Yes, for Classroom 50 assignments. With the private answer key, each student gets a quiz file you can import into Brightspace, Canvas, Moodle and most others. See Importing quizzes into your LMS.

How can I see which students have questions?​

Search your organization for open assessment issues, or turn on repository labels, which the Workflow Wizard does by default. See Tracking assessed repositories.