Skip to main content
Version: v0 (current)

FAQ

Short answers, each with a link to the full story. For something that isn't working, see Troubleshooting.

About GrillMyCode​

What is GrillMyCode?​

A tool that runs in each student's GitHub repository and uses AI to write questions about the code that student wrote. You use the questions for a short conversation or written check, often called a code viva. See How it works.

Does it grade the code?​

No, and that's deliberate. It writes questions and leaves the judgment to you. A slightly off-target question is easy to skip; an unfair automatic grade isn't. See Why GrillMyCode?

Do I need to install anything?​

No. It runs on GitHub's servers. You add one workflow file to the assignment's template repository, and the Workflow Wizard writes that file for you.

Do my students need to do anything?​

No setup at all. If you use submission tags, they need to know the tag commands. What your students see has wording you can copy into your assignment instructions.

Does it work without Classroom 50?​

Mostly. The questions, issue and PDF work in any GitHub repository. The private answer key, and the LMS quizzes and resubmission tracking that come with it, need Classroom 50's repository naming. See Using GrillMyCode with Classroom 50.

Cost and setup​

What does it cost?​

GrillMyCode itself is free. The AI is paid through OpenRouter from a prepaid balance. With the recommended models, an assessment usually costs less than one cent, and $5 US typically lasts a large class a semester. See Choosing a model and managing cost.

What secrets do I need?​

One, OPENROUTER_API_KEY, saved once as an organization secret. The private answer key needs a second one, INSTRUCTOR_REPO_TOKEN. See Tokens, secrets and permissions.

Can I change the settings after setting it up?​

Yes. Edit the workflow file, or run the Wizard again and replace the file. To change a setting for one run only, see Running it yourself.

Questions​

How many questions are generated?​

20 by default, and anywhere from 1 to 50. See Tailoring the questions.

How do I focus the questions on my assignment?​

Write a few sentences of instructions, and optionally point GrillMyCode at the assignment brief or rubric. See Tailoring the questions.

Can students see the answers?​

No, unless you turn on Include answers, which defeats the purpose. To see the answers yourself, set up the private answer key.

Why are some questions missing from a student's report?​

GrillMyCode holds back any question it can't be sure doesn't give away its answer, and says how many. See Troubleshooting.

Why doesn't the AI see the code comments?​

They're removed on purpose, so questions are about what the code does. You can keep them; see Tailoring the questions.

Is my starter code assessed?​

No. Anything in the template when the student accepted is left out. See Choosing which files are assessed.

When it runs​

Which trigger should I use?​

Every push suits short assignments and practice. A submission tag suits finished or staged work. Manual only suits you choosing the timing. See Choosing a trigger.

Can students trigger the assessment only when they're done?​

Yes, with a submission tag: the student pushes a tag you named, such as complete. See Choosing a trigger.

Does gh student submit run GrillMyCode?​

With the every-push trigger, yes, like any push. With a submission tag, no: students must also push your tag. See Using GrillMyCode with Classroom 50.

What happens if a student pushes again while a run is in progress?​

The older run is cancelled and a new one starts on the latest commit, so only the latest code is assessed. See Triggers in depth.

Can I tell when a student resubmits under the same tag?​

Yes, with the private answer key set up. Every resubmission is counted, and the replaced questions are kept. See Tracking assessed repositories.

Results​

Where are the questions?​

In a GitHub issue in the student's repository, called GrillMyCode Questions, with a PDF linked from the top. See What your students see.

Can I get the questions into my LMS?​

Yes. With the private answer key, each student gets a quiz file you can import into Brightspace, Canvas, Moodle and most others. See Importing quizzes into your LMS.

How can I see which students have questions?​

Search your organization for open assessment issues, or turn on repository labels, which the Workflow Wizard does by default. See Tracking assessed repositories.

Upgrading​

I've used GitHub Models with GrillMyCode in the past and now they no longer function. Why?​

GitHub permanently discontinued GitHub Models, so there's nothing to reconnect to. Switch the workflow to OpenRouter; Upgrade notes has the steps.