Engineering Flight Simulator Instructor
Running system-design interviews for a class

From a list of names to twenty graded interviews in an afternoon

You pick the scenario and the engineer level, create a session per challenger, send links, then watch replays and grade. This guide follows that order and explains every setting you will meet on the way.

01

Get into the instructor dashboard

Where: /instructor

There are two ways in, depending on how the server is run.

ServerHow you sign inWho sets it up
Hosted (the class URL)An admin creates an instructor account for your email. You get a set-password email, choose a password, and sign in at /login. The dashboard link is in the menu bar.The admin (see the admin guide)
Local or classroom LANThe dashboard asks for the instructor password. It is the INSTRUCTOR_PASSWORD value the server was started with.Whoever runs the server

First visit shows a one-question welcome: the default engineer level. Pick senior unless your cohort is early-career; you can change it per session and later in Settings.

You then land on Overview: four counters (sessions, submitted, in progress, not started), a 14-day activity chart, sessions by scenario and by level, a Needs attention list of submissions waiting for review, and the most recent sessions. Every row opens its replay.

02

Understand what an interview scenario contains

Where: Settings → Scenario library, and sim/scenarios/iv_*/scenario.yaml if you are curious

Every iv_* scenario has the same moving parts. Knowing them makes the replays much easier to read.

PersonaThe interviewerPosts a one-line brief at session start and answers only what is asked, with facts, never architecture. Holds hidden constraints behind a reveal ladder: each rung unlocks when the challenger asks the right kind of question.
PersonaThe assessorSilent until the challenger submits DESIGN.md. Then asks one probing question at a time about a decision the challenger wrote. Never proposes a better design.
StarterDESIGN.md and TICKETS.mdA templated design document (problem, questions, requirements, capacity, API, architecture, data model, trade-offs) and an empty tickets file, edited in the browser.
RubricFour criteria plus extra creditDiscovery 2.0, scoping 2.0, stakeholders 1.0, communication 2.0. Tickets are graded only when you turn the extra-credit toggle on.
LevelEngineer levelIntern through distinguished. It changes persona posture (how much is volunteered, how hard the assessor pushes) and the grader's expectation.
TimeboxAbout 20 minutesStated in the brief. The simulator does not enforce it; you do, or the cohort schedule does.
Interview scenarios do not need GitHub or GitLabChallengers write and submit inside the browser on every deployment. The starter-URL field in the scenario library is only for build scenarios (the product ones), where hosted learners fork a repo. Leave it blank for iv_*.
03

Create one session and send the link

Where: New session
  1. Scenario. Pick an interview, for example Interview: parking garage occupancy. Disabled scenarios do not appear.
  2. Engineer level. Defaults to your setting. Match it to the challenger, not to the problem.
  3. Assign to challenger (email, optional). On a hosted server this is what lets that person, and only that person, open the link from their dashboard. Leave it blank on a local server, or to hand out an unassigned link that the first signed-in challenger claims.
  4. Press Create session & get link. Copy the trainee link and send it however you like. Open replay for this session takes you straight to the live view.
Assign by email on hosted serversAn assigned session shows up under Your sessions on the challenger's dashboard, so they never need the raw link. It also stops one person opening another's interview.
04

Create sessions for a whole cohort at once

Where: Cohort

Cohorts are groups of challenger accounts that an admin sets up (or that you ask them to create from a pasted class list). With one in place:

  1. Choose the Cohort, the Scenario, and the Engineer level.
  2. Press Create sessions. A progress bar counts the sessions as they are created (a few at a time), so a class of forty takes a visible half-minute rather than looking stuck. If the connection drops part-way, press the button again: members who already have the scenario are skipped.
  3. Use Copy all links for a message to the group, or Download CSV for your own records. Each row also has its own Copy link and Replay.

After the interviews, Results for this scenario on the same tab shows how the cohort did: mean and median per criterion with each member's score on a strip, who has not submitted, every member's model and reviewed totals with a Replay link, and a CSV download.

Members see their new session on their dashboard immediately, so on a hosted server you may not need to send links at all. The member list also shows Groq and GitHub chips per person, green with a tick when they have saved their own key, so you can see before class who will be sharing the classroom limit.

05

Know what the challenger sees

Read the challenger guide; the short version is here
  1. Sign in, set a display name, optionally add their own Groq key.
  2. Open the session: a desktop with Team Chat, Files, Submit, Tickets, Mail.
  3. Read the brief, ask the interviewer questions, write DESIGN.md in the Files editor.
  4. Press Submit DESIGN.md. The assessor joins and probes their decisions.
  5. Press Grade run for directional feedback.

Point them at the challenger guide before the day. It covers how to get a Groq key from console.groq.com/keys and, for build scenarios, how to link a repo, connect GitHub or GitLab so Files can commit, and (optionally) a read-only token from github.com/settings/tokens or their GitLab profile.

06

Watch sessions live or replay them

Where: Sessions → a row → the replay view

The Sessions tab lists every session you own with its scenario, engineer level, challenger, state (submitted, in progress, not started), event count and last activity. Search by id, name or scenario, and filter by state or scenario. Click a row to get the replay:

  • ▶ Play and the scrubber step through the transcript in order; the counter shows where you are.
  • ● Live in the replay header follows the session as it happens: new messages, signals, submissions and grades append to the timeline without refreshing, and the Design panel redraws when a design lands. Click it again to stop.
  • Filter chips: Chat, Mail, Tickets, Signals. Signals are the simulator's own lines: reveal-ladder unlocks, the assessor joining, submissions. They are the fastest way to see whether discovery happened.
  • The Design panel on the right draws the diagram the simulator made from their submitted DESIGN.md, the same picture the assessor and grader work from. Refresh re-reads it after a resubmission; until a submission exists it says so. Any ```mermaid block in the transcript is drawn inline too.

Reading an interview replay

  • Count the opened up (rung n) signals in the interviewer's channel. Three rungs unlocked means the challenger extracted the hidden constraints; none means they designed blind.
  • Look for Submitted work: DESIGN.md and then the assessor's first question. Everything after it is the defense.
  • Answers that cite the document by section are the strongest signal of authorship.
07

Grade, add extra credit, export

Where: the replay view → Grade, Export

Grade runs the rubric over the transcript, the submitted design, its diagram and, with the Tickets extra credit box ticked, the tickets. Each criterion comes back with a score and cited evidence; extra credit is shown separately and adds up to 15 points to the base total.

The grade is saved. Reopening the replay shows it with when and by whom it was graded, the session's state becomes graded with the percentage on the Overview and Sessions tabs, and Export uses it as is. Press Regrade to replace it; if the challenger submits again after a grade, the session drops back to submitted so you know to look again.

Your review is the grade that counts. Each criterion has a number box beside the model's score: change any you disagree with, write a comment, and press Save review. The total is recomputed with the same weights, the session shows ✓ reviewed, the challenger's Team Chat shows your verdict and note ahead of the model's, and the export records both. The model grade stays on file, so a later Regrade never erases your review; Clear review does.

Export downloads a markdown audit of the session: transcript, submissions, and the grade, ready for your own records or a feedback email.

The grader is directional until calibratedEvery grade carries a caveat until the grader has been checked against human scores on this server. Treat it as a first read and let the replay decide. The calibration harness is described in calibration/README.md for whoever runs the server.
08

Settings and the scenario library

Where: Settings
SettingMeaningInterview advice
Default engineer levelUsed when a session has no level of its own.Set it to your cohort's typical level.
Reset onboardingShows the welcome question again next time.Harmless.
Scenario library: starter URLThe public repo hosted learners fork for a build scenario: on GitHub, or on your class's GitLab when the admin has configured one (docs/PUBLISH-SCENARIOS.md).Leave blank for iv_*; they are written in the browser.
Scenario library: enabledWhether the scenario appears in the New session list. Admins control this.Ask an admin to hide scenarios you are not running, so the list stays short.

A session's scenario and level are fixed when you create it. To move a challenger to a different interview, create a new session for them; an admin can delete the old one.

09

Model keys, rate limits, and why challengers can bring their own

Where: the server's environment (admin) and each challenger's Settings

Two shared resources can bottleneck a class:

  • The language model. On a hosted server it is Groq with a classroom key (GROQ_API_KEY). Every persona reply, reveal-ladder judgment, diagram, and grade is a model call. When twenty people submit at once, the shared key's rate limit is the queue they all wait in.
  • GitHub reads, for build scenarios only. The classroom token (GITHUB_TOKEN) has one rate limit for everyone.
  • GitLab reads, when the admin has pointed the server at your own GitLab instance. Starter links on that host work exactly like GitHub ones: challengers fork, push, and paste their project URL; Connect GitLab lets them edit from Files.

Challengers can store their own Groq key and GitHub or GitLab token in Settings. All are encrypted, never shown again, and used automatically for that person's sessions. Ask the class to do it before the day; the challenger guide has the steps and links.

10

Day-of checklist

Ten minutes before the first interview
  1. Sign in to /instructor; confirm the Sessions tab loads.
  2. Create one throwaway session, open the link yourself, ask the interviewer one question, and check a reply arrives. That proves the model key is live. Type a line into DESIGN.md, submit it, and confirm the Design panel in your replay draws something.
  3. Create the cohort's sessions. Copy all links or tell the group to open their dashboards.
  4. Remind everyone: ask first, write second, submit before talking to the assessor, save with Ctrl+S.
  5. Keep the Sessions tab open. Message counts tick up as people work; a session stuck at the start signal means someone cannot get in.
  6. Afterwards, grade with the extra-credit toggle if you asked for tickets, then export.