Six-Step AI Workflow with Human Oversight
Each step shows the workspace, what the AI completed, and what still needs human review.
Steps 5 and 6 need a person. Step 5 is waiting for approval right now. Click it to open the full walkthrough.
Estimates are directional and based on stated assumptions. All names, organizations, and identifying details have been anonymized in accordance with our confidentiality agreements.
How a Growth-Stage Technology Company Built Courses That Change Because of Learners, with Instructor Approval
Every learner on the platform, from the day they arrive to the day the course changes because of them. The agents map what each new learner already knows, serve every lesson at the level that learner can climb from, guide them when they are stuck, and grade their work in minutes instead of days. A person is handed exactly two things: the learner who is slipping, and the change to the course itself. Six stations, run as a loop: the exercise station 6 re-rates on Friday is the one station 2 serves on Monday, at the new level.
What Was Broken
What We Built
Six stations and 12 subagents. Each subagent carries its own tasks and its own refusal.
How It Runs
Placement Mapping
Nobody is asked anything hereAgents read every placement answer, trace each miss to the earliest skill it rests on, and build each learner a path that starts where their knowledge actually stops. Nothing waits on a person here. A learner whose answers are too few to map is put on a short probe set, never on a guess.
Lesson Pacing
Nobody is asked anything hereAgents serve each next exercise off the learner's own recent record, move the level up or down as the answers come in, and log every session against that learner's own usual rhythm. Nothing waits on a person here. A level is never changed on one bad evening, and a quiet holiday week is marked unscored, not disengaged.
Guided Help
Nobody is asked anything hereAgents take each question mid-lesson, walk the learner through a worked example a step at a time, and count every wall that more than one learner hits. Nothing waits on a person here. A learner still stuck after three guided attempts reaches their instructor with the whole transcript, not with a summary.
Work Grading
Nobody is asked anything hereAgents grade each submission against its rubric the minute it lands, write feedback that names the step where the work turned, and post both to the learner's record. Nothing waits on a person here. A submission the rubric cannot score reaches an instructor ungraded and marked why, never scored on a guess.
Progress Watch
A person answers hereAgents move each learner along their path or loop a failed skill back with fresh exercises, and draft a check-in for every learner gone quiet past their own usual rhythm. This is where a person works. A message to a struggling learner goes out in a person's name, and a learner whose own note explains their silence is never sent a template.
Course Tuning
A person answers hereAgents read the week's attempts across the whole cohort, propose a re-rating for every exercise the attempts contradict, and flag the exercise the strong learners fail too. A person accepts or rejects each change. What the course teaches, and at what level, is not something an agent changes quietly.
Where a Person Decides
Operating Model
This changes how work flows through the team.
What Transfers, What Must Be True
Failure Modes
What breaks this pattern:
A learner answers a handful of questions and the map fills in the rest. They start a month too high or too low, and most learners who start wrong quit before anyone finds out why.
A tutor built to be helpful gives the answer to graded work. The learner passes the exercise, keeps the gap, and the record now says they know something they do not.
The grader scores a submission the rubric cannot read, and that wrong grade enters the record. Every pacing and path decision after that builds on a grade nobody checked.
An automated message goes to a learner whose own note already said why they stopped. The learner reads it, sees that nobody read the note, and does not come back.
Directional Outcomes
What the agent counts, and the station that counts it.
These counts are the tallies from one monitored run of the agents. They are not monthly or annual totals.
What Runs Where
Every step names the subagent that does the work, the record it writes, the thing that raises a question for a person, and what it is allowed to touch. This is drawn from the source, not from a diagram somebody kept in sync by hand.
placement-maplearners/<learner-id>/path.jsontoo-few-answers-to-maplearners/**, the lesson library read-onlypace-lessonslearners/<learner-id>/sessions/<session-id>.jsonsignal-too-thin-to-adjustlearners/**, lessons/** read-onlytutor-chatlearners/<learner-id>/help/<chat-id>.jsonstuck-after-three-attemptslearners/**, lessons/** read-onlygrade-and-feedbacklearners/<learner-id>/grades/<submission-id>.jsonrubric-cannot-scorelearners/**, the submission queue read-onlyprogress-watchwatchlist/<learner-id>.json, outreach/drafts/<learner-id>.mdawait-your-sendwatchlist/**, outreach/**, learners/** read-onlycourse-tuneratings/proposed/<exercise-id>.json, reports/week-<date>.jsonaccept-rating-changeratings/proposed/**, reports/**, the attempt log read-onlyStack
Every system this agent reads or writes.