Student-level AI insights on student profile

# Student-level AI insights on student profile

A per-student insights module on the teacher's student profile pop-out, synthesizing patterns across *both* the student's graded essays and their Daily Pages. Two sub-sections live inside one module: **Academic Patterns**, drawn from the AI feedback on released graded essays (persistent strengths, recurring weaknesses, suggested teacher next moves), and **Intellectual Interests**, drawn from Daily Pages (recurring subjects, curiosities, hobbies, open questions). This is the closest the platform gets to the north-star dream — a real longitudinal portrait of an individual student that compounds over time.

## Problem

Teachers don't currently have a synthesized view of an individual student's intellectual life across their submissions. Strengths and growth areas surface assignment-by-assignment; the rich signal in Daily Pages — what students are curious about, what they keep returning to, what hobbies or open questions they're circling — is essentially invisible at the teacher-dashboard level.

Today a teacher opening a student's profile pop-out gets identifying info and a list of papers in various states. To answer "how is *this* student doing — and what should I do differently for them next time?" the teacher has to open submissions one by one, mentally diff feedback across them, and reconstruct trends the system already has the data for. At 150+ students (UA professor), or even a more typical 30–60-student load, that reconstruction doesn't happen — teachers fall back on memory and gut.

At the same time, a student writing about *A Bronx Tale* across three Daily Pages entries is exactly the hook a teacher needs to (a) personalize instruction, (b) suggest paper topics rooted in what the student actually cares about, and (c) connect with the student on a human level around their real interests. The Daily Pages corpus exists but only as a flat per-entry list — there's no surface where the patterns add up.

What's missing is the synthesis. The assistant has already produced rubric-anchored feedback on every released submission; the Daily Pages corpus carries a year's worth of subjects and curiosities. Rolling those up by student is what this feature does.

This is the student-level analog of `assignment-level-feedback`: same pattern, different aggregation axis.

## Goals

- A single Student Insights module on the student profile pop-out with two clearly labeled sub-sections:
  - **Academic Patterns** — strengths, growth areas, and suggested teacher next moves drawn from released graded essays.
  - **Intellectual Interests** — recurring interests, hobbies, curiosities, open questions, and repeated subjects drawn from Daily Pages.
- Surface trend-grounded content — patterns that show up across multiple papers/entries, not one-off observations from a single submission.
- "Consider next time" suggestions framed as offers of teacher actions — never as instructions to the student.
- Refreshable, with persistence and caching so insights don't regenerate on every profile open.
- A "new insights available" indicator when meaningful new activity has happened since the cached version.
- Telemetry on generation events and on teacher engagement (opens, dwell time, regenerates, clicks into source submissions).
- Sit at the top of the existing student profile pop-out so the teacher sees it the moment they open the panel.
- Architecturally set the (axis, subject, source-set) abstraction in the service layer up so the same primitives can later run against "all submissions in selected date range across selected assignment types" — the future report-generation surface Brian is pointing toward — without rework.

## Non-goals

- **Not student-facing.** Students never see this module. It's a teacher coaching tool.
- **No cross-course aggregation.** Scoped to the course the teacher is currently viewing. A student in multiple courses gets a different module per course. (Multi-course aggregation belongs in the future report surface.)
- **No cross-student comparison or class ranking.** That's a different surface.
- **No grade prediction.** We're not forecasting their next score.
- **No assignment-specific drilldown inside the module.** Clicking a submission still navigates to the submission view. Per-assignment grouping below the module is `student-details-popout-by-assignment`'s job.
- **No insights history.** Latest only; regenerating overwrites in place. (Same shape as `assignment-level-feedback`.)
- **No autonomous teacher-on-student messaging or scheduling.** "Consider next time" suggestions are reading material for the teacher, not actions the system takes.
- **No teacher-authored editing of the generated content** in this iteration. Generated only; editorial overlays are a later question.

## Domain notes

Entities involved:

- **Student** — the subject of the module.
- **Course** — scopes which submissions count. A student in multiple courses gets a module per course.
- **Assignment / AssignmentType** — provides the context the AI uses to weight what matters (a Thesis-Driven Essay vs. Daily Pages have different rubrics; trends should be interpreted in light of that).
- **Submission** — the essay-stream input set. Only released graded submissions in this course feed Academic Patterns. Unreleased grades and in-progress documents are excluded — they're not stable signal and they confuse the teacher about visibility state.
- **Document** — Daily Pages submissions are the substrate for the Intellectual Interests stream.
- **Daily Pages holistic GA** (engineering backlog Issue #1) — produces a consolidated idea-level GA output across a student's Daily Pages. The Intellectual Interests stream may consume that output directly rather than re-scraping every entry (see open questions).
- **Rubric / individual feedback** — the existing assistant-generated feedback on each graded essay is the primary substrate for Academic Patterns. The AI reads across those feedback artifacts and the rubric scores to find recurring patterns.

Two streams feed one module:

- **Essay stream → Academic Patterns.** Reads released graded essays for this student in this course. Substrate is the GA feedback already produced per submission.
- **Daily Pages stream → Intellectual Interests.** Reads either every Daily Pages submission the student has written (Option A) or the consolidated GA output from the Daily Pages holistic GA (Option B). See open questions for the cost/fidelity tradeoff and the recommended prototype path.

This feature shares its underlying AI substrate with `assignment-level-feedback`. That feature rolls feedback *across students within one assignment*; this one rolls feedback *across assignments for one student* — and adds a second corpus (Daily Pages) on a different axis. The same submission-feedback corpus feeds both — design the LLM service layer so it can serve either axis without duplicating prompt scaffolding, and so the Daily Pages stream plugs into the same axis abstraction rather than a parallel pipeline.

## UX sketch

The module sits at the top of the existing student profile pop-out, above the identifying info / grade list (or, once `student-details-popout-by-assignment` lands, above the Assignment-grouped list). It is the first thing the teacher sees. The profile itself should already be organized by Assignment — confirm baseline with Bryant; if `student-details-popout-by-assignment` doesn't already cover that, that reorganization is part of this work too.

```
+----------------------------------------------------+
| Brian (sample student)                  [x close]  |
| brian+gregorio@theconnellschool.com                |
|                                                    |
| +------------------------------------------------+ |
| | Student insights                  [Regenerate] | |
| |                                                | |
| | Academic patterns                              | |
| |   Strengths                                    | |
| |    • Clear topic sentences across body paras   | |
| |    • Strong use of textual evidence            | |
| |   Growth areas                                 | |
| |    • Thesis often too broad / unfocused        | |
| |    • Conclusions don't link back to thesis     | |
| |    • Evidence cited but not analyzed           | |
| |   Consider next time                           | |
| |    • Drafting his thesis with him before he    | |
| |      starts writing                            | |
| |    • Conferencing to pick textual evidence     | |
| |      together                                  | |
| |    • Proofreading the conclusion with him      | |
| |      before he submits                         | |
| |                                                | |
| | Intellectual interests                         | |
| |   Recurring subjects                           | |
| |    • A Bronx Tale (3 entries)                  | |
| |    • Coaching basketball, his brother          | |
| |   Curiosities and open questions               | |
| |    • Why mob films romanticize violence        | |
| |    • How loyalty becomes a trap                | |
| |   Hobbies and interests                        | |
| |    • Pickup basketball, vinyl collecting       | |
| |                                                | |
| | Based on 4 released essays · 22 Daily Pages    | |
| | Updated 2d ago                                  | |
| +------------------------------------------------+ |
|                                                    |
| Assignments                                        |
|   Macbeth thesis essay        Released 96% (A)     |
|   Daily Pages — Week 4        Submitted (5/5)      |
|   ...                                              |
+----------------------------------------------------+
```

**Module label.** The platform's brand stance is **no AI branding in user-facing UI** (no "AI," no sparkle icons, no Copilot framing). "Student insights" is the working candidate; "Coaching summary" and "Student snapshot" are alternates worth weighing. Final label is an open question for Brian.

Tone and framing rules for the generated content:

**Academic Patterns** sub-section:

- **Strengths** — 2–3 bullets. Phrased as the student doing something well consistently.
- **Growth areas** — 2–3 bullets. Phrased as recurring places the student loses points, in plain language (not rubric-score jargon).
- **Consider next time** — 2–3 bullets. Each one phrased as "Consider [verb-ing] with him/her..." — an *offer* of a teacher action, never a prescription. Action verbs the AI should reach for: drafting, conferencing, modeling, proofreading, outlining, brainstorming, walking through, reviewing. Never tell the teacher *how* to teach — only what to focus on with this student.
- Always trend-grounded. The AI must explicitly look across submissions; a single anomalous submission shouldn't drive a growth area unless it confirms an existing pattern.

**Intellectual Interests** sub-section — calm, observational, non-diagnostic. Three sub-sub-sections, each capped at 2–4 bullets:

- **Recurring subjects** — what the student keeps returning to (a book, a topic, a person). Name the entry count where helpful ("3 entries").
- **Curiosities and open questions** — what the student seems to be working out in writing.
- **Hobbies and interests** — what the student does outside school that surfaces in their writing.
- Each bullet must be grounded in specific Daily Pages entries. The model should not extrapolate beyond what the student actually wrote.
- If a sub-sub-section has no clear signal, it doesn't render. Better silence than manufactured "interests."

Pronoun handling — use the student's preferred pronouns if we store them; otherwise use "this student" rather than guessing.

Generation triggers (mirroring `assignment-level-feedback`'s pattern, adapted to two streams):

- **Per-stream minimums.**
  - Academic Patterns requires **at least 2 released graded essays** in this course. Below the floor, the sub-section shows the threshold placeholder.
  - Intellectual Interests requires **at least 5 Daily Pages submissions**, or — if consuming Option B — at least one consolidated Daily Pages GA output to exist. Below the floor, that sub-section shows a calm placeholder ("Appears once Brian has written a few more Daily Pages.") and the other sub-section still renders.
- **Auto-generate** the first time a stream crosses its floor.
- **Auto-regenerate** per stream:
  - Academic Patterns regenerates on each subsequent release of a graded essay for that student in this course (releases are the meaningful event; in-progress drafts and unreleased grades don't shift the picture).
  - Intellectual Interests regenerates on a configurable cadence — per N new Daily Pages submissions or weekly. Avoid regenerating on every individual Daily Pages entry; the marginal new signal is low and the cost adds up fast. Exact policy is an open question.
- **Regenerate button** refreshes both streams. Grayed out before either floor is crossed.
- **"New insights available" indicator** — small dot or text affordance when the cached module is stale relative to recent activity. No notification spam.

Empty, loading, and failure states:

- **Below both floors** — module shows a single calm placeholder ("Insights appear as Brian builds up more graded essays and Daily Pages."). Don't render two redundant placeholders.
- **Exactly one released essay** — Academic Patterns shows: "One paper released so far. Academic patterns appear once a second graded paper is released, so it can show trends rather than one-off observations." Intellectual Interests renders normally if it's above its floor.
- **Loading / generating** — spinner with "Reviewing this student's work…"
- **Failed generation on either stream** — fall back to "Couldn't generate this section right now. Try Regenerate." per sub-section. Don't silently hide the module.

Footer line: "Based on N released essays · M Daily Pages · updated <relative time>" so the teacher can judge freshness at a glance.

The module is collapsible. Default state on first view of a given student is expanded; the teacher's collapsed/expanded preference persists per-teacher (not per-student) so a teacher who prefers to keep it tucked away gets it tucked away everywhere.

## Data model implications

- New table, additive: `student_course_insights` (or similar), keyed unique on `(student_id, course_id)`. One row holds both streams' output.
  - `generated_at` overall, plus per-stream `academic_patterns_generated_at` and `intellectual_interests_generated_at`.
  - `essays_count_at_generation` and `daily_pages_count_at_generation` so the UI can render the "based on N · M" footer and detect staleness when new releases or new Daily Pages happen after generation.
  - `daily_pages_source_version` so we can record which Option A/B path produced the cached output and invalidate if we switch.
  - `insights_json` — versioned, structured: `{ schema_version, academic_patterns: { strengths[], growth_areas[], next_moves[], grounded_submission_ids[] }, intellectual_interests: { recurring_subjects[], curiosities[], hobbies[], grounded_submission_ids[] } }`. `grounded_*_ids` let us link bullets back to source submissions later if we want; not surfaced in v1 UI.
- Latest only — regeneration overwrites in place per stream. No history table. If we later want history, that's an additive append-only change to this table.
- Backward compat: purely additive, no changes to `Submission`, `Assignment`, `Course`, or user tables.
- `insights_json` schema is versioned from day one so we can evolve the structure without a migration.
- Storage scale rough cut: one row per `(student, course)` pair — well-bounded; nothing close to the volume of submissions themselves.
- Telemetry events go into the existing analytics table; new event names: `student_insights_module_viewed`, `student_insights_regenerated`, `student_insights_section_expanded`, `student_insights_source_submission_clicked`.

## File paths in `yawp-2.0` likely to change

Best-guess; engineering to confirm:

- `services/web-app/app/routes/app.my-classes.$courseId/components/student-details-popout.tsx` — add the insights module at the top of the popout. Coordinate with `student-details-popout-by-assignment` if both are in flight (same surface).
- New `services/web-app/app/routes/app.my-classes.$courseId/components/student-insights-module.tsx` — module shell; renders the two sub-section components, the header, the Regenerate button, the footer, and the collapsed state.
- New `services/web-app/app/routes/app.my-classes.$courseId/components/academic-patterns-section.tsx` — strengths / growth areas / consider-next-time content.
- New `services/web-app/app/routes/app.my-classes.$courseId/components/intellectual-interests-section.tsx` — recurring subjects / curiosities / hobbies content.
- Loader for the student profile popout — needs to return the latest `student_course_insights` row for this `(student, course)`, plus the current released-essay count and Daily Pages count so the UI can render threshold and staleness states.
- `packages/db/` — new `student_course_insights` table migration.
- AI/LLM service layer — two prompts + endpoints (or one endpoint that takes an `(axis, subject, source-set)` tuple). Shared substrate with `assignment-level-feedback`. Background-job orchestrator with per-stream throttling so the Daily Pages stream doesn't fire on every submission.
- Telemetry pipeline — wire the new event names.

## Open questions

- [x] **Daily Pages input source — Option B confirmed.** Use the consolidated GA output from the Daily Pages holistic GA rather than scraping every individual submission. Cheaper, and the GA has already done the idea-level pass. Escalate to Option A only if Intellectual Interests reads thin after pilot testing. This means this spec is blocked on the Daily Pages GA being live.
- [ ] **Insights window.** Current semester only, or all-time? Pulls in different directions: longitudinal portrait wants all-time; teacher-this-semester wants current course only. Default v1 assumption: current course's submissions, which usually maps cleanly to current semester. Revisit when the report surface lands.
- [ ] **Privacy / FERPA.** Are student insights visible to anyone besides the assigned teachers? Default: no. This is teacher-only content, but it is a synthesized profile of a student — confirm storage and access patterns clear our existing privacy posture before pilot. Likely fine (derived from data already stored) but worth a deliberate check.
- [ ] **Daily Pages regeneration cadence.** Per N new submissions? Weekly? On teacher visit if stale? Needs a policy that balances freshness vs. cost. Probably tunable behind the feature flag for the pilot.
- [ ] **Module label.** "Student insights" is the working candidate. Brian to confirm — must comply with the no-AI-branding rule. Alternates: "Coaching summary," "Student snapshot," "Trends across released grades."
- [ ] **Sub-section labels for Intellectual Interests.** Recurring subjects / Curiosities and open questions / Hobbies and interests is the working set. Brian/Kevin to validate against real pilot output.
- [ ] **Profile reorganization baseline.** Confirm with Bryant whether `student-details-popout-by-assignment` already covers the by-Assignment grouping this spec assumes. If yes, this spec depends on it landing; if no, that reorganization is part of this work.
- [ ] **Pronoun handling.** Do we store student preferred pronouns anywhere? If yes, the AI should use them. If no, the AI defaults to "this student" / "the student" — no guessing from name.
- [ ] **Cross-AssignmentType weighting (Academic Patterns).** Should Daily Pages submissions weigh the same as a major Thesis-Driven Essay when looking for trends? Default v1 assumption: AI sees all released submissions equally, but the prompt names the AssignmentType per submission so the model can use its own judgment. Kevin to validate the output reads sensibly with mixed types.
- [ ] **Cross-stream interaction.** Should the Essay stream see the Daily Pages output (or vice versa) when generating? A student whose Daily Pages obsessively circle a topic might inform the essay-side "Consider next time" suggestion. Probably defer — keep streams independent in v1 to keep prompt scope contained.
- [ ] **Background job vs. on-demand generation.** Mirrors the same open question on `assignment-level-feedback`. Preferred shape: background job triggered by submission-release / Daily Pages cadence events, with the Regenerate button hitting the same endpoint synchronously.
- [ ] **Should "Consider next time" ever name a specific upcoming Assignment?** E.g., "Consider drafting his thesis with him before the Macbeth essay due 5/27." Higher signal but requires the AI to know about upcoming Assignments. Probably v2 — flag and defer.

## Edge cases

- **Zero released essays and zero Daily Pages.** Module shows the combined threshold placeholder; no generation, no spinner.
- **One released essay, no Daily Pages (or vice versa).** Per-stream threshold placeholders render side-by-side. Don't try to generate one section's content from the other's corpus.
- **All released essays on the same AssignmentType** (e.g., all Daily Pages-only courses). Academic Patterns should still work but the AI should be careful not to overgeneralize beyond that genre. Prompt makes this explicit.
- **Daily Pages stream produces nothing of interest.** Model is allowed (and prompted) to return zero bullets in a sub-sub-section if there isn't enough signal. Sub-sub-section just doesn't render.
- **Inappropriate or surveillance-feeling output.** The Intellectual Interests stream is the highest-risk surface in this repo for reading as creepy. Calibration set explicitly tests for: avoid value judgments, avoid armchair psychology, avoid pretending to know the student's life outside what they wrote, frame as observation of the writing not of the person. Manual QA must explicitly check for this failure mode.
- **Sensitive Daily Pages content** — entries that mention difficult personal topics (mental health, family situations, abuse). The Intellectual Interests stream must not surface these as "interests" or "subjects." Prompt-level guardrails plus a manual-QA fixture covering this case. Same rule for Academic Patterns — never personalize the critique to the student.
- **Highly variable submissions** — one essay where the student aced it, one where they cratered. Academic Patterns should describe variance honestly ("Strong on focused topics, less consistent when scope widens") rather than averaging it into a flat picture.
- **Teacher re-grades or re-releases.** Academic Patterns becomes stale and regenerates on the next release event; Regenerate handles manual cases. Footer counts let the teacher spot stale state. Intellectual Interests is independent and not affected.
- **Submission with no comment text** (just a numeric grade, no rubric comments) — feeds the count but not the qualitative substrate. AI should gracefully handle missing feedback rather than fabricating.
- **Option B path: consolidated Daily Pages GA hasn't been generated yet for this student.** Intellectual Interests sub-section shows the below-floor placeholder regardless of raw entry count.
- **Student in multiple courses.** Each `(student, course)` pair has its own row. The popout, when opened from a class roster, shows only the current course's module.
- **Student dropped from course mid-semester.** Past released grades and Daily Pages still produce a module if accessed historically; freezing behavior follows whatever pattern the dropped-student rules use elsewhere.
- **Legacy submissions predating the AssignmentType refactor.** Still fed into the Academic Patterns corpus; AI gets "AssignmentType: unknown" for those and weights them as it would any other piece of evidence.
- **Legacy Daily Pages predating the Daily Pages holistic GA feature.** Option A still works; Option B can only cover entries with GA output.
- **High-volume teacher (UA professor, 150+ students).** Per-stream cost is the binding constraint — Option A vs. Option B cost-estimate work must assume this load.
- **Student who happens to also be a teacher (or test account).** Same teacher-only visibility rule — only teachers in the course see the module. If a student account ever opens a route that includes this data, the module must not render and the payload must not be in the response.

## Test plan

- **Unit / prompt:**
  - Academic Patterns: given 4 released essays with consistent thesis weakness, AI surfaces "thesis specificity" or equivalent under growth areas and a teacher action under "Consider next time."
  - "Consider next time" bullets always start with a teacher-action verb (drafting / conferencing / modeling / proofreading / etc.). Lint-style check.
  - With one released essay, Academic Patterns refuses and returns the threshold-state response.
  - With zero released essays, Academic Patterns isn't attempted.
  - Intellectual Interests, Option B: fixture of 4 Daily Pages GA consolidations referencing the same novel → module surfaces it under recurring subjects.
  - Intellectual Interests, Option A: fixture of 8 Daily Pages entries with a hobby mentioned in 3 of them → module surfaces it under hobbies.
  - Calibration: fixture where Daily Pages contain no clear pattern → model returns zero or near-zero bullets, doesn't manufacture interests.
  - Calibration: fixture containing sensitive personal content → model does not surface it as an "interest" and Academic Patterns doesn't personalize the critique.
  - Lint: Intellectual Interests bullets read as observational ("returns to X across three entries") not diagnostic ("this student is obsessed with X").
- **Integration:**
  - Loader returns both streams' content plus essays/Daily Pages counts; releases and new Daily Pages that have happened *since* generation are visible in the counts so the staleness footer is accurate.
  - Essay-release event triggers Academic Patterns regen only; Daily Pages cadence event triggers Intellectual Interests regen only; Regenerate button triggers both.
  - Summary row is upserted, not duplicated.
  - Telemetry events fire on view, regenerate, expand, source-click.
- **Manual QA:**
  - UA pilot student with rich Daily Pages corpus and several released essays — verify the module reads as trend-grounded, calm, teacher-actionable, and non-creepy. Compare Intellectual Interests against the same student's actual interests where the teacher can validate. **Most important manual gate before flag-on.**
  - Student with 4+ released grades — verify Academic Patterns reads as trend-grounded, calm, teacher-actionable.
  - Student with exactly 2 released grades — verify Academic Patterns is honest about the thin evidence base.
  - Student with 1 released grade — verify the threshold-state prompt.
  - Student with sensitive content in Daily Pages — verify the module handles gracefully and doesn't surface sensitive content as an "interest."
  - Student with only essays — verify the Intellectual Interests placeholder renders correctly.
  - Student with only Daily Pages — verify the Academic Patterns placeholder renders correctly.
  - Student whose grades vary widely across papers — verify Academic Patterns describes variance rather than papering over it.
  - Student attended-then-dropped — verify behavior matches the dropped-student conventions elsewhere.
  - Confirm module does not render and payload is absent for student-side accounts.
  - Cross-check: student in 2 courses — open the popout from each course; verify each course's module is scoped to its own corpus.

## Rollout

**Pre-handoff gate.** Same pattern as `assignment-level-feedback`: Bryant drafts a prototype PR from this spec; Kevin reviews generated output against real released-graded students and their Daily Pages (UA pilot data is the best substrate). Brian signs off on tone, framing, and the absence of AI branding before this spec flips to ready-for-engineering. The Intellectual Interests stream gets an extra calibration pass given its higher reputational risk.

Feature flag (`student_insights` or similar, naming TBD with the user-facing label). Roll out to UA professor first — highest volume of released graded work and Daily Pages, best stress test for both synthesis streams. Then Washington and Birmingham City after a week of pilot validation. Default flag OFF until pilot validation.

## Dependencies

- **Daily Pages holistic GA (engineering backlog Issue #1).** Option B for the Intellectual Interests source depends on its output existing. Option A doesn't depend on it but is more expensive.
- **Class insights (engineering backlog Issue #2).** Not a hard dependency, but the model/prompt/caching infrastructure built there should be reused here; the LLM service layer is shared.
- **`student-details-popout-by-assignment`.** The by-Assignment grouping is a companion change; confirm baseline with Bryant.

## North star context

This is the closest of the three insights-family issues to the "ultimate dream" — AI insight on an individual student over time. The stickiness/moat argument is that once a teacher has a year of synthesized insight on each of their students, switching costs become real. Build the module so it can scale to "all submissions in selected date range across selected assignment types" without architectural rework — that's the report-generation surface Brian is pointing toward, and the (axis, subject, source-set) abstraction in the service layer is what gets us there.

## Engineering handoff checklist

- [ ] Domain context covered
- [ ] File paths in `yawp-2.0` listed
- [ ] Data model implications spelled out, including backward-compat plan
- [ ] UX sketch in prose
- [ ] Edge cases enumerated
- [ ] Test plan written
- [ ] Rollout plan decided