The IntelliQ DSR paper: spec and plan

This file is the single source of truth for the research paper on our scheduler. It has two jobs. First, it pins down the actual algorithm as it exists in code today, with every constant and equation, so the paper can't drift from what shipped. Second, it lays out how I think the paper should be written, evaluated, and shipped, in order, so we can execute it.

The writing and figure standards live in PAPER_STYLE.md. Read this file for what we claim; read that file for how the claim gets typeset, drawn, and worded.

If the code and this file ever disagree, the code wins and this file gets updated the same day.

Part 1: What the system is

The one-sentence version

DSR (Difficulty-Stability-Retention) is a per-concept spaced repetition scheduler that predicts recall with a power-law forgetting curve, keeps two memory state variables per concept (stability and difficulty), schedules reviews at the moment predicted recall hits a retention target derived from the student's GPA goal and exam calendar, and weights each graded retrieval by how much effort the retrieval format demanded.

The name

"DSR" refers to the three quantities the model tracks and updates for every concept: difficulty, stability, and predicted retention. Nobody documented this until now, which is exactly why the paper should define it in the first section.

What makes it different from prior work

I want to be careful here. FSRS is the closest relative and it's good. Our claims are narrower and production-shaped. Five things we do that neither SM-2, Leitner, nor stock FSRS does:

1. Scheduling is per concept, not per card. A concept carries multiple prompt variants (a flashcard face and a short-answer rephrase at minimum). A fail on one variant schedules the sibling variant for the retry, which tests the knowledge instead of the wording.
2. Evidence weighting. A typed free-recall answer moves memory state more than tapping a multiple-choice option. Concretely: FREE_RECALL 1.0, SHORT_ANSWER 0.9, MULTIPLE_CHOICE 0.55, TRUE_FALSE 0.35. No mainstream scheduler distinguishes retrieval effort at all.
3. Goal-conditioned target retention. The interval at which we schedule is derived from the student's stated GPA goal, tightened as exams approach. The scheduler optimizes for a target the student chose, not a fixed 0.9.
4. Same-day discipline. Reviews within one calendar day earn only 40% of normal stability growth, and a passed card can never resurface the same day. Massed practice can't inflate the schedule.
5. Audit-first design. Every graded retrieval is an immutable event with a client-generated idempotency key. Offline batches replay exactly (a retry produces the same fuzzed interval because the fuzz is seeded by the event id). This is what makes shadow A/B tests of scheduler parameters possible in production.

There's a sixth thing that's more engineering than science but worth a paragraph in the paper: the scheduler is one server-side module shared by web, mobile, and chat surfaces, with no client-side scheduling arithmetic anywhere. Every claim we make about scheduling behavior is therefore a claim about one code path.

Related work to cite and position against

Ebbinghaus (forgetting curve), Leitner box, SM-2 (Pimsleur lineage, SuperMemo), FSRS (power-law curve, stability/difficulty, open optimizer), half-life regression (Settles & Meeder, Duolingo), learner-paced scheduling, retrieval-practice literature (testing effect: Roediger & Karpicke), desirable difficulties (Bjork), and the multimedia/massed-vs-spaced practice literature (Cepeda et al. meta-analysis). The paper's positioning: FSRS fit the model to review logs; DSR bakes retrieval-effort structure and student goals into the update rules, then verifies calibration in production.

Part 2: The algorithm, exactly

All of this lives in intelliq-web/src/lib/scheduler/index.ts at version intelliq-dsr-v2. I pulled each number from the code while writing this.

2.1 The forgetting curve

equationR(t, S) = (1 + F * t / S) ^ -0.5,    F = 0.9^-2 - 1 ≈ 0.2346

• R is predicted recall probability, t is elapsed hours since last review, S is stability in hours.
• Anchors: R(0, S) = 1 and R(S, S) = 0.9 by construction. One stability unit is the time recall decays to 90%.
• S is clamped to [1, 8760] hours (one hour to one year).

The inverse gives the interval that hits a target retention:

equationinterval(S, p) = S * (p^-2 - 1) / F

2.2 Target retention

The student's GPA goal maps to a baseline retention target by linear interpolation over anchors:

equationGPA 2.0 -> 0.88, 2.5 -> 0.89, 3.0 -> 0.90, 3.5 -> 0.92, 4.0 -> 0.94

Exam proximity adds urgency: up to +0.03 as the exam nears (urgency = clamp(1 - daysToExam/30)), capped at 0.97 overall. A 4.0-goal student the night before an exam reviews at 0.97; a 2.0-goal student a month out reviews at 0.88. The cost is real: going from 0.88 to 0.97 roughly doubles the review frequency. The paper should show this tradeoff curve.

2.3 The four-button grade vocabulary

Grades are AGAIN, HARD, GOOD, EASY, with two lookup tables:

"Asserted recall" is the recall probability the grade claims about the student's memory state at the moment of grading. The gap between predicted and asserted recall is the surprise term below.

2.4 Evidence weights

equationFREE_RECALL 1.0, SHORT_ANSWER 0.9, MULTIPLE_CHOICE 0.55, TRUE_FALSE 0.35

A failed grade is floored at weight 0.9 regardless of format, on the theory that a fail under easy conditions is still a fail.

The LLM grader (suggestGrade) proposes a grade from verdict, recall score, grader confidence, response time (≤8s), and assistance flags: wrong → AGAIN, partial → HARD, and a correct answer only earns EASY when the score is ≥0.97, confidence ≥0.85, fast, and unassisted. Students may override free-recall and short-answer grades but not objective quiz formats. The paper should report how often overrides happen and whether the grader or the student is better calibrated. That's a section of its own.

2.5 Difficulty update

On every graded retrieval (other than the first):

equationd' = clamp( d
            + decay * (5 - d)
            + w * 0.18 * (seed(grade) - d)
            + w * 1.2 * clamp(R - asserted(grade), -0.6, +0.6)
            + offset,
          1, 10)

• decay heals difficulty toward neutral 5 in proportion to elapsed time: 0 at 24 h, linear to a maximum of 0.15 of the remaining distance at 30 days. Difficulty is a state, not a verdict; time heals it.
• The pull term moves difficulty 18% of the way toward the grade's seed, scaled by evidence weight w.
• The surprise term is clamped at ±0.6 so a single review after a long absence can't crash or spike difficulty. Unclamped, one lucky EASY after five months away swung difficulty by 0.88 points; clamped, the most it can add through surprise is 0.72.

First-ever review sets difficulty from a separate initial-memory table (AGAIN 8.5, HARD 6.5, GOOD 5.0, EASY 3.5, plus the profile offset), not from the update seed: the first grade starts the concept closer to neutral than the update seed would leave it. The seed table in 2.3 drives the pull term on later reviews.

2.6 Stability update

Three regimes.

Same session (retry flag set, or under 6 h elapsed):

• AGAIN: S' = 0.8 S
• pass: S' = S * exp(w * gain * (1 - d/12)), gain per the table (0.05 to 0.18)

Cross-session, AGAIN (a lapse):

equationS' = max(1, S * (0.18 + 0.32 * (1 - R)) * (1.15 - 0.04 * d))

A lapse collapses stability by roughly 80%, worse for harder concepts and more surprising failures.

Cross-session, pass:

equationS' = S * exp( w * gradeGain * (1.35 - 0.07 d) * (0.6 + 2 (1 - R)) * saturation * massed )

• gradeGain: 0.35 / 1.00 / 1.35 for HARD / GOOD / EASY
• the difficulty factor slows growth as concepts get harder
• the spacing factor rewards reviews that happened when recall was genuinely low
• saturation = clamp((S/24)^-0.08, 0.55, 1.25) damps growth of already-stable concepts
• massed = 0.4 when the review happened within 24 h of the last one (same-day damping), else 1

The spacing factor is the heart of the model's responsiveness: reviewing at R ≈ 0.9 (easy, well-timed) earns a growth factor near exp(0.8) ≈ 2.2, while reviewing at R ≈ 0.3 (barely remembered) earns up to exp(2.0) ≈ 7.4 times more stability growth. Struggling is when the schedule learns the most. That sentence should survive into the paper.

2.7 Next due date

Pass, normal path:

equationdue = now + max(20 h, fuzz(interval(S', target) * profileMultiplier))

• The 20-hour floor is the same-day guarantee: a passed card never resurfaces on the calendar day it was passed.
• Fuzz is ±5% (max ±12 h) and only applies at 72 h or beyond. It's deterministic, seeded by the event's client id, so an offline replay produces the identical interval.
• profileMultiplier is the shadow-profile stability multiplier, bounded to [0.85, 1.15].

AGAIN, or second in-session failure: due in 10 minutes, state RELEARNING. Relearning steps: pass → 24 h + fuzz(0-2 h), second pass → graduate with at least 72 h. Two failures in one session flip the concept to NEEDS_INSTRUCTION, which routes the student to guided recovery material instead of more repetition. The paper should defend this as an explicit model of "the student is missing background knowledge, not attention."

First review after a lecture completes: due 18-24 h later (18 h floor plus a per-concept seeded spread of 0-6 h so a lecture's concepts don't come due in one wall).

2.8 Session construction

The queue builder scores every candidate:

equationrisk = 0.45 (1 - R̂) + 0.2 d/10 + 0.2 examUrgency + 0.15 min(lapses/3, 1)
     + min(overdueHours / 720, 1) * 0.2

Then: 30-minute default budget (45 under exam pressure, urgency ≥ 0.6 on any candidate), never below 5 minutes even with a blown weekly budget, new concepts capped at 4 per session and only admitted when nothing due was left out, lectures interleaved so no three consecutive items share a lecture, and failed concepts reinserted with their sibling prompt 2 positions later (risk ≥ 0.75) or 4 later (risk ≥ 0.6). The risk weights are round numbers from design judgment, not fitted. The paper should say so plainly and make fitting them a future-work item.

2.9 Shadow profiles and promotion

Scheduler parameters (stabilityMultiplier, difficultyOffset) live in a SchedulerProfile table with scopes: GLOBAL, CONTENT_FAMILY (keyed by evidence type, so each retrieval format fits its own stability multiplier), USER_OFFSET (per student). A canary profile runs on a deterministic 10% hash bucket. Every review event logs both the incumbent's and the challenger's predicted recall.

Fitting grid-searches the parameter box to minimize Brier score on logged events; promotion requires the challenger's Brier to be at most 98% of the incumbent's with calibration error no worse. Training gates before a profile may even be fit: 2,000 events across 100 users for content-family scope, 200 events and 20 failures for user-offset scope. adjustRecallProbability applies the multiplier as p^(1/m), which keeps predictions honest when intervals were computed under a different multiplier.

This is, as far as I know, the paper's most distinctive methodological section: scheduler parameters evaluated like an ML model, in production, with a calibration-based promotion rule, rather than tuned by feel.

2.10 Offline and idempotency guarantees

Each event carries a clientEventId unique per user. The recording path runs in a serializable transaction with three retries, dedupes by client id (a replayed event returns its prior result as a duplicate), replays events in occurredAt order, clamps device clock skew (a future timestamp can't push lastReview past server time, a late event can't rewind it), and never schedules below the server's now. Mobile writes every event to local storage before attempting sync. The paper needs one solid paragraph and a state diagram here, because "the schedule survives a phone dying mid-review" is part of the contribution.

Part 3: How I want the paper to go

Title candidates

1. "Scheduling at the Student's Goal: Evidence-Weighted, Goal-Conditioned Spaced Repetition for Lecture-Capture Study"
2. "DSR: A Difficulty-Stability-Retention Scheduler with Retrieval-Effort Weighting and Calibrated A/B Evaluation"
3. "From Lecture to Long-Term Memory: Production Evaluation of a Goal-Conditioned Spaced Repetition System"

I lean toward 1 for an education-data venue, 2 if we go to an ML-adjacent venue. Decide after we pick the target.

Target venue

My order of preference: EDM (Educational Data Mining) or LAK for the audience fit, then AIED, then JEDM if we want journal length. L@S (Learning at Scale) works if the shadow-A/B story is the headline. Deadlines cluster in January-March for most of these, which sets the schedule below.

The thesis, stated so we can test it

A spaced repetition scheduler that (a) operates per concept across prompt variants, (b) weights updates by retrieval effort, and (c) schedules to a student-selected retention target, will achieve equal or better measured retention at equal or lower review cost than SM-2 and fixed-target FSRS baselines, and its predicted recall probabilities will be calibratable in production via shadow evaluation.

Every experiment below exists to support or break one clause of that sentence.

Experiments, in order of execution

E1: Simulator baselines (offline, no users needed, do this first). Replay the existing ReviewEvent log through DSR and through SM-2 and Leitner reimplementations, holding the event sequence fixed and comparing predicted-vs-actual recall. Metrics: Brier score, calibration error (reliability diagrams in 10 bins), and retention-per-review-minute. This gives us the core table. Nothing else starts until this one runs clean.

E2: Ablations. Each ablation disables one mechanism and reruns E1: evidence weights set to 1.0 everywhere, same-day damping removed, 20 h pass floor removed, difficulty decay removed, surprise clamp removed, GPA conditioning replaced with fixed 0.9. Six runs, one table, and honest effect sizes. If same-day damping turns out not to matter, we say so and simplify the system. This is the section reviewers will attack, so it needs to be airtight and unflattering where reality is unflattering.

E3: Shadow A/B production results. Pull the SchedulerProfile history: incumbent vs challenger Brier over time, promotion decisions, and per-scope (content family, user offset) deltas. This is real-world evidence no simulator can provide, and it's the section I'd lead with in a talk.

E4: Grader-vs-student calibration. We log the LLM's suggested grade, whether the student overrode it, and the final grade. Compare long-run retention of concepts graded by machine vs overridden by student. This answers "can an LLM grade recall" with data, and it's a second paper hiding in our logs if we need one.

E5: The goal-conditioning tradeoff study. Simulate review load at target retention 0.88 through 0.97 and report reviews-per-week vs predicted exam performance. This turns the GPA-to-retention mapping from a design choice into a measured curve, and it doubles as the figure for the intro.

E6 (stretch): A live comparison arm. If we can ethically randomize, a small cohort on SM-2 vs DSR for one semester. Honestly, I think E1-E5 stand on their own and E6 should only happen if a reviewer demands a controlled trial and our IRB allows it.

Data requirements and honesty constraints

• Minimum viable dataset: I'd want at least 50,000 review events across at least 300 users before submitting E1/E2 anywhere. The training gates in 2.9 exist for a reason; our evaluation should clear a higher bar than our promotion rule.
• Split by user, never by event, in every train/test partition. Event-level splits leak per-concept state across the boundary and will inflate everything.
• Report the distribution of evidence types. If 80% of logged events are multiple choice, the evidence-weighting claim is weak in practice and we must say so.
• The synthetic seed data (demo users) must be excluded from everything. Mark and filter by a seed flag.
• Ethics: student GPA goals and exam calendars are sensitive. De-identify, aggregate, and get the IRB determination before touching any export. Budget two weeks for this; it's always longer than expected.

Paper skeleton

1. Introduction: students record lectures and then study vaguely; we close the loop from capture to schedule. Fig 1 is the E5 tradeoff curve.
2. Related work: FSRS and half-life regression as model relatives; testing-effect literature for why retrieval grading matters.
3. The DSR model: sections 2.1-2.7 of this file, cleaned into equations with a notation table.
4. System design: concept graph, event log, idempotent replay, one scheduler shared across surfaces (section 2.10, plus the queue builder 2.8).
5. Calibrated evaluation: shadow profiles and promotion (2.9). This is where the production rigor lives.
6. Experiments E1-E5 with the ablation table as the centerpiece.
7. Discussion: what the round-number constants cost us, where fitting the initial-memory table and risk weights is future work.
8. Limitations: no controlled trial (unless E6), self-selected GPA goals, single institution population, LLM grading dependence.
9. Reproducibility statement: we release the pure scheduler module (it has no I/O and 64 passing tests today), the simulator harness, and the anonymized event schema.

Execution plan

Phase 0 (week 1): freeze the algorithm. Tag the current commit as the paper's reference implementation, write the version note in the code, and lock this file as the spec. Any scheduler change after the tag gets a changelog line or the experiments rerun.

Phase 1 (weeks 1-3): build the replay simulator. It reads ReviewEvent rows, reconstructs memory state by pure replay through the module, and computes Brier/calibration for a parameterization. The scheduler module is already pure, so this is mostly plumbing plus the SM-2/Leitner reimplementation for comparison. Deliverable: E1 table.

Phase 2 (weeks 3-4): ablations (E2) and the load tradeoff study (E5). Both reuse the simulator.

Phase 3 (weeks 4-5): pull production shadow metrics (E3) and grader-override analysis (E4). Needs the export script and the de-identification pass, which the IRB window covers.

Phase 4 (weeks 5-7): write. I draft sections 3-5 from this file and the experiment outputs; the humans do the intro, related work voice, and the IRB/ethics framing. Related work needs an actual literature pass, which I'd rather do as a dedicated research task with real sources than from memory.

Phase 5 (week 8): internal review against this file, then submission.

What I need from the humans

1. A decision on venue before phase 1 starts, because page limits shape which experiments get figures.
2. A decision on whether student data can be exported for analysis, and who signs off.
3. A named contact for the IRB determination.
4. Confirmation that we can publish the scheduler module as open source, since the reproducibility statement depends on it.

Risks I want on the record now

• The evidence-weight numbers (1.0 / 0.9 / 0.55 / 0.35) are design guesses. If E2 shows the MULTIPLE_CHOICE discount doesn't improve calibration, the honest paper drops the claim to "no worse than unweighted" and keeps the mechanism because it's principled. Say it that plainly.
• Sample skew: free recall happens on the web study surface and multiple choice in quizzes, so evidence type confounds with user segment. E1 must stratify by evidence type, not just pool.
• The GPA-to-retention anchors were picked to feel right. E5 is what defends them; if the curve says 0.94 costs twice the reviews for marginal retention, we report it and consider flattening the mapping in production.
• Reviewer reflex: "this is just FSRS with extra constants." The answer is E2 plus the shadow-evaluation section; if those underdeliver, the paper isn't ready and we wait a cycle rather than ship a thin version.