Project reportPLXN WorldSep 2024 – Apr 2026

CLUE AI: a syllabus-aware tutor grounded in the student’s textbooks and their own past lessons

Srinath H1

1Founding engineer, PLXN World · New Delhi, India · 1406srinath@gmail.com

Abstract

IB students don’t need a chatbot that knows everything. They need one that knows their syllabus, their textbooks, and what happened in last Tuesday’s lesson. I built CLUE AI to be that tutor. It parses textbooks with layout-aware PDF parsing[1], transcribes hours-long 1-on-1 sessions[2], and embeds both into one vector store[3]. A ReAct agent[4] answers with a three-tier memory and checks itself against textbook citations before it replies. Vector search returns in ≤ 10 ms; the first token arrives in under 120 ms.

Keywords retrieval-augmented generation · agent memory · education · speech recognition · model routing

Figure 1. The three-tier memory. Recent turns live verbatim in a window sized by tokens, not messages. What falls out is folded into a running summary. When a session ends, what the student learned and struggled with is kept across sessions, so the next lesson starts where the last one stopped.

1Introduction: the problem

IB students spend hours in 1-on-1 lessons. Between lessons, they’re on their own with a textbook and a general-purpose chatbot. The chatbot didn’t know which syllabus they were on, which edition of the book they used, or what their teacher had already explained — so it would answer correctly and still be no help.

I wanted a tutor with three properties. It should be grounded: every explanation traceable to the student’s own textbook. It should remember: the lesson recordings and past chats are part of the context. And it should be fast enough to feel like a conversation.

This report describes the system I built as the founding engineer at PLXN World¹, from September 2024 to April 2026.

2System overview

The front end is Next.js 16[7]. The back end is FastAPI[8], with long jobs pushed through Redis to Celery workers[5] and files kept in MinIO[6].

Two ingestion paths feed one store. Textbooks go through Docling, which keeps headings, tables and figures in reading order. Lesson recordings — hours long, files up to 200 MB² — go through Whisper v3 Turbo. Both are chunked and embedded into pgvector (Figure 2).

Architecture: Next.js 16 client calls FastAPI; FastAPI queues jobs on Redis for Celery workers; workers run Docling on PDFs and Whisper v3 Turbo on recordings stored in MinIO; both write embeddings to pgvector, which the agent queries. Next.js 16student app FastAPIAPI + agent Redis → Celeryjob queue Doclinglayout-aware PDF Whisper v3 Turbolesson audio pgvector uploads ≤ 200 MB → MinIO retrieval ≤ 10 ms → agent
Figure 2. System architecture. Solid arrows: ingestion. Dashed: uploads to object storage, and retrieval back into the agent.

3The agent and its memory

The tutor is a LangChain ReAct agent[4]. It decides when to search the textbook, when to search past lessons and when to just answer. Its context C is assembled from the three tiers of Figure 1:

C = Wtok ⊕ Sprog ⊕ Mped ⊕ R(q)(1)

where Wtok is the token-aware window, Sprog the progressive summary, Mped the cross-session pedagogical memory and R(q) the passages retrieved for question q.

3.1Grounded self-correction

Before an answer reaches the student, the agent checks its own draft against textbook citations, and revises until every claim is supported (Algorithm 1).

Algorithm 1 Grounded self-correction

  1. draft ← agent(q, C)
  2. cites ← retrieve(textbook, draft)
  3. while not supported(draft, cites) do
  4. draft ← revise(draft, cites)
  5. cites ← retrieve(textbook, draft)
  6. end while
  7. return draft with cites

4Multi-model routing

No single model was best at everything a student asks. A router sends each request to Gemini, Claude or GPT-4.1 — GPT-4.1 for step-by-step maths written out in LaTeX (Figure 3).

Figure 3. The multi-model router. Each request goes to one of three models.

4.1Latency budget

A tutor that pauses feels like it’s thinking about you instead of with you. I held the system to a budget:

tfirst token = tretrieve + troute + tgenerate < 120 mstretrieve ≤ 10 ms(2)

5Results

Table 1. System figures.
MeasureValueComponent
Vector search≤ 10 mspgvector[3]
Time to first token< 120 msstreaming agent
Largest lesson upload200 MBMinIO[6] + Whisper v3 Turbo[2]
Memory tiers3window · summary · pedagogical
Models routed3Gemini · Claude · GPT-4.1

6Plates

Placeholder photographs. Screenshots of the tutor chat, the citation view and the lesson library go here.

Chalkboard covered in maths (placeholder)
Plate I. Step-by-step maths.
Open books on a table (placeholder)
Plate II. The textbook as ground truth.
Hand writing notes in a notebook (placeholder)
Plate III. Lesson notes.
Library shelves (placeholder)
Plate IV. The library of past sessions.
Small robot reading on a bench (placeholder)
Plate V. The tutor at work.
Supplementary video S1. Placeholder montage; to be replaced by a recorded tutoring session.

AImplementation details

Front end
Next.js 16
API
FastAPI
Jobs
Redis · Celery workers
Storage
MinIO · PostgreSQL + pgvector
Parsing
Docling (layout-aware PDF)
Speech
Whisper v3 Turbo
Agent
LangChain ReAct · 3-tier memory
Models
Gemini · Claude · GPT-4.1

BCode availability

The code belongs to PLXN World and is not public. I’m glad to walk through the design.

References

  1. [1]Docling. Layout-aware document conversion: keeps headings, tables and figures in reading order.
  2. [2]Whisper large-v3 Turbo. Speech recognition, used to transcribe hours-long 1-on-1 lessons.
  3. [3]pgvector. Vector similarity search inside PostgreSQL; one store for textbooks and lessons.
  4. [4]LangChain ReAct agent. Interleaves reasoning with tool calls: search textbook, search lessons, answer.
  5. [5]Celery on Redis. Background workers for parsing and transcription jobs.
  6. [6]MinIO. S3-compatible object storage for uploads up to 200 MB.
  7. [7]Next.js 16. The student-facing web app.
  8. [8]FastAPI. The Python API that hosts the agent.
  1. ¹ CLUE, the tutoring marketplace for IB, IGCSE and A-Level students (Jun 2023 – Mar 2024), is a separate product I also built at PLXN World.
  2. ² One 1-on-1 lesson can run for hours; the upload path accepts files up to 200 MB.

Next slice in time

Conversation Brain

Mar 2026 – May 2026Live transcription that flags claims that don’t add upOpen

All work