SAM
Smart Assembly Mentor — turning a 2,191-page admin guide into a grounded, multi-agent assistant SmartAssembly developers can actually ask.
The story, in brief.
SmartAssembly developers needed an assistant grounded in the official admin guide — but that guide runs 2,191 pages, where keyword search returns a page number, not an answer. SAM parses it into a structured, verifiable knowledge layer and puts an orchestrative multi-agent front end on it: ask in natural language, get a cited, grounded answer — and never a fabricated command.
How it’s built.
The multi-agent build: an Orchestrator that classifies intent and routes to one of three specialists — Reference (command lookup), Guide (concept training), or Debug (code analysis). The 2,191-page admin guide was parsed into 1,491 discrete structured commands loaded into a Dataverse retrieval layer for exact lookup, paired with semantic search for fuzzier questions.
Steps through 2 generations — from Stage 0 to Stage 1.
What AI actually does here.
Parsing at scale
A parsing script read the entire 2,191-page admin guide and restructured it into 1,491 discrete, verifiable commands — work no manual indexing effort could absorb on top of everything else.
Exact + semantic retrieval
Exact lookup for “what does this command do,” semantic search for “which command does this” — so a rigid index doesn’t miss the fuzzier questions developers actually ask.
Evaluation engineering
A documented eval set (by Neha Choudhary) grades every build: ≥80% to promote to TEST, 100% the goal before PROD, and any fabricated command is an automatic fail — turning “is it accurate?” into a number.