AI and Torah Accuracy
Why models fabricate Torah citations, and the workarounds the group has found.
On this page
The single most charged technical topic in the group. Every model the group tests will, eventually, fabricate a Likkutei Sichos citation, blend two acharonim into one ruling, or attribute an invented page number to Igros Kodesh. The group has gone through several phases trying to solve it.
The problem, stated plainly
"Without structured data, all of Igros indexed, this problem will persist for years. Unless OpenAI or Anthropic gives access to all texts and structures them, hallucinations for Torah references will continue."
"AI has very little access to actual Torah sources. It's skimming basic English sites, regurgitating Chabad.org back at you."
"Torah doesn't fit the shape that AI is expecting to read. It is layered, subtle, nuanced. The process is lost on it."
LLMs are word generators, not databases. Hebrew sources are under-represented in training data. Layered commentary collapses into a flat "sounds plausible" output.
Workarounds the group has found
1. Constrain sources
- Pull primary text from a trusted corpus (Sefaria, Chabad.org, HebrewBooks, dach.dev) and paste it in. Do not ask the model to fetch from memory.
- NotebookLM indexed with the Rebbe's Likkutei Sichos, Igros Kodesh, Toras Menachem, and Maamorim. Caveat: NotebookLM retrieves only about a dozen sources per query.
- Dicta Mekorotai (
chabad.mekorotai.dicta.org.il), previewed at the Kinus in late 2025, is purpose-built: claims come with a citation, and it rejects what it cannot ground.
2. Use the right model for the right register
- Claude for iyyun and technical Torah, and organized shiur structure. See Shiur Prep.
- Gemini Pro for popular Chassidus and for Hebrew translation with grammatical metadata (when Qwen is not available).
- Qwen for pure Hebrew and Yiddish translation with etymology and shoresh.
- ChatGPT is generally weakest on Torah specifics but strong as a writing partner once you have supplied the sources.
3. Plain-language rules instead of code
The group has drafted a community document of nine principles for AI Torah citation: search methods, citation formats, verification steps, and domain-specific instructions for different text types. Posted for community feedback, not yet finalized, but treated as one of the more important community efforts. See Master Prompts.
4. Build buffer layers (do not expose AI raw to talmidim)
"Never release an AI-coded app with login functionality, financial transactions, or mission-critical email integrations."
Tools like dach.dev and the Maamorim app succeed by curating the corpus and letting the LLM only re-present, not invent.
Halachic and hashkafic threads
- "Bina vs. Chochma." Bina melachutit, artificial understanding, without chochma (the spark of insight). The thread generated a full Shabbos-table drosho (written by Gemini, ironically) tying the Eved Ivri to AI's nature.
- "Can I use AI to write a Devar Torah?" by Menachem Posner (Chabad.org) was shared and debated. The conclusion: AI as an editing or brainstorming aid is fine; AI as the source of a devar Torah is geneivas daas and lacks the Divine unity created when a human is mechadesh.
- AI agents running on Shabbos. See AI and Shabbos. Rough consensus: backend-only is generally OK; anything that triggers front-end notifications, sends customer-facing email, or transacts is problematic (including maris ayin).
Open questions
- Will Mekorotai and Dicta ship to the public? (Promised "this year" as of the Kinus in late 2025.) As of early 2026, still not working well.
- Is there a way to get AI to read Rashi script reliably? Still no good answer.
- Hebrew handwriting (ksav yad) OCR remains unsolved.
Related
- AI and Shabbos
- Hebrew and Yiddish
- Shiur Prep
- Master Prompts
- The tools mentioned here (Dicta Mekorotai, dach.dev, NotebookLM, the Maamorim app, LearningTanach.org) are catalogued in the Toolbox.