The Taiwan legal AI community page "Legal Detective" has announced the open-source release of "TW Legal RAG," a semantic retrieval tool that structures and vectorizes approximately 22 million Taiwan court rulings. Users can search rulings using everyday language and package results into a format readable by any AI model (ChatGPT, Claude, Gemini, or local models). The tool features a built-in citation verification function to check whether case numbers cited by AI actually exist.
Background: RAG and Legal AI Pain Points
RAG (Retrieval-Augmented Generation) is a mainstream technique to mitigate LLM hallucinations, retrieving relevant facts from an external knowledge base before generating responses. This is critical in law, where fictitious case citations have caused real-world consequences, such as U.S. lawyers being sanctioned for using ChatGPT-invented precedents.
Architecture: BYO-AI Design
TW Legal RAG adopts a unique "BYO-AI" approach: it does not call any LLM itself. Instead, users employ a CLI tool to send semantic search requests to a backend infrastructure. The system returns relevant rulings, and the tool packages them into a structured Bundle that can be fed directly to ChatGPT, Claude, Gemini, or any local model. Users do not need to deploy embedding models or vector indices locally; dependencies are minimal, requiring only httpx, typer, and rich. Installation is via pip install twlegalrag.
The tool offers four core commands: search (natural language semantic search), pack (bundle search results into AI-readable format), check (citation verification to confirm case numbers in AI outputs exist in the bundle), and health (service status check).
Citation Verification: Limitations and Highlights
The citation verification feature uses pure regex matching (no LLM) to check if case numbers mentioned by AI are present in the bundle. However, the developer clearly notes limitations: it cannot verify whether the cited content is correct, detect reasoning errors, or identify semantic paraphrasing hallucinations—only existence of the case number. Nonetheless, this provides a foundational safeguard for legal AI applications.
Behind the Free Open Source
The developer, a solo creator, invested thousands of hours over six months optimizing the retrieval pipeline, with estimated costs nearing NT$1 million. Despite this, the project is released under an MIT license, free and open source, partly to support Taiwan's government push for departmental LLM services. This move significantly impacts Taiwan's legal tech ecosystem, allowing other teams to integrate the tool without building a legal knowledge base from scratch. The tool logs query data for retrieval analysis but does not use it for model training.
Real-world user feedback from legal professionals is still pending.