Beyond Vector Search: Building a Clause Forest (FoC) Architecture for Financial RAG

Business Pain Point: In highly hierarchical, long-document domains such as finance and healthcare, traditional vector search (including hybrid retrieval) suffers from structural recall failure when facing cross-chapter logical comparison queries. Architectural Breakthrough: Designed the FoC (Forest of Clauses) architecture, elevating the document's table of contents to a "first-class citizen." It employs a dual-engine concurrent retrieval of Top-down (LLM tree-structure routing) and Bottom-up (vector fragment search), assembling a precision "subtree" in memory. Engineering Barriers: Built a custom $O(N)$ stack-based parser to dynamically construct clause forests with non-standard hierarchies (a blind spot for general-purpose commercial parsers); introduced vLLM Prefix Caching to resolve long-context performance bottlenecks, reducing TTFT from seconds to milliseconds at medium-to-high concurrency; integrated Guided Decoding to guarantee 100% structured output. Bottom Line: Eliminated th