- General LLMs like ChatGPT struggle to accurately analyze complex financial documents, achieving just 79% accuracy on SEC filings.
- Consulting firms like JPMorgan are developing open source, customized LLMs like DocLLM specifically for financial documents.
- Major firms like KPMG, PwC, and McKinsey have introduced specialized AI to improve efficiency.
- As OpenAI releases more advanced general models, startups worry these could outcompete current specialized LLMs.
Financial consulting firms like JPMorgan are developing their own specialized large language models (LLMs) to analyze documents and data. General LLMs like ChatGPT struggle with accurately interpreting complex financial filings, only achieving a disappointing 79% accuracy on SEC filings. This poor performance on mission-critical documents has pushed financial firms to create customized LLMs specialized for financial documents across modalities.
Financial Firms Build Custom LLMs to Analyze Documents
JPMorgan’s open source DocLLM stands out as a lightweight LLM extension designed specifically for enterprise documents like forms, invoices, reports and contracts. These documents carry intricate semantics across textual and spatial modalities that general LLMs fail to fully capture. DocLLM’s top configurations include a 1 billion parameter model built on Falcon-1B architecture and a more powerful 7 billion parameter version built on Llama2-7B. By open sourcing DocLLM, JPMorgan enables other financial firms to leverage the model while maintaining security over sensitive data.
JPMorgan is not alone in developing specialized AI for financial documents. Other major consulting firms have introduced their own custom virtual assistants and LLMs to enhance operations. For example, KPMG created KaiChat based on OpenAI to aid staff with financial data queries. PwC aims to invest $1 billion over three years to advance generative AI for automating tasks like tax preparation and auditing. EY developed an AI system integrating tax laws that can instantly respond to payroll queries in a ChatGPT interface.
In August 2022, McKinsey launched “Lilli,” an LLM designed to streamline and improve utilization of the firm’s extensive knowledge base. Wells Fargo introduced “Fargo” – a virtual assistant leveraging Google Cloud AI to provide personalized and convenient banking. In October 2022, Deloitte released DARTbot, an internal chatbot that enhances efficiency for over 18,000 Audit & Assurance professionals.
As more advanced general purpose LLMs like GPT-5 release from companies like OpenAI, startups building specialized financial LLMs worry these models could outcompete them. When OpenAI launches major ChatGPT updates like “multimodal” document uploading, hundreds of startups offering similar services can go out of business almost overnight.
However, fine-tuned general LLMs may struggle to match the accuracy and security of financial models like DocLLM designed specifically for that vertical. For now, consulting firms building customized LLMs for accurate financial planning and decisions appears a better path than relying on horizontal AI offerings. Specialized models trained on confidential data can analyze complex filings and contracts with greater precision while preventing sensitive information leaks.
Two products launched right now:
AI Agent: https://orbitmoonalpha.com/shop/ai-tool-agent/
AI Drawsth: https://orbitmoonalpha.com/shop/ai-tool-drawsth/