Automating the comparison of technical documents, such as legal and financial ones, through a Large Language Model like GPT-3 or GPT-4
The automation of technical document comparison using Large Language Models like GPT-3 or GPT-4 offers several significant benefits, particularly in the realm of legal and financial documentation. Firstly, speed and efficiency are markedly improved. AI models can process and analyze vast amounts of text swiftly, making them invaluable when dealing with large volumes of documents. This can streamline workflows and significantly reduce the time spent on document comparison, allowing professionals to focus on more complex, higher-value tasks.
Another benefit is the ability of AI models to provide consistent, unbiased analysis. They apply the same criteria to every document they analyze, eliminating the risk of human error or subjective interpretation that can occasionally lead to inconsistency. Moreover, these models can operate 24/7 without breaks, offering an "always-on" solution that can deliver results on demand, any time they are required.
In terms of scalability, AI models excel. They can handle increasing volumes of documents without a corresponding increase in personnel or resources, providing a cost-effective solution for large-scale operations. Furthermore, Large Language Models have the capacity to navigate the complexity of intricate documents, identifying patterns or discrepancies that may be difficult for humans to spot.
AI can also serve as a valuable preliminary filtering tool. It can rapidly scan documents to flag potential issues or differences, thereby guiding further detailed examination by human experts. While AI should not replace human judgment—particularly given its current limitations in understanding complex nuances and context—it can play a crucial role as a complementary tool, aiding human professionals in managing the complexity and volume of modern legal and financial document analysis.
Use Case: Deploy a Large Language Model to contrast the 10-Q quarterly financial statements of auto industry leaders Ford, Tesla, and Rivian.
The complexity of these documents makes them an ideal test case to highlight the advantages of using a Large Language Model. For instance, the substantial size of these documents - 66 pages for Ford, 67 for Rivian, and 34 for Tesla - underlines the efficiency of the AI tool.
Typically, an average individual, reading at a speed of 200-300 words per minute, would take substantial time to thoroughly read and comprehend these documents. Given a combined total of 167 pages across the three 10-Qs, an individual would likely require anywhere from 4.5 to 6 hours to complete the reading process, depending on the reading speed. This is where the large language model shows its value, effectively navigating the massive volumes of data far more quickly, thereby showcasing the power of AI in accelerating document review and comparison.
How I retrieved the financial information for Ford, Tesla and Rivian.
Document Source: The U.S. Securities and Exchange Commission's database contains millions of filings from U.S. public companies, including annual reports, quarterly reports, and other filings that often contain various legal documents.
I followed these steps to perform retrieve the financial data for Ford, Tesla and Rivian:
Visit the SEC EDGAR database.
You'll find a search bar where you can enter the name of the company you're interested in.
Once you have the company's page open, you can scroll through their filed documents to find their 10-Qs for Ford, Rivian and Tesla.
You will then need to download and open each of these documents individually and compare the numbers to look for inconsistencies. But rather than downloading the documents, I opted to use Bard to manage this step and simply provided the links to the forms that I wanted compared.
I did this by constructing the following prompt:
“compare the Rivian financial form Q-10 (here: https://www.sec.gov/ix?doc=/Archives/edgar/data/1874178/000187417823000029/rivn-20230331.htm) to the Tesla financial form Q-10 (here: https://www.sec.gov/ix?doc=/Archives/edgar/data/1318605/000095017023013890/tsla-20230331.htm) to the Ford financial form Q-10 (here: https://www.sec.gov/ix?doc=/Archives/edgar/data/37996/000003799623000029/f-20230331.htm)”
Google Bard returned the following comparison in a few seconds: “...comparison of the Rivian, Tesla, and Ford financial forms Q-10 for the quarter ended March 31, 2023:”
Additionally, Google Bard returned a summary of the results in summary form:
“As you can see, Rivian is a much smaller company than Tesla or Ford, with significantly lower revenue, net income, and total assets. However, Rivian has a higher gross margin than Ford and a lower net margin than Tesla. Rivian also has a much higher debt-to-equity ratio than either Tesla or Ford. Despite its size, Rivian is growing rapidly. Revenue grew 295% year-over-year in the first quarter of 2023. Rivian also expects to deliver 25,000 vehicles in 2023, which would be a significant increase from the 1,015 vehicles it delivered in 2022.
Tesla is the largest electric vehicle company in the world. It has a significant lead in terms of production capacity and sales. Tesla also has a strong brand and a loyal customer base. However, Tesla is facing increasing competition from other electric vehicle manufacturers, such as Rivian and Ford. Ford is a traditional automaker that is making a major investment in electric vehicles. Ford has a strong manufacturing network and a large dealer network. Ford is also benefiting from the popularity of its Mustang Mach-E electric SUV. However, Ford faces challenges in terms of battery supply and production capacity. Overall, Rivian is a high-growth company with a lot of potential. However, it is still a relatively small company with significant financial challenges. Tesla is the dominant player in the electric vehicle market, but it faces increasing competition. Ford is a traditional automaker that is making a major investment in electric vehicles. It remains to be seen which company will be the long-term winner in the electric vehicle market.”
While it's important to note that while there are significant benefits to using AI for document comparison, AI models have limitations. They lack human judgment and the ability to understand complex nuances, especially in legal and financial contexts. They also lack the ability to explain their reasoning in detail, which is particularly important in legal and financial decisions.
Therefore, while AI can provide valuable assistance in comparing technical documents, their results should be used as tools to aid human experts, not replace them.
