Building better AI systems starts with knowing their limits

When our team set out to rebuild how we track company mergers and acquisitions, we knew AI would be crucial to processing the large amounts of relevant information available. But we also knew that relying on an LLM alone would introduce mistakes that aren't always easy to spot.

When a deal isn't a done deal

Take a press release announcing that one company plans to acquire another.

An LLM can easily identify this as a done deal, but “to acquire” does not mean “acquired.” And if that deal never closes, that relationship can sit incorrectly recorded for years. This nuance is not one that our team had anticipated, but as we dug into the LLM’s interpretations of these press releases, we realized we needed a way to account for uncertainty as deals progressed. 

To handle this, we built a system that tracks deals over time rather than treating each announcement as a final record. The process starts with a daily automated scan of the news to identify potential acquisitions and records their status. New deals that are announced but not completed get added to a queue that will be rechecked weekly against the latest news for approximately nine months. 

Every new deal that is identified as ‘closed’ is flagged with our curators, who can approve or reject the new status based on the supporting evidence. If a deal is approved it will be incorporated into our data. Rejected deals are logged as such and will not be re-evaluated, this prevents previously rejected deals from being repeatedly presented to the curation team. If a subsequent scan of the news feed identifies new evidence that a deal has closed, it would then move to a human review. 

No new company relationship is added to our knowledgebase without approval from a human curator.

Take Vertex’s acquisition of Crinetics Pharmaceuticals. The deal was announced on July 6, 2026 but wasn’t completed until September 1. 

Previously we would have had to choose between recording the relationship before it closed, or relying on someone to remember to check its status later. With the new system, the deal entered our queue and was tracked weekly. When the completion was announced, our curators reviewed the new evidence and approved the change.

Where the LLM got it wrong

Then there are asset purchases that can be mistaken for full acquisitions, subsidiaries operating under different names, and company relationships that change over time. 

While working through a quality review of our historical backfill data, which was created using an LLM searching SEC filings and public sources, we uncovered a number of complex cases where the LLM confidently identified the relationships incorrectly. 

In 2017, Biogen paid $120 million upfront to acquire CIRARA, a Phase 3 drug, from Remedy Pharmaceuticals. The LLM had interpreted this as Biogen acquiring Remedy itself. Similarly, when Steris purchased the truFreeze product line from CSA Medical in 2019, the LLM recorded CSA as a subsidiary of Steris, even though the company remained independent.

Bristol Myers Squibb acquired Amylin in 2012, but sold it to AstraZeneca in 2014. Both acquisitions were real, but if the first acquisition is kept as a current relationship it would inaccurately place Amylin’s trials and products under the wrong owner today. Even when individual facts are accurate, missing context can create a misleading picture.

Each of these real errors have been identified and corrected. They also directly informed the structures and checks we built into our new M&A tracking system to prevent mistakes like these from making it into our data in the first place.

Building more reliable M&A data

We started with a large-scale backfill of historical company data and completely rebuilt how we identify and resolve company names. That meant recognizing when different names, subsidiaries, and historical identities referred to the same company, so information scattered across sources could be connected to a consistent company identity. 

We searched the deal history of more than 6,200 sponsor companies and extracted 42,000 candidate deal mentions. We then applied filtering rules to remove more than 10,000 asset deals, as well as terminated and unclosed deals, leaving 14,000 deals to resolve against DrugBank’s company records. Through this major backfill we increased our records of company names from 2,325 to 26,009, marking a roughly 11x increase. 

For ongoing M&A tracking, we use Elasticsearch to narrow down potential company matches before bringing in an LLM to help interpret and resolve more complex cases. Rather than leaving those decisions entirely to AI, we apply deterministic rules to constrain how relationships are classified.

We also built extensive evaluation suites around the cases most likely to cause errors. That lets us test changes before deployment, identify regressions, and continue checking performance as new information comes in. Evaluations like this have become a central part of our development process, helping us catch regressions before changes reach production. 

We also maintain a decision ledger that records every decision our curators make, covering both approvals and rejections. Our daily and weekly workflows check against this ledger to identify which relationships are in need of review, without resurfacing decisions that have already been made. 

The work doesn't stop at identifying a deal

As a final step, every new company relationship also goes through human review. But identifying the right relationships is only part of the work. As new information comes in, we're continuously resolving company identities and connecting them to the drugs, clinical trials, and other evidence already in DrugBank. Each new connection adds context to what we already know.

Without that foundation, even a capable LLM has to work from the bottom up, finding individual sources, determining which information is relevant, and trying to piece together the relationships on its own. That process can miss important connections, especially when information is scattered across sources or recorded under different names. Rather than asking an LLM to reconstruct those relationships every time, we resolve and maintain them as part of the underlying data.

Engineering around AI's limitations

One thing this project reinforced is how much work goes into understanding a problem before deciding where AI fits. Before we build anything, we spend time understanding the problem, figuring out where things are likely to go wrong, and defining what a correct result should look like. That work informs where we use AI, what we ask it to do, and the rules and evaluations we build around the LLM.

AI has made it possible to tackle problems at a scale that would have been impractical before, but it doesn't remove the need to understand those problems in depth. If anything, this project reinforced just how important that work is. Many of the issues we encountered weren't ones we anticipated at the outset. We found them by working through the data, testing our assumptions, and adjusting the system as we learned more.