Selecting and Evaluating Data Sources
The trust hierarchy High trust: Your firm’s internal data (portfolio companies, investment amounts, ownership, LP data). When external sources conflict with your internal data, your data is almost always correct. Use it as ground truth. If it’s incorrect, you have a workflow issue, not a technical issue. Medium trust: Established vendors for their core competency. PitchBook for funding data. CoreSignal for employee counts. Your CRM for companies you track. These are reliable enough for analysis, but understand the tradeoffs: PitchBook is accurate but slow. Signal providers like Specter or Harmonic are fast but incomplete. Pick based on your use case: accuracy for LP reporting, speed for deal flow alerts. Low trust: Scraped data, meeting transcriptions, AI summaries. Useful for context and exploration, not definitive analysis. Transcriptions mishear names. Scraped data is stale. AI hallucinates. Validate with your portfolio Take 10-20 portfolio companies and check vendor accuracy: funding data, employee counts, descriptions, update speed. This reveals which vendors are reliable for which data types. Use this analysis to pick your authoritative source for each data type. Don’t guess. Make one row per company and one column per field you actually buy the feed for. Use the round amount you know from closing documents, the date the round closed, and the date the vendor first reported it. A correct amount reported three months late is still useful for historical analysis. For an alert that’s supposed to tell a partner who is raising this week, it’s a miss. Keep a place for ‘unknown’ in the results. If your firm didn’t participate in a round and you don’t know the amount, you can’t score the vendor as wrong. Ask an associate to look at the disagreements: the vendor might be counting a seed extension separately while fund ops includes it in the original round. That tells you more about whether you can use the feed than a single accuracy percentage. Pick one source per use case Choose an authoritative source for each figure. If you do need to combine records from several vendors, Entity Resolution covers how to match them.The Validation Challenge
In operational systems, you can validate that email addresses have @ symbols, that phone numbers are numeric, that dates are in the future for scheduled events. These rules work because the domain is constrained and you control data entry. A valid number can still be a surprise. Down rounds happen, employee counts fall after layoffs, and a round announced today may have closed last quarter. Those cases are worth checking before a validation rule drops them. The most effective validation is humans with industry knowledge reviewing data. Someone who knows the fintech space will immediately spot that “Stripe raised a $10M Series A in 2023” is wrong because Stripe raised their Series A over a decade ago. Someone who follows AI infrastructure companies will know which competitors belong in a market map. Automated systems won’t catch these errors because they lack context. This is why you should know the industries and markets your fund focuses on. If you’re building data infrastructure for a fintech fund, learn fintech. If you’re supporting a deep tech fund, understand deep tech. Domain knowledge is your best validation tool. When you build an analysis, look at it yourself before showing it to anyone. Does it pass the gut feel test? If you’re showing a market map of 200 AI companies and you’ve never heard of 150 of them, something is probably wrong with your sourcing criteria. If funding totals seem way too high or too low compared to what you know about the market, investigate. Your intuition catches problems that automated validation misses.Handling Data Staleness
Always show when data was last updated. If you’re displaying employee count, show “500 employees (as of Dec 2024).” Don’t present stale data as if it’s current. Some data vendors have refresh or rescrape endpoints. If you need current data on a specific company, you can request that they update it. This works for high-priority companies (companies you’re actively diligencing, portfolio companies) but doesn’t scale to refreshing your entire database daily. Use selective refresh strategically. When you’re preparing a market map for Monday’s partner meeting, refresh the key companies in that market over the weekend. Don’t try to keep everything fresh all the time. For some use cases, stale data is fine. If you’re analyzing funding trends over the past five years, using data from three months ago is perfectly adequate. If you’re building a list of potential competitors for a portfolio company, employee counts from six months ago are probably close enough.Building Trust with GPs
Demo to friendly associates first. Before presenting to partners, show it to a junior associate who knows the market. They’ll spot embarrassing mistakes before decision-makers see them. An associate telling you “Airbnb didn’t raise a Series A in 2023” is much better than a GP saying it in front of the partnership. Cite the source next to the figure. “500 employees on LinkedIn” and “500 employees reported by the CFO” give the deal team different answers to “how sure are we?” Keep the source for each field when you combine records. A single ‘last updated’ timestamp for the company profile is misleading if the employee count came in yesterday but the valuation hasn’t changed since 2022. When a partner spots an error, you want to know which feed and which observation produced it. Learn from each mistake. When bad data reaches GPs, figure out why and fix the root cause. Report errors to vendors. Improve entity resolution. Don’t use low-trust data for high-stakes analysis. Each mistake is a chance to improve your infrastructure. Accept imperfection. Even with perfect processes, external data will have errors. Set this expectation with your team: “We use PitchBook, which is generally accurate but has occasional errors.” Better than promising perfect data and failing to deliver.Prevention and Detection
Prevention: Good ingestion practices When you ingest data from external sources:- Normalize data (consistent date formats, consistent naming, lowercase domains)
- Store what you received so you can trace errors back to the source
- Version your data so you can see what changed and when
- Flag data from low-trust sources so it doesn’t get used inappropriately
- Uniqueness: Each company should have only one canonical ID
- Not null: Every company has a name. Leave the founding date empty if the source doesn’t know it.
- Relationships: Every investment references a valid company and investor