Comparing two million-object lists for regression testing with Xero.DataComparer

Xero.DataComparer is a small, dependency-free .NET library for non-regression testing of pricing and risk engines. Instead of a boolean verdict, it reports which rows differ, on which properties, and which rows are missing on either side. The author says it compares two lists of one million objects in about 0.7 seconds, and explains the design choices behind that speed.

Key points

• CompareListAsync pairs rows from two List<T> instances by key properties and reports differing properties
• OnlyInReference, OnlyInTarget and duplicate-key lists mean no row is silently lost
• CompareResult<T> implements ITypedList, so data-bound grids such as WinForms or DevExpress show it without column setup
• Author's benchmark: about 0.7 s and 53 MB for 1,000,000 objects on a 16-core machine
• For expression-tree benchmarks in file-based apps, set #:property PublishAot=false to avoid interpreted compilation
• Install with dotnet add package Xero.DataComparer

Pricing and risk engines need to answer a simple question on every release: does the new build produce the same numbers as the old one? Xero.DataComparer goes beyond a yes or no answer. It reports which rows differ, which properties changed, the values on each side, and which rows exist on only one side. You name the key properties that pair rows across two lists and the volatile properties to ignore, and the result keeps the original objects so you can navigate to any other field.

The author reports about 0.7 seconds and 53 MB for two lists of one million objects on a 16-core machine. The speed comes from compiled, typed property accessors, a presized dual hash index, parallel stages, and compact diff rows. Duplicate keys are reported instead of being silently dropped. A benchmarking caveat: file-based apps default to PublishAot, which can make expression compilation fall back to interpretation and distort measurements.

Real-world use

• Checking that a new pricing engine release matches the previous build across millions of output rows
• Reporting which positions changed on which risk metrics, such as PnL or VaR, between two runs
• Browsing regression differences in a WinForms grid without writing a view model

Why it matters

Row, property and value level differences make regression failures actionable instead of a bare failed check. Reporting duplicate keys also prevents a comparison from wrongly reporting identical results when rows were dropped.

Related stories

• Collection Return Types in C#: Interfaces, Views, and SnapshotsDLNick Cosentino (Dev Leader)2 d ago
• Dynamic Gauss-Legendre numerical integration in C#: computing nodes and weights on the fly without overflowHBHabr C# (ru)2 d ago
• .NET 8 and 9 End of Support: A Deployment-First Migration ChecklistDVDEV Community: #dotnet2 d ago