Measuring .NET performance with AI help using BenchmarkDotNet
The author optimizes a .NET reporting method step by step with BenchmarkDotNet, measuring each change separately. The final version drops from about 1.55 s to 11.9 ms and allocations fall from 33.1 MB to 0.23 MB. It is a useful case for anyone who wants to check AI-proposed optimizations with benchmarks and output comparisons before merging them.
Key points
- Measure with BenchmarkDotNet and [MemoryDiagnoser] before asking AI for optimizations.
- Change one thing at a time and record the time and allocations after each step.
- Replacing the nested Where scan with a single Dictionary pass gave the biggest win (1.5 s to 40 ms).
- A HashSet with StringComparer.Ordinal and TryGetAlternateLookup<ReadOnlySpan<char>> avoids per-line string allocations on .NET 9 and later.
- Compare every version's output with the original before trusting benchmark numbers.
The article argues that AI performance suggestions are often unmeasured: some change nothing, some slow code down, and some alter behavior silently. The author optimizes a sales report method that parses order lines, totals sales per customer, skips blocked customers and returns the top ten. The workflow starts with a BenchmarkDotNet benchmark and a project rules file, then asks the AI for ranked hypotheses with one change at a time.
The changes are applied separately: a single pass with a Dictionary removes the nested per-customer scan, a HashSet speeds up blocked-customer lookups, and span-based parsing with alternate lookups, available from .NET 9, removes most allocations. Every version is checked against the original output before benchmarking. On .NET 10, runtime drops from about 1.55 s to 12 ms and allocations from 33 MB to 0.23 MB.
Real-world use
- A nightly batch job that parses large delimited files and aggregates totals per customer.
- A frequently called reporting endpoint in an ASP.NET Core API that creates GC pressure.
- Reviewing AI-generated performance patches in a pull request, backed by a benchmark in the repo.
Why it matters
AI performance suggestions should be treated as hypotheses; a benchmark and an output equality check show which change really helps and that behavior did not shift.