Reading large JSON with PipeReader in .NET 10 System.Text.Json
Ali Hamza Ansari compares the PipeReader overload that .NET 10 adds to System.Text.Json with reading the same large JSON payload from a Stream. He measures execution time, allocated memory and GC collection counts on a file with one million records. The results matter if your high-traffic endpoints deserialize big request bodies.
Key points
- System.Text.Json in .NET 10 accepts a PipeReader in its deserialization overloads.
- In the author's test, the PipeReader path ran about 21% faster than the Stream path.
- The benchmark used one 72 MB file with one million records on a single machine.
- Complete the PipeReader with CompleteAsync() after deserializing.
- Total allocations may rise slightly; the gain comes from lower GC pressure.
- Measure with your own payload sizes and GC counters before changing hot endpoints.
elmah.io's Ali Hamza Ansari checks whether the PipeReader overload that .NET 10 adds to System.Text.Json reduces the cost of deserializing a very large JSON payload. The demo is a minimal API that generates a roughly 72 MB file containing one million telemetry readings. The same file is then deserialized once through a plain stream and once through a PipeReader wrapped around that stream, with a forced GC before each run and timing, allocation and Gen0/Gen1/Gen2 counts recorded.
In the author's runs, the pipeline variant finished about 21% faster and produced less GC pressure, even though it allocates slightly more in total. He argues that trade is worthwhile on hot request paths because it avoids large object heap churn. The numbers come from one demo on one machine, so treat them as directional and measure with your own payload shapes before switching.
Real-world use
- A bulk import minimal API that accepts very large JSON request bodies
- A high-throughput telemetry or IoT ingestion endpoint on ASP.NET Core
- A background worker that deserializes large JSON files from storage
Why it matters
If your services regularly deserialize large JSON bodies, GC cost during deserialization can become a real bottleneck. The PipeReader overload gives you a small code change to test against your current Stream-based code.