Steven Giesel • 12/9/2025

GitHub Runners might not be a good benchmark baseline

This technical article argues against using GitHub Actions runners as a baseline for performance benchmarks. It demonstrates with a C# BenchmarkDotNet example that results can vary significantly between runs, especially on Windows, due to unpredictable underlying hardware (different VMs, CPU spikes). The author provides code and workflow examples to highlight the instability, suggesting developers seek more controlled environments for accurate benchmarking.

0 comments

#benchmarking #performance testing #Github Actions