Skip to content

Conversation

@SungJin1212
Copy link
Member

Add -querier.store-gateway-series-batch-size flag to configure the maximum number of series to be batched in a single gRPC response message from Store Gateways.

Benchmark results

goos: darwin
goarch: arm64
pkg: github.com/cortexproject/cortex/pkg/storegateway
cpu: Apple M4 Max
BenchmarkThanosBucketStores_SeriesBatch
BenchmarkThanosBucketStores_SeriesBatch/series_100
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=1
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=1-14         	    3273	    315173 ns/op	  260090 B/op	    4560 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=10
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=10-14        	    4335	    265844 ns/op	  234986 B/op	    3017 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=100
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=100-14       	    4653	    255788 ns/op	  203221 B/op	    2768 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=1000
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=1000-14      	    4700	    253746 ns/op	  210531 B/op	    2768 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=10000
BenchmarkThanosBucketStores_SeriesBatch/series_100/batchSize=10000-14     	    4394	    265009 ns/op	  286143 B/op	    2769 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_1000
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=1
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=1-14        	     656	   1782411 ns/op	 2282108 B/op	   41688 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=10
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=10-14       	     819	   1447217 ns/op	 2057748 B/op	   26274 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=100
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=100-14      	     886	   1326311 ns/op	 1763974 B/op	   23841 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=1000
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=1000-14     	     832	   1423374 ns/op	 2280276 B/op	   23535 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=10000
BenchmarkThanosBucketStores_SeriesBatch/series_1000/batchSize=10000-14    	     825	   1429912 ns/op	 2378003 B/op	   23535 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_10000
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=1
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=1-14       	      63	  18745491 ns/op	22518443 B/op	  423704 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=10
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=10-14      	      76	  15242607 ns/op	20108438 B/op	  268911 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=100
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=100-14     	      85	  13425516 ns/op	17205850 B/op	  244663 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=1000
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=1000-14    	      93	  12481578 ns/op	20406169 B/op	  241365 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=10000
BenchmarkThanosBucketStores_SeriesBatch/series_10000/batchSize=10000-14   	      81	  14098264 ns/op	18610246 B/op	  240819 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100000
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=1
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=1-14      	       6	 187283375 ns/op	245824413 B/op	 4249119 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=10
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=10-14     	       7	 154968667 ns/op	220216622 B/op	 2685126 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=100
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=100-14    	       8	 135575479 ns/op	190494851 B/op	 2443041 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=1000
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=1000-14   	       8	 125515792 ns/op	214022162 B/op	 2409687 allocs/op
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=10000
BenchmarkThanosBucketStores_SeriesBatch/series_100000/batchSize=10000-14  	       8	 128369026 ns/op	202337341 B/op	 2405048 allocs/op
PASS

Which issue(s) this PR fixes:
Fixes #7202

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: SungJin1212 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Store Gateway: Enable series response batching

1 participant