Single epoll thread per 28 cores by adamsitnik · Pull Request #35800 · dotnet/runtime (original) (raw)

I've shared the numbers from my most recent experiment in a comment above. PTAL

Based on these numbers I came up with the following proposal for the heuristic that determines the number of epoll threads:

we need one epoll thread for every 28 cores
we need to "round up" in a way that 29 cores get 2 epoll threads
we need to double that for ARM

The code that I've just pushed gives the following results:

ratio is (#35800/before #35330) - 1.0

Machine	Connections	Benchmark	before #35330	#35330	#35800	ratio
Citrine 28 cores	128	PlaintextPlatform	7,274,914	7,389,508	7,753,695	6.58%
		JsonPlatform	728,738	753,185	824,774	13.18%
		FortunesPlatform	288,242	293,637	301,217	4.50%
		Fortunes Batching	169,087	167,237	175,886	4.02%

	256	PlaintextPlatform	8,855,217	8,898,258	8,949,456	1.06%
		JsonPlatform	941,176	952,582	1,078,476	14.59%
		FortunesPlatform	291,339	301,213	334,134	14.69%
		Fortunes Batching	311,945	299,331	336,176	7.77%

	512	PlaintextPlatform	8,785,644	9,139,882	9,251,310	5.30%
		JsonPlatform	919,425	956,259	1,124,823	22.34%
		FortunesPlatform	289,177	302,984	305,273	5.57%
		Fortunes Batching	358,163	349,256	411,341	14.85%

	1,024	PlaintextPlatform	8,798,429	9,093,448	9,329,115	6.03%
		JsonPlatform	917,482	983,014	1,135,564	23.77%
		FortunesPlatform	261,790	273,522	298,583	14.05%
		Fortunes Batching	372,989	374,679	407,914	9.36%

	20,000	PlaintextPlatform	6,711,039	6,707,423	7,251,077	8.05%
		JsonPlatform	742,247	754,620	732,218	-1.35%
		FortunesPlatform	208,385	220,029	227,443	9.15%
		Fortunes Batching	289,530	301,026	347,141	19.90%


Perf 12 cores	128	PlaintextPlatform	4,548,601	4,581,534	4,439,358	-2.40%
		JsonPlatform	438,914	456,929	502,222	14.42%
		FortunesPlatform	120,766	127,799	136,628	13.13%

	256	PlaintextPlatform	4,520,799	4,728,698	5,288,782	16.99%
		JsonPlatform	441,074	464,803	545,610	23.70%
		FortunesPlatform	123,775	132,081	138,271	11.71%

	512	PlaintextPlatform	4,439,709	4,915,243	5,368,813	20.93%
		JsonPlatform	456,198	480,191	554,399	21.53%
		FortunesPlatform	121,289	130,383	129,009	6.36%

	1,024	PlaintextPlatform	4,270,802	4,856,757	5,282,018	23.68%
		JsonPlatform	453,737	480,158	561,559	23.76%
		FortunesPlatform	108,143	118,506	123,213	13.94%

	20,000	PlaintextPlatform	3,886,569	3,960,775	4,039,859	3.94%
		JsonPlatform	309,933	333,290	388,005	25.19%
		FortunesPlatform	94,303	105,309	110,534	17.21%


ARM 32 cores	128	PlaintextPlatform	5,325,320	5,248,309	5,320,333	-0.09%
		JsonPlatform	470,719	467,996	430,931	-8.45%
		FortunesPlatform	70,159	79,601	87,110	24.16%

	256	PlaintextPlatform	5,443,043	5,433,406	5,571,782	2.37%
		JsonPlatform	455,767	420,229	458,268	0.55%
		FortunesPlatform	73,379	76,414	86,376	17.71%

	512	PlaintextPlatform	5,143,935	5,644,389	5,773,530	12.24%
		JsonPlatform	425,086	397,756	454,824	7.00%
		FortunesPlatform	80,027	79,361	84,626	5.75%

	1,024	PlaintextPlatform	5,289,294	5,409,985	5,817,791	9.99%
		JsonPlatform	350,471	376,589	440,122	25.58%
		FortunesPlatform	59,300	53,292	60,414	1.88%

	20,000	PlaintextPlatform	3,799,859	4,109,911	4,229,440	11.31%
		JsonPlatform	246,717	258,675	309,816	25.58%
		FortunesPlatform	44,415	36,242	40,573	-8.65%


Mono 56 cores	128	PlaintextPlatform	6,011,013	6,508,597	6,577,628	9.43%
		JsonPlatform	462,300	673,968	655,741	41.84%

	256	PlaintextPlatform	6,896,236	6,906,699	6,930,573	0.50%
		JsonPlatform	600,973	980,908	1,052,635	75.16%

	512	PlaintextPlatform	6,941,870	6,941,820	6,954,308	0.18%
		JsonPlatform	623,578	1,079,661	1,136,029	82.18%

	1,024	PlaintextPlatform	6,960,810	6,962,596	6,957,445	-0.05%
		JsonPlatform	741,710	1,138,508	1,166,838	57.32%

	20,000	PlaintextPlatform	6,825,034	6,784,191	6,786,858	-0.56%
		JsonPlatform	660,291	919,557	944,349	43.02%