Rake on the way to keep-alive

Increasing the activity of data exchange between microservices is often a problem in the architecture of modern IT solutions. Squeeze the maximum and survive at all costs - a serious challenge for any development. Therefore, the search for optimal solutions is a non-stop process. The article briefly outlines the problems that may arise when using highly loaded http requests and ways to bypass them.

This story begins with a mistake. Somehow we carried out load testing, the main element of which was the execution of a large number of short http requests. A client written under netcore 2.2 , starting at some point, throws a System.Net.Sockets.SocketException: Address already in use . It quickly became clear that the ports did not have time to free up on the client, and at some point the system was refused to open a new one. Now, if we go to the code, the problem was in using the old approach with the HttpWebRequest class and construction:

var request = WebRequest.CreateHttp(uri);
using(var resp = request.GetResponse()){ … }

It would seem that we are releasing the resource, and the port should be released in a timely manner. However, netstat signaled a rapid increase in the number of ports in TIME_WAIT state. This state means waiting for the connection to close (and possibly receiving lost data). As a result, the port may be in it for 1-2 minutes. This problem is discussed in some detail in many articles ( Problems with the TIME_WAIT queue , History about TIME_WAIT ). Nevertheless, this means that dotnet is “honestly” trying to close the connection, and further happens already through the fault of the timeout settings in the system.

Why is this happening and how to deal with it

keep-alive. . , . msdn, KeepAlive HttpWebRequest true. HttpWebRequest «» , , . , HttpWebRequest «Connection: keep-alive», HTTP/1.1. , , KeepAlive. HttpWebRequest.KeepAlive = false, «Connection: close». , . nginx .

:

while (true)
{
  var request = WebRequest.CreateHttp(uri);
  request.KeepAlive = false;
  var resp = await request.GetResponseAsync();
  using (var sr = new StreamReader(resp.GetResponseStream()))
  {
    var content = sr.ReadToEnd();
  }
}

, ( 1000 ) . CLOSE_WAIT, LAST_ACK. - , . , «» .

,

, , . keep-alive HttpClient. .

, , ? keep-alive nginx:

• keepalive_timeout – ( 15)
• keepalive_requests – ( 100)

netstat wireshark, . keepalive_requests (> 1000) , .

http , . . , , keep-alive. keep-alive , .

:

• RunHttpClient – HttpClient "Connection: keep-alive"
• RunHttpClientClosed – HttpClient "Connection: closed"
• RunWebRequestClosed - uses the class HttpWebRequest mode "Connection: closed"

The nginx server is configured with the following parameters:

• keepalive_timeout 60s;
• keepalive_requests 100000;