It turns out no browsers implement HTTP/2 GOAWAY handling properly, and large enough number of resources on a page results in failures to load some resources. In particular, Chrome seems to experience errors if loading of all resources requires more than 1 connection (while it is usually able to retry requests at least once, even with 2 connections there are occasional failures for some reason), Safari if loading requires more than 3 connections, and Firefox if loading requires more than 10 connections (can be configured with network.http.request.max-attempts, defaults to 10). It does not seem to be possible to resolve this on nginx side, even strict limiting of maximum concurrency does not help, and loading issues seems to be triggered by merely queueing of a request for a particular connection. The only available mitigation seems to use higher keepalive_requests value. The new default is 1000 and matches previously used default for http2_max_requests. It is expected to be enough for 99.98% of the pages (https://httparchive.org/reports/state-of-the-web?start=latest#reqTotal) even in Chrome.
Similar to lingering_time, it limits total connection lifetime before keepalive is switched off. The default is 1 hour, which is close to the total maximum connection lifetime possible with default keepalive_requests and keepalive_timeout.
Previously, connections returned from keepalive cache had c->data pointing to the keepalive cache item. While this shouldn't be a problem for correct code, as c->data is not expected to be used before it is set, explicitly clearing it might help to avoid confusion.
The directive configures maximum number of requests allowed on a connection kept in the cache. Once a connection reaches the number of requests configured, it is no longer saved to the cache. The default is 100. Much like keepalive_requests for client connections, this is mostly a safeguard to make sure connections are closed periodically and the memory allocated from the connection pool is freed.
The directive configures maximum time a connection can be kept in the cache. By configuring a time which is smaller than the corresponding timeout on the backend side one can avoid the race between closing a connection by the backend and nginx trying to use the same connection to send a request at the same time.
If a connection with the read delayed flag set was stored in the keepalive cache, and after picking it from the cache a read timer was set on that connection, this timer was considered a delay timer rather than a socket read event timer as expected. The latter timeout is usually much longer than the former, which caused a significant delay in request processing. The issue manifested itself with proxy_limit_rate and upstream keepalive enabled and exists since 973ee2276300 (1.7.7) when proxy_limit_rate was introduced.
It's properly aligned and can hold any supported sockaddr.
Iterating through all connections takes a lot of CPU time, especially with large number of worker connections configured. As a result nginx processes used to consume CPU time during graceful shutdown. To mitigate this we now only do a full scan for idle connections when shutdown signal is received. Transitions of connections to idle ones are now expected to be avoided if the ngx_exiting flag is set. The upstream keepalive module was modified to follow this.