summaryrefslogtreecommitdiffhomepage
path: root/src/http (follow)
AgeCommit message (Collapse)AuthorFilesLines
2017-01-20Upstream: fixed cache corruption and socket leaks with aio_write.Maxim Dounin1-0/+15
The ngx_event_pipe() function wasn't called on write events with wev->delayed set. As a result, threaded writing results weren't properly collected in ngx_event_pipe_write_to_downstream() when a write event was triggered for a completed write. Further, this wasn't detected, as p->aio was reset by a thread completion handler, and results were later collected in ngx_event_pipe_read_upstream() instead of scheduling a new write of additional data. If this happened on the last reading from an upstream, last part of the response was never written to the cache file. Similar problems might also happen in case of timeouts when writing to client, as this also results in ngx_event_pipe() not being called on write events. In this scenario socket leaks were observed. Fix is to check if p->writing is set in ngx_event_pipe_read_upstream(), and therefore collect results of previous write operations in case of read events as well, similar to how we do so in ngx_event_pipe_write_downstream(). This is enough to fix the wev->delayed case. Additionally, we now call ngx_event_pipe() from ngx_http_upstream_process_request() if there are uncollected write operations (p->writing and !p->aio). This also fixes the wev->timedout case.
2016-12-24Win32: support 64-bit compilation with MSVC.Maxim Dounin2-13/+13
There are lots of C4244 warnings (conversion from 'type1' to 'type2', possible loss of data), so they were disabled. The same applies to C4267 warnings (conversion from 'size_t' to 'type', possible loss of data), most notably - conversion from ngx_str_t.len to ngx_variable_value_t.len (which is unsigned:28). Additionally, there is at least one case when it is not possible to fix the warning properly without introducing win32-specific code: recv() on win32 uses "int len", while POSIX defines "size_t len". The ssize_t type now properly defined for 64-bit compilation with MSVC. Caught by warning C4305 (truncation from '__int64' to 'ssize_t'), on "cutoff = NGX_MAX_SIZE_T_VALUE / 10" in ngx_atosz()). Several C4334 warnings (result of 32-bit shift implicitly converted to 64 bits) were fixed by adding explicit conversions. Several C4214 warnings (nonstandard extension used: bit field types other than int) in ngx_http_script.h fixed by changing bit field types from uintptr_t to unsigned.
2016-12-24Win32: fixed some warnings reported by Borland C.Maxim Dounin2-5/+5
Most notably, warning W8012 (comparing signed and unsigned values) reported in multiple places where an unsigned value of small type (e.g., u_short) is promoted to an int and compared to an unsigned value. Warning W8072 (suspicious pointer arithmetic) disabled, it is reported when we increment base pointer in ngx_shm_alloc().
2016-12-22Fixed missing "Location" field with some relative redirects.Ruslan Ermilov4-18/+10
Relative redirects did not work with directory redirects and auto redirects issued by nginx.
2016-12-21Core: relative redirects (closes #1000).Ruslan Ermilov4-2/+16
The current version of HTTP/1.1 standard allows relative references in redirects (https://tools.ietf.org/html/rfc7231#section-7.1.2). Allow this form for redirects generated by nginx by introducing the new directive absolute_redirect.
2016-12-21Limited recursion when evaluating variables.Ruslan Ermilov1-6/+30
Unlimited recursion might cause stack exhaustion in some misconfigurations.
2016-12-15Access log: support for json escaping.Valentin Bartenev1-6/+77
2016-12-13The size of cmcf->phase_engine.handlers explained.Ruslan Ermilov1-1/+4
2016-12-10HTTP/2: prevented creating temp files for requests without body.Valentin Bartenev1-2/+4
The problem was introduced by 52bd8cc17f34.
2016-11-28HTTP/2: fixed posted streams handling.Valentin Bartenev3-38/+42
A bug was introduced by 82efcedb310b that could lead to timing out of responses or segmentation fault, when accept_mutex was enabled. The output queue in HTTP/2 can contain frames from different streams. When the queue is sent, all related write handlers need to be called. In order to do so, the streams were added to the h2c->posted queue after handling sent frames. Then this queue was processed in ngx_http_v2_write_handler(). If accept_mutex is enabled, the event's "ready" flag is set but its handler is not called immediately. Instead, the event is added to the ngx_posted_events queue. At the same time in this queue can be events from upstream connections. Such events can result in sending output queue before ngx_http_v2_write_handler() is triggered. And at the time ngx_http_v2_write_handler() is called, the output queue can be already empty with some streams added to h2c->posted. But after 82efcedb310b, these streams weren't processed if all frames have already been sent and the output queue was empty. This might lead to a situation when a number of streams were get stuck in h2c->posted queue for a long time. Eventually these streams might get closed by the send timeout. In the worst case this might also lead to a segmentation fault, if already freed stream was left in the h2c->posted queue. This could happen if one of the streams was terminated but wasn't closed, due to the HEADERS frame or a partially sent DATA frame left in the output queue. If this happened the ngx_http_v2_filter_cleanup() handler removed the stream from the h2c->waiting or h2c->posted queue on termination stage, before the frame has been sent, and the stream was again added to the h2c->posted queue after the frame was sent. In order to fix all these problems and simplify the code, write events of fake stream connections are now added to ngx_posted_events instead of using a custom h2c->posted queue.
2016-12-08Map: the "volatile" parameter.Ruslan Ermilov1-1/+14
By default, "map" creates cacheable variables [1]. With this parameter it creates a non-cacheable variable. An original idea was to deduce the cacheability of the "map" variable by checking the cacheability of variables specified in source and resulting values, but it turned to be too hard. For example, a cacheable variable can be overridden with the "set" directive or with the SSI "set" command. Also, keeping "map" variables cacheable by default is good for performance reasons. This required adding a new parameter. [1] Before db699978a33f (1.11.0), the cacheability of the "map" variable could vary depending on the cacheability of variables specified in resulting values (ticket #1090). This is believed to be a bug rather than a feature.
2016-12-08Map: simplified "map" block parser.Ruslan Ermilov1-1/+2
No functional changes.
2016-12-07Perl: added PERL_SET_INTERP().Maxim Dounin1-0/+8
For Perl compiled with threads, without PERL_SET_INTERP() the PL_curinterp remains set to the first interpreter created (that is, one created at original start). As a result after a reload Perl thinks that operations are done withing a thread, and, most notably, denies to change environment. For example, the following code properly works on original start, but fails after a reload: perl 'sub { my $r = shift; $r->send_http_header("text/plain"); $ENV{TZ} = "UTC"; $r->print("tz: " . $ENV{TZ} . " (localtime " . (localtime()) . ")\n"); $ENV{TZ} = "Europe/Moscow"; $r->print("tz: " . $ENV{TZ} . " (localtime " . (localtime()) . ")\n"); return OK; }'; To fix this, PERL_SET_INTERP() added anywhere where PERL_SET_CONTEXT() was previously used. Note that PERL_SET_INTERP() doesn't seem to be documented anywhere. Yet it is used in some other software, and also seems to be the only solution possible.
2016-11-22Mp4: fixed setting wrong mdat atom size in very rare cases.hucongcong1-1/+3
Atom size is the sum of atom header size and atom data size. The specification says that the first 4 bytes are set to one when the atom size is greater than the maximum unsigned 32-bit value. Which means atom header size should be considered when the comparison takes place between atom data size and 0xffffffff.
2016-12-05SSL: $ssl_curves (ticket #1088).Maxim Dounin1-0/+3
The variable contains a list of curves as supported by the client. Known curves are listed by their names, unknown ones are shown in hex, e.g., "0x001d:prime256v1:secp521r1:secp384r1". Note that OpenSSL uses session data for SSL_get1_curves(), and it doesn't store full list of curves supported by the client when serializing a session. As a result $ssl_curves is only available for new sessions (and will be empty for reused ones). The variable is only meaningful when using OpenSSL 1.0.2 and above. With older versions the variable is empty.
2016-12-05SSL: $ssl_ciphers (ticket #870).Maxim Dounin1-0/+3
The variable contains list of ciphers as supported by the client. Known ciphers are listed by their names, unknown ones are shown in hex, e.g., ""AES128-SHA:AES256-SHA:0x00ff". The variable is fully supported only when using OpenSSL 1.0.2 and above. With older version there is an attempt to provide some information using SSL_get_shared_ciphers(). It only lists known ciphers though. Moreover, as OpenSSL uses session data for SSL_get_shared_ciphers(), and it doesn't store relevant data when serializing a session. As a result $ssl_ciphers is only available for new sessions (and not available for reused ones) when using OpenSSL older than 1.0.2.
2016-12-05SSL: $ssl_client_v_start, $ssl_client_v_end, $ssl_client_v_remain.Maxim Dounin1-0/+9
2016-11-28HTTP/2: fixed saving preread buffer to temp file (ticket #1143).Valentin Bartenev1-0/+3
Previously, a request body bigger than "client_body_buffer_size" wasn't written into a temporary file if it has been pre-read entirely. The preread buffer is freed after processing, thus subsequent use of it might result in sending corrupted body or cause a segfault.
2016-11-14Upstream: handling of upstream SSL handshake timeouts.Maxim Dounin1-0/+7
Previously SSL handshake timeouts were not properly logged, and resulted in 502 errors instead of 504 (ticket #1126).
2016-11-10Range filter: only initialize ctx->ranges in main request.hucongcong1-6/+6
It is not necessary to initialize ctx->ranges in all request, because ctx->ranges in subrequest will be reassigned to ctx->ranges of main request.
2016-11-04Style: switch.Ruslan Ermilov2-3/+0
2016-11-03Cache: prefix-based temporary files.Maxim Dounin3-35/+19
On Linux, the rename syscall can be slow due to a global file system lock, acquired for the entire rename operation, unless both old and new files are in the same directory. To address this temporary files are now created in the same directory as the expected resulting cache file when using the "use_temp_path=off" parameter. This change mostly reverts 99639bfdfa2a and 3281de8142f5, restoring the behaviour as of a9138c35120d (with minor changes).
2016-11-03Upstream: avoid holding a cache node with upgraded connections.Maxim Dounin1-0/+17
Holding a cache node lock doesn't make sense as we can't use caching anyway, and results in "ignore long locked inactive cache entry" alerts if a node is locked for a long time. The same is done for unbuffered connections, as they can be alive for a long time as well.
2016-11-02Cache: proxy_cache_max_range_offset and friends.Dmitry Volyntsev6-0/+105
It configures a threshold in bytes, above which client range requests are not cached. In such a case the client's Range header is passed directly to a proxied server.
2016-11-02HTTP/2: flow control debugging.Sergey Kandaurov1-0/+4
2016-11-01Perl: fixed optimization in SSI command handler.Maxim Dounin1-1/+1
As the pointer to the first argument was tested instead of the argument itself, array of arguments was always created, even if there were no arguments. Fix is to test args[0] instead of args. Found by Coverity (CID 1356862).
2016-10-31HTTP/2: slightly improved debugging.Ruslan Ermilov1-3/+4
2016-10-17Upstream: removed ngx_http_upstream_srv_conf_t.default_port.Ruslan Ermilov2-3/+0
This is an API change.
2016-10-17Upstream: don't consider default_port when matching upstreams.Ruslan Ermilov1-6/+0
The only thing that default_port comparison did in the current code is prevented implicit upstreams to the same address/port from being aliased for http and https, e.g.: proxy_pass http://10.0.0.1:12345; proxy_pass https://10.0.0.1:12345; This is inconsistent because it doesn't work for a similar case with uswgi_pass: uwsgi_pass uwsgi://10.0.0.1:12345; uwsgi_pass suwsgi://10.0.0.1:12345; or with an explicit upstream: upstream u { server 10.0.0.1:12345; } proxy_pass http://u; proxy_pass https://u; Before c9059bd5445b, default_port comparison was needed to differentiate implicit upstreams in proxy_pass http://example.com; and proxy_pass https://example.com; as u->port was not set.
2016-10-17Upstream: consistently initialize explicit upstreams.Ruslan Ermilov1-0/+2
When an upstream{} block follows a proxy_pass reference to it, such an upstream inherited port and default_port settings from proxy_pass. This was different from when they came in another order (see ticket #1059). Explicit upstreams should not have port and default_port in any case. This fixes the following case: server { location / { proxy_pass http://u; } ... } upstream u { server 127.0.0.1; } server { location / { proxy_pass https://u; } ... } but not the following: server { location / { proxy_pass http://u; } ... } server { location / { proxy_pass https://u; } ... } upstream u { server 127.0.0.1; }
2016-10-31Upstream: do not unnecessarily create per-request upstreams.Ruslan Ermilov1-17/+17
If proxy_pass (and friends) with variables evaluates an upstream specified with literal address, nginx always created a per-request upstream. Now, if there's a matching upstream specified in the configuration (either implicit or explicit), it will be used instead.
2016-10-31Upstream: added the ngx_http_upstream_resolved_t.name field.Ruslan Ermilov6-17/+10
This fixes inconsistency in what is stored in the "host" field. Normally it would contain the "host" part of the parsed URL (e.g., proxy_pass with variables), but for the case of an implicit upstream specified with literal address it contained the text representation of the socket address (that is, host including port for IP). Now the "host" field always contains the "host" part of the URL, while the text representation of the socket address is stored in the newly added "name" field. The ngx_http_upstream_create_round_robin_peer() function was modified accordingly in a way to be compatible with the code that does not know about the new "name" field. The "stream" code was similarly modified except for not adding compatibility in ngx_stream_upstream_create_round_robin_peer(). This change is also a prerequisite for the next change.
2016-10-31Upstream: removed unnecessary condition in proxy_eval() and friends.Ruslan Ermilov4-4/+4
The first condition added in d3454e719bbb should have just replaced the second one.
2016-10-31HTTP/2: limited maximum number of requests in connection.Valentin Bartenev3-8/+32
The new directive "http2_max_requests" is introduced. From users point of view it works quite similar to "keepalive_requests" but has significantly bigger default value that is more suitable for HTTP/2.
2016-10-27Mp4: introduced custom version of ngx_atofp().Maxim Dounin1-14/+60
This allows to correctly parse "start" and "end" arguments without null-termination (ticket #475), and also fixes rounding errors observed with strtod() when using i387 instructions.
2016-10-16Proxy: support variables for proxy_method directive.Dmitry Lazurkin1-6/+10
2016-10-21SSL: RFC2253 compliant $ssl_client_s_dn and $ssl_client_i_dn.Dmitry Volyntsev1-0/+6
Originally, the variables kept a result of X509_NAME_oneline(), which is, according to the official documentation, a legacy function. It produces a non standard output form and has various quirks and inconsistencies. The RFC2253 compliant behavior is introduced for these variables. The original variables are available through $ssl_client_s_dn_legacy and $ssl_client_i_dn_legacy.
2016-10-21Image filter: support for WebP.Valentin Bartenev1-5/+159
In collaboration with Ivan Poluyanov.
2016-10-20HTTP/2: graceful shutdown of active connections (closes #1106).Valentin Bartenev2-9/+39
Previously, while shutting down gracefully, the HTTP/2 connections were closed in transition to idle state after all active streams have been processed. That might never happen if the client continued opening new streams. Now, nginx sends GOAWAY to all HTTP/2 connections and ignores further attempts to open new streams. A worker process will quit as soon as processing of already opened streams is finished.
2016-10-19SSL: compatibility with BoringSSL.Maxim Dounin1-1/+4
BoringSSL changed SSL_set_tlsext_host_name() to be a real function with a (const char *) argument, so it now triggers a warning due to conversion from (u_char *). Added an explicit cast to silence the warning. Prodded by Piotr Sikora, Alessandro Ghedini.
2016-10-14Upstream: handling of proxy_set_header at http level.Maxim Dounin4-0/+56
When headers are set at the "http" level and not redefined in a server block, we now preserve conf->headers into the "http" section configuration to inherit it to all servers. The same applies to conf->headers_cache, though it may not be effective if no servers use cache at the "server" level as conf->headers_cache is only initialized if cache is enabled on a given level. Similar changes made in fastcgi/scgi/uwsgi to preserve conf->params and conf->params_cache.
2016-10-14Proxy: do not create conf->headers_source when not needed.Maxim Dounin1-15/+10
2016-10-14Upstream: hide_headers_hash handling at http level.Maxim Dounin1-1/+17
When headers to hide are set at the "http" level and not redefined in a server block, we now preserve compiled headers hash into the "http" section configuration to inherit this hash to all servers.
2016-10-14Upstream: hide_headers_hash inherited regardless of cache settings.Maxim Dounin1-6/+1
Dependency on cache settings existed prior to 2728c4e4a9ae (0.8.44) as Set-Cookie header was automatically hidden from responses when using cache. This is no longer the case, and hide_headers_hash can be safely inherited regardless of cache settings.
2016-10-13Cache: cache manager debugging.Ruslan Ermilov1-9/+23
2016-10-10Modules compatibility: compatibility with NGX_HTTP_SSL.Maxim Dounin4-10/+4
With this change it is now possible to load modules compiled without the "--with-http_ssl_module" configure option into nginx binary compiled with it, and vice versa (if a module doesn't use ssl-specific functions), assuming both use the "--with-compat" option.
2016-10-10Modules compatibility: compatibility with NGX_HAVE_FILE_AIO.Maxim Dounin1-2/+0
With this change it is now possible to load modules compiled without the "--with-file-aio" configure option into nginx binary compiled with it, and vice versa, assuming both use the "--with-compat" option.
2016-10-10Modules compatibility: compatibility with NGX_THREADS.Maxim Dounin2-2/+4
With this change it is now possible to load modules compiled without the "--with-threads" configure option into nginx binary compiled with it, and vice versa (if a module does not use thread-specific functions), assuming both use the "--with-compat" option.
2016-10-10Allowed '-' in method names.Maxim Dounin2-3/+3
It is used at least by SOAP (M-POST method, defined by RFC 2774) and by WebDAV versioning (VERSION-CONTROL and BASELINE-CONTROL methods, defined by RFC 3253).
2016-10-06Realip: fixed duplicate processing on redirects (ticket #1098).Maxim Dounin1-4/+4
Duplicate processing was possible if the address set by realip was listed in set_realip_from, and there was an internal redirect so module context was cleared. This resulted in exactly the same address being set, so this wasn't a problem before the $realip_remote_addr variable was introduced, though now results in incorrect $realip_remote_addr being picked. Fix is to use ngx_http_realip_get_module_ctx() to look up module context even if it was cleared. Additionally, the order of checks was switched to check the configuration first as it looks more effective.