<feed xmlns='http://www.w3.org/2005/Atom'>
<title>nginx.git/src/core/ngx_output_chain.c, branch release-1.28.2</title>
<subtitle>nginx</subtitle>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/'/>
<entry>
<title>Optimized chain link usage (ticket #2614).</title>
<updated>2024-05-23T15:15:38+00:00</updated>
<author>
<name>Roman Arutyunyan</name>
<email>arut@nginx.com</email>
</author>
<published>2024-05-23T15:15:38+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=ea8270c6142869367c5608bff92df9f5b3f32d37'/>
<id>ea8270c6142869367c5608bff92df9f5b3f32d37</id>
<content type='text'>
Previously chain links could sometimes be dropped instead of being reused,
which could result in increased memory consumption during long requests.

A similar chain link issue in ngx_http_gzip_filter_module was fixed in
da46bfc484ef (1.11.10).

Based on a patch by Sangmin Lee.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Previously chain links could sometimes be dropped instead of being reused,
which could result in increased memory consumption during long requests.

A similar chain link issue in ngx_http_gzip_filter_module was fixed in
da46bfc484ef (1.11.10).

Based on a patch by Sangmin Lee.
</pre>
</div>
</content>
</entry>
<entry>
<title>Support for sendfile(SF_NOCACHE).</title>
<updated>2021-12-27T16:49:26+00:00</updated>
<author>
<name>Maxim Dounin</name>
<email>mdounin@mdounin.ru</email>
</author>
<published>2021-12-27T16:49:26+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=1f01183b9e6658749934313fd72f7f16c1918b54'/>
<id>1f01183b9e6658749934313fd72f7f16c1918b54</id>
<content type='text'>
The SF_NOCACHE flag, introduced in FreeBSD 11 along with the new non-blocking
sendfile() implementation by glebius@, makes it possible to use sendfile()
along with the "directio" directive.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The SF_NOCACHE flag, introduced in FreeBSD 11 along with the new non-blocking
sendfile() implementation by glebius@, makes it possible to use sendfile()
along with the "directio" directive.
</pre>
</div>
</content>
</entry>
<entry>
<title>Simplified sendfile(SF_NODISKIO) usage.</title>
<updated>2021-12-27T16:48:33+00:00</updated>
<author>
<name>Maxim Dounin</name>
<email>mdounin@mdounin.ru</email>
</author>
<published>2021-12-27T16:48:33+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=20c35434ef8d185cc70e8d68ef4730ce08f8b7d6'/>
<id>20c35434ef8d185cc70e8d68ef4730ce08f8b7d6</id>
<content type='text'>
Starting with FreeBSD 11, there is no need to use AIO operations to preload
data into cache for sendfile(SF_NODISKIO) to work.  Instead, sendfile()
handles non-blocking loading data from disk by itself.  It still can, however,
return EBUSY if a page is already being loaded (for example, by a different
process).  If this happens, we now post an event for the next event loop
iteration, so sendfile() is retried "after a short period", as manpage
recommends.

The limit of the number of EBUSY tolerated without any progress is preserved,
but now it does not result in an alert, since on an idle system event loop
iteration might be very short and EBUSY can happen many times in a row.
Instead, SF_NODISKIO is simply disabled for one call once the limit is
reached.

With this change, sendfile(SF_NODISKIO) is now used automatically as long as
sendfile() is enabled, and no longer requires "aio on;".
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Starting with FreeBSD 11, there is no need to use AIO operations to preload
data into cache for sendfile(SF_NODISKIO) to work.  Instead, sendfile()
handles non-blocking loading data from disk by itself.  It still can, however,
return EBUSY if a page is already being loaded (for example, by a different
process).  If this happens, we now post an event for the next event loop
iteration, so sendfile() is retried "after a short period", as manpage
recommends.

The limit of the number of EBUSY tolerated without any progress is preserved,
but now it does not result in an alert, since on an idle system event loop
iteration might be very short and EBUSY can happen many times in a row.
Instead, SF_NODISKIO is simply disabled for one call once the limit is
reached.

With this change, sendfile(SF_NODISKIO) is now used automatically as long as
sendfile() is enabled, and no longer requires "aio on;".
</pre>
</div>
</content>
</entry>
<entry>
<title>Upstream: sendfile_max_chunk support.</title>
<updated>2021-10-29T17:21:54+00:00</updated>
<author>
<name>Maxim Dounin</name>
<email>mdounin@mdounin.ru</email>
</author>
<published>2021-10-29T17:21:54+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=b3b368184b1e3c82da6703e8d7367f38fdc98d1a'/>
<id>b3b368184b1e3c82da6703e8d7367f38fdc98d1a</id>
<content type='text'>
Previously, connections to upstream servers used sendfile() if it was
enabled, but never honored sendfile_max_chunk.  This might result
in worker monopolization for a long time if large request bodies
are allowed.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Previously, connections to upstream servers used sendfile() if it was
enabled, but never honored sendfile_max_chunk.  This might result
in worker monopolization for a long time if large request bodies
are allowed.
</pre>
</div>
</content>
</entry>
<entry>
<title>Negative size buffers detection.</title>
<updated>2018-11-26T15:29:56+00:00</updated>
<author>
<name>Maxim Dounin</name>
<email>mdounin@mdounin.ru</email>
</author>
<published>2018-11-26T15:29:56+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=f4c70589ce2875b67554113dc7fe6efc581444d6'/>
<id>f4c70589ce2875b67554113dc7fe6efc581444d6</id>
<content type='text'>
In the past, there were several security issues which resulted in
worker process memory disclosure due to buffers with negative size.
It looks reasonable to check for such buffers in various places,
much like we already check for zero size buffers.

While here, removed "#if 1 / #endif" around zero size buffer checks.
It looks highly unlikely that we'll disable these checks anytime soon.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
In the past, there were several security issues which resulted in
worker process memory disclosure due to buffers with negative size.
It looks reasonable to check for such buffers in various places,
much like we already check for zero size buffers.

While here, removed "#if 1 / #endif" around zero size buffer checks.
It looks highly unlikely that we'll disable these checks anytime soon.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fixed spelling of logical AND operator, no functional changes.</title>
<updated>2016-12-07T10:54:30+00:00</updated>
<author>
<name>Sergey Kandaurov</name>
<email>pluknet@nginx.com</email>
</author>
<published>2016-12-07T10:54:30+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=fcb2e2864e0c7a245daef1fc11ffedf6598f21df'/>
<id>fcb2e2864e0c7a245daef1fc11ffedf6598f21df</id>
<content type='text'>
Found by PVS-Studio.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Found by PVS-Studio.
</pre>
</div>
</content>
</entry>
<entry>
<title>Threads: task pointer stored in ngx_file_t.</title>
<updated>2016-03-18T03:43:52+00:00</updated>
<author>
<name>Maxim Dounin</name>
<email>mdounin@mdounin.ru</email>
</author>
<published>2016-03-18T03:43:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=ac78b26324a348ebdd108de0c022729b671549fb'/>
<id>ac78b26324a348ebdd108de0c022729b671549fb</id>
<content type='text'>
This simplifies the interface of the ngx_thread_read() function.

Additionally, most of the thread operations now explicitly set
file-&gt;thread_task, file-&gt;thread_handler and file-&gt;thread_ctx,
to facilitate use of thread operations in other places.

(Potential problems remain with sendfile in threads though - it uses
file-&gt;thread_handler as set in ngx_output_chain(), and it should not
be overwritten to an incompatible one.)

In collaboration with Valentin Bartenev.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This simplifies the interface of the ngx_thread_read() function.

Additionally, most of the thread operations now explicitly set
file-&gt;thread_task, file-&gt;thread_handler and file-&gt;thread_ctx,
to facilitate use of thread operations in other places.

(Potential problems remain with sendfile in threads though - it uses
file-&gt;thread_handler as set in ngx_output_chain(), and it should not
be overwritten to an incompatible one.)

In collaboration with Valentin Bartenev.
</pre>
</div>
</content>
</entry>
<entry>
<title>Copy filter: fixed sendfile aio handlers to set ctx-&gt;aio.</title>
<updated>2016-03-03T18:14:19+00:00</updated>
<author>
<name>Maxim Dounin</name>
<email>mdounin@mdounin.ru</email>
</author>
<published>2016-03-03T18:14:19+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=00fcc6a987f1d45a9eac91242de7e3527451242e'/>
<id>00fcc6a987f1d45a9eac91242de7e3527451242e</id>
<content type='text'>
Sendfile handlers (aio preload and thread handler) are called within
ctx-&gt;output_filter() in ngx_output_chain(), and hence ctx-&gt;aio cannot
be set directly in ngx_output_chain().  Meanwhile, it must be set to
make sure loop within ngx_output_chain() will be properly terminated.

There are no known cases that trigger the problem, though in theory
something like aio + sub filter (something that needs body in memory,
and can also free some memory buffers) + sendfile can result in
"task already active" and "second aio post" alerts.

The fix is to set ctx-&gt;aio in ngx_http_copy_aio_sendfile_preload()
and ngx_http_copy_thread_handler().

For consistency, ctx-&gt;aio is no longer set explicitly in
ngx_output_chain_copy_buf(), as it's now done in
ngx_http_copy_thread_handler().
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Sendfile handlers (aio preload and thread handler) are called within
ctx-&gt;output_filter() in ngx_output_chain(), and hence ctx-&gt;aio cannot
be set directly in ngx_output_chain().  Meanwhile, it must be set to
make sure loop within ngx_output_chain() will be properly terminated.

There are no known cases that trigger the problem, though in theory
something like aio + sub filter (something that needs body in memory,
and can also free some memory buffers) + sendfile can result in
"task already active" and "second aio post" alerts.

The fix is to set ctx-&gt;aio in ngx_http_copy_aio_sendfile_preload()
and ngx_http_copy_thread_handler().

For consistency, ctx-&gt;aio is no longer set explicitly in
ngx_output_chain_copy_buf(), as it's now done in
ngx_http_copy_thread_handler().
</pre>
</div>
</content>
</entry>
<entry>
<title>Output chain: free chain links in ngx_chain_writer().</title>
<updated>2015-03-23T18:09:05+00:00</updated>
<author>
<name>Maxim Dounin</name>
<email>mdounin@mdounin.ru</email>
</author>
<published>2015-03-23T18:09:05+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=4df7aa6f92e6c6f70002f72caaef6eb0ba6aea82'/>
<id>4df7aa6f92e6c6f70002f72caaef6eb0ba6aea82</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Output chain: zero size buf alerts in ngx_chain_writer().</title>
<updated>2015-03-23T17:56:58+00:00</updated>
<author>
<name>Maxim Dounin</name>
<email>mdounin@mdounin.ru</email>
</author>
<published>2015-03-23T17:56:58+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sigsegv.uk/nginx.git/commit/?id=9131eb11c9b204cb12991013f84ffe0bfa64ac14'/>
<id>9131eb11c9b204cb12991013f84ffe0bfa64ac14</id>
<content type='text'>
Now we log a "zero size buf in chain writer" alert if we encounter a zero
sized buffer in ngx_chain_writer(), and skip the buffer.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Now we log a "zero size buf in chain writer" alert if we encounter a zero
sized buffer in ngx_chain_writer(), and skip the buffer.
</pre>
</div>
</content>
</entry>
</feed>
