Implement a strict RFC 8941 Structured Fields Dictionary parser for the
RFC 9218 Extensible Prioritization Scheme, shared by HTTP/2 (and, later,
HTTP/3).
New files:
- ngx_http_priority.h: the ngx_http_priority_t and
ngx_http_priority_state_t types and the parse/state API.
- ngx_http_priority.c: the parser plus ngx_http_priority_state_init()
and ngx_http_priority_state_update() (client/server merge).
The field is parsed strictly: a syntax error fails the whole field
(RFC 8941, Section 4.2, treats it as absent), and only on a successful
parse are the RFC 9218, Section 4, rules applied, ignoring unknown
members, out-of-range urgency and members of an unexpected type. On
error the caller keeps any previously signalled priority. The parser
bounds the number of members, inner-list items and parameters, and the
length of individual strings, tokens and byte sequences, that it will
accept.
The sources are built only with HTTP/2 (for now), alongside the Huffman
sources, rather than unconditionally into the core HTTP module.
Assisted-by: opencode:claude-opus-4.8
Since 2fb32ff (1.27.5), the connection frame threshold degraded to a
few or zero frames when using a small stream buffer. The previously
hardcoded minimum of 10000 frames is now used.
When set to zero, this resulted in a buffer with no room to read into.
As there is no meaningful interpretation of a zero-sized buffer, such
configuration is explicitly rejected.
Reported by kh-lee
OpenSSL 1.0.2+ validates IP addresses only against iPAddress
subjectAltName entries via X509_check_ip(). For older OpenSSL
versions, equivalent validation is now performed.
Note that dNSName entries are still logged when checking an IP
address, they are not used for matching.
Co-authored-by: Sergey Kandaurov <pluknet@nginx.com>
Previously, ASN1_STRING_length() was used as a length for "%*s" format
specifier, which is wrong, since string length is expected to be "size_t",
and ASN1_STRING_length() returns "int".
Origin: <https://freenginx.org/hg/nginx/rev/a540fac7e04a>
Signed-off-by: Eugene Grebenschikov <e.grebenshchikov@f5.com>
It is possible to receive application data together with socket close.
In this case, SSL_read() may return SSL_ERROR_SYSCALL with ECONNRESET,
specifically after OpenSSL sent TLSv1.3 tickets on a closed connection.
Before OpenSSL 4.0, a subsequent SSL_write() returned SSL_ERROR_SYSCALL
with EPIPE in this case. The behaviour was changed to remember the
error state and handle such a call as a protocol error, see for details
https://github.com/openssl/openssl/issues/30631. This makes SSL_write()
return SSL_ERROR_SSL with the empty error queue, which results in
"[crit] ... SSL_write() failed" errors, as caught by ssl_stapling.t.
The fix is to check the result of the last read stored in c->ssl->last,
to avoid calling SSL_write() after a failed read. For consistency, the
check is added to ngx_ssl_sendfile(): SSL_sendfile() resets the error
state the same way SSL_write() does, though it does not seem to be
reachable in http.
Based on a patch by Maxim Dounin.
Signed-off-by: Sergey Kandaurov <pluknet@nginx.com>
Origin: <https://freenginx.org/hg/nginx/rev/ff5becaf5444>
OpenSSL up to 3.0, when a TCP connection is closed by the peer, returns
SSL_ERROR_SYSCALL without an error queue and with errno set to 0. Since
OpenSSL 3.0 with SSL_OP_IGNORE_UNEXPECTED_EOF, which nginx sets, this is
reported as SSL_ERROR_ZERO_RETURN instead.
Closing a connection without the close_notify alert is incorrect, yet
common enough in the real world to be handled as a non-error condition.
Truncation attacks are expected to be prevented at the protocol level;
notably, they are not possible for HTTP/1.x requests at all, and only
possible for HTTP/1.x responses if neither Content-Length nor chunked
transfer encoding is used.
Previously, "ERR_peek_error() == 0" was used to detect this, which is
too broad: it also matches TCP-level errors, such as ECONNRESET, which
are better reported as errors, the way the plain TCP code does it in
ngx_unix_recv().
Now only a clean TCP close is treated as a clean connection close. Most
notably, connection resets now result in "SSL_read() failed" errors.
This change is a modified version of a patch by Maxim Dounin. The
modifications include resolved conflicts and the refined commit log.
Signed-off-by: Sergey Kandaurov <pluknet@nginx.com>
Origin: <https://freenginx.org/hg/nginx/rev/4c0b66b5046f>
If COPY or MOVE methods are enabled, the DAV module assumes that it is
configured for the whole server, and does not try validate that the
destination URI is within the particular location. This, however, does
not work well if it is in fact configured in a non-root location, and
this location uses the "alias" directive, such as in the following
configuration:
location /prefix/ {
dav_methods COPY;
alias /foo/;
}
In particular, if the destination URI is shorter than the aliased
location prefix, calling ngx_http_map_uri_to_path() with such
destination URI used to result in segmentation faults (9739e755b). And
if the destination URI is longer than the aliased location prefix, but
does not match it, calling ngx_http_map_uri_to_path() resulted in
unexpected path values, including ones above the "alias" directory
specified.
The fix is to check destination URI prefix if "alias" is used, similarly
to what we do with "try_files". Additionally, the
ngx_http_map_uri_to_path() function was updated with additional sanity
checking to prevent similar issues.
This change is a modified version of a patch by Maxim Dounin. The
modifications include skipping the prefix check if the URI was
rewritten, which preserves the "alias" diagnostics of
ngx_http_map_uri_to_path().
Signed-off-by: Vadim Zhestikov <v.zhestikov@f5.com>
Origin: <https://freenginx.org/hg/nginx/rev/aae28b083279>
Instead of failing the handshake, it is now accepted and ignored like
any other unknown extension. Unlike stated in RFC 9001, Section 8.2,
this represents less strict behaviour following RFC 8446, Section 4.2.
With the OpenSSL compatibility layer, the transport parameters custom
TLS extension used to be registered only in SSL contexts of virtual
servers reachable over QUIC.
The ClientHello callback introduced in 0373fe5d9 (1.29.2) is called
by OpenSSL between collecting and parsing extensions, so the custom
extensions may not match the expected number if the SSL context was
replaced with SSL_set_SSL_CTX().
The fix is to register the extension in all SSL contexts, including
those that cannot be reached over QUIC, so that the array bounds are
preserved; the compatibility layer as a whole is initialized only
if the configuration has QUIC listeners. The keylog callback,
meaningful for QUIC connections, is kept on QUIC addresses.
Reported by Banny Liao.
The environment may be replaced while parsing configuration, as it
happens with the perl module, which calls ngx_set_environment() when
creating an interpreter. As a result, the control socket was not
inherited on binary upgrade.
The variable is now read before ngx_init_cycle(), similarly to
inherited listening sockets.
Previously, when looking up a nested location after a prefix location
match, the nested regex or predicate match error could be ignored if
more regex or predicate locations were configured at the current level.
Now such error leads to request finalization.
A new /control/config endpoint provides configuration reload function
with feedback returned to the user on the outcome.
Example:
$ curl 127.0.0.1:1234/1/control/config -X PATCH -v | jq
< HTTP/1.1 200 OK
< ...
<
{
"logs": []
}
$ curl 127.0.0.1:1234/1/control/config -X PATCH -v | jq
< HTTP/1.1 422 Unprocessable Entity
< ...
<
{
"logs": [
"...: invalid number of arguments in \"listen\" directive"
]
}
Co-authored-by: Pavel Pautov <p.pautov@f5.com>
A predicate location name starts with '$' and is interpreted as a
string evaluated at runtime, and its value is used to route the
request into the location. A predicate location matches if the
string evaluates to a non-empty value not equal to "0". At each
nesting level, the first matching predicate location in the
configuration order is chosen. Predicate locations are matched
after prefix and regex locations. They are not matched if an exact
or a regex location matched, or if the longest matching prefix location
has the "^~" modifier. Predicate locations can be nested inside
other location types and can contain nested locations of their own.
Example:
map $arg_foo $pred {
bar 1;
qux 1;
}
location $pred {
root html;
}
When a body filter returned an error, ngx_event_pipe_drain_chains()
returned all buffers to p->free_raw_bufs, including those in p->busy,
which are still owned by the downstream. If upstream EOF was detected
in the same ngx_event_pipe() pass, ngx_event_pipe_read_upstream() then
freed them while still referenced. With HTTP/2 only the stream's fake
connection is marked on error, so ngx_http_v2_send_output_queue() kept
sending DATA frames pointing at the freed buffers, a use-after-free.
Fixed by keeping the p->busy chains intact, the buffers are freed with
the request pool. This also stops the pipe from reusing them for new
upstream reads, which previously could overwrite response body data
still queued for the client. Additionally, the pipe no longer treats
a downstream it has given up on as merely blocked, so the loop is
still guaranteed to terminate.
Co-authored-by: Roman Arutyunyan <arut@nginx.com>
When the byte range is passed in the client Range header, the range
start is accepted up to NGX_MAX_OFF_T_VALUE, while
ngx_http_slice_range_variable() adds the slice size to it when
formatting the slice end. With start values close to the maximum
this overflowed.
Now such range starts are ignored, similarly to other ranges not
handled by ngx_http_slice_get_start().
If a CRYPTO frame was received after the handshake, ngx_quic_handshake()
ran the handshake completion for the second time. With the check added
in 6f8c520f4, this is limited to short header packets.
Since a client has nothing to send in a CRYPTO frame once the handshake
is complete, such frames are now rejected with the "unexpected_message"
connection error, as does TLS 1.3 for handshake messages received out of
order, and QUIC requires for KeyUpdate.
The length in the "VALUE" line was accepted up to NGX_MAX_OFF_T_VALUE,
while ngx_http_memcached_filter_init() adds the size of the end marker
to it. With lengths close to the maximum this overflowed u->length.
Now such lengths are rejected, similarly to other invalid lengths.
Previously, when the worker file descriptor table was full, a listen
connection might be disabled and a timeout might be set for re-enabling
it. If during this period the worker received the NGX_CMD_QUIT signal,
the accept connection would be recycled, but the timer would remain set.
On timer expiration, an attempt to accept a new connection would result
in "accept4() failed (9: Bad file descriptor)" alert because c->fd was
reset to -1.
Fix is to delete accept timer when closing listen connections.
If a worker process' file descriptor table is full, recvmsg() is not
able to read out the channel fd sent from the master. This results in
different behavior on different platforms.
On Linux, recvmsg() succeeds, returns the payload and drops the fd.
On BSD systems, the first recvmsg() call fails with EMSGSIZE/EMFILE,
but the next call succeeds and returns the payload without the fd.
On Solaris, recvmsg() fails with EMFILE and drops both the payload
and the fd.
Previously, all of these paths resulted in fatal errors. The worker
channel was closed and the worker was no longer capable of receiving
messages from the master. In particular, the NGX_CMD_QUIT command
could not be delivered anymore. As a result, the worker never exited
after nginx reload and kept running and accepting new connections
forever until killed by a signal.
Fix is to consider those errors non-fatal. An additional check is
added to the NGX_CMD_CLOSE_CHANNEL handler to make sure the worker
has received a file descriptor for this entry before. Also, the
MSG_CTRUNC check is removed since a proper ancillary data size
validation is now implemented.
Added the http_json_module with a json_set directive that
extracts values from JSON stored in nginx variables and exposes them
as new nginx variables. Parsed results are cached per request, so
multiple json_set directives using the same source expression trigger
only one parse. The directive is available in the http main context.
Example usage:
http {
json_set $user_name $arg_data user.name;
json_set $user_age $arg_data user.age;
json_set $tag0 $arg_data tags[0];
server {
listen 8080;
location / {
return 200 "name=$user_name age=$user_age "
"tag0=$tag0\n";
}
}
# quoted keys for names containing '.', '[', ']', or that
# are empty; the whole argument is single-quoted so that the
# inner '"' reach the path parser intact
json_set $host $arg_data '["a.b.com"]';
json_set $nested $arg_data 'obj["a.b"].c[2]';
json_set $quoted $arg_data 'obj["a\\"b"]';
}
Co-authored-by: Dmitry Volyntsev <d.volyntsev@f5.com>
RFC 8259 permits any JSON value as a complete document, not just
objects and arrays. The start state now accepts strings, numbers,
and the true/false/null literals as well. A bare top-level number,
having no closing delimiter, is emitted when input ends.
An event-driven JSON parser implementing the grammar of RFC 8259.
The full grammar is recognized: objects, arrays, strings with \uXXXX
escapes (including UTF-16 surrogate pairs), numbers, and the
true/false/null literals. Tokens are reported to a caller-supplied
handler, with nesting bounded by a configurable maximum depth.
Based on previous work by Maxim Dounin.
Co-authored-by: Dmitry Volyntsev <d.volyntsev@f5.com>
Decodes JSON string escape sequences in place as defined by
RFC 8259, Section 7: the two-character escapes and \uXXXX,
with non-BMP code points encoded as UTF-16 surrogate pairs
and emitted as UTF-8.
The directive accepts predicates. When enabled, request body is read
immediately after the headers and before choosing a location.
Body read settings from the server block are used in this case.
Example:
http {
map $http_content_type $is_json {
application/json 1;
}
server {
listen 8000;
client_body_early_read $is_json;
client_max_body_size 256;
client_body_buffer_size 256;
return 200 $request_body;
}
}
The name length of a configured parameter was truncated to one byte
before the per-parameter length field was written, so that a
uwsgi_param name of 256 bytes or longer was encoded with a wrong
length while the name itself was copied in full. The resulting request
was malformed, and upstream servers responded by closing the connection.
While long names are barely used in practice, it is easy to support
them. The length is now properly encoded up to the allowed length.
The name length of a configured parameter was truncated to one byte
before the per-parameter length field was written, so that a
fastcgi_param name of 128 bytes or longer was encoded with a wrong
length while the name itself was copied in full. The resulting request
was malformed, and upstream servers responded by closing the connection.
While long names are barely used in practice, it is easy to support
them. The length is now properly encoded up to the allowed length.
Add ngx_proxy_protocol_v2_write() that writes the fixed 16-byte
PPv2 header and address block, and appends a list of TLVs.
An address block is written using IPv4/IPv6 mixed-family promotion:
the highest family present on either side (IPv6 > IPv4) is used for
the header, and the other address is promoted into it. Both v4-mapped
addresses are demoted to IPv4.
When PROXY protocol v2 is configured and the downstream connection
is SSL, standard PROXY protocol v2 TLVs are populated:
PP2_TYPE_ALPN from SSL_get0_alpn_selected()
PP2_TYPE_AUTHORITY from SSL_get_servername()
PP2_SUBTYPE_SSL_VERSION from SSL_get_version()
PP2_SUBTYPE_SSL_CIPHER from SSL_get_cipher_name()
PP2_SUBTYPE_SSL_CN from client cert subject CN
PP2_SUBTYPE_SSL_SIG_ALG from server certificate
PP2_SUBTYPE_SSL_KEY_ALG from server certificate
The CRC32c TLV is written last and covers the entire header in place.
Co-authored-by: Vadim Zhestikov <v.zhestikov@f5.com>
This change implements the Castagnoli polynomial code 0x11EDC6F41,
as described in RFC4960, Appendix B. It will be used to calculate
PROXY protocol v2 CRC32c checksums.
