Previously, a 32-bit overflow could happen while validating atom entries
count. This allowed processing of an invalid atom with entrires beyond
its boundaries with reads and writes outside of the allocated mp4 buffer.
Reported by Prabhav Srinath (sprabhav7).
Previously, data validation checks did not cover the cases when the output
contained empty buffers. Such buffers are considered illegal and produce
"zero size buf in output" alerts. The change rejects the mp4 files which
produce such alerts.
Also, the change fixes possible buffer overread and overwrite that could
happen while processing empty stco and co64 atoms, as reported by
Pavel Kohout (Aisle Research) and Tim Becker.
Updates the proxy module to use HTTP/1.1 as the default protocol when
communicating with upstream servers. This change unlocks features
such as persistent connections and chunked transfer encoding. Configurations
that require HTTP/1.0 can still override the protocol explicitly.
Keepalive is now automatically enabled in the "local" mode for upstreams
defined in configuration files. Cached keepalive connections are no longer
shared between different locations referencing the same explicit upstream
unless keepalive is explicitly configured without the "local" parameter.
To disable keepalive entirely, use keepalive 0; inside the upstream block.
To allow sharing cached connections between locations, configure
keepalive <max_cached>; without the "local" parameter.
The new "local" parameter prevents sharing cached keepalive connections
between location blocks. Connections are now reused only within the same
location.
When configured, it enables Multipath TCP support on a listen socket.
As of now it works on Linux starting with Linux 5.6 and glibc 2.32,
where it is enabled with an IPPROTO_MPTCP socket(2) protocol.
To avoid EADDRINUSE errors in bind() and listen() when transitioning
between sockets with different protocols, SO_REUSEPORT is set on both
sockets. See f7f1607bf for potential implications.
Based on previous work by Maxime Dourov and Anthony Doeraene.
Previously, ctx->out was not cleared in ngx_http_grpc_reinit_request(),
which could cause queued HTTP/2 control frames (SETTINGS ACK, PING ACK,
WINDOW_UPDATE) to be sent on next upstream.
Additionally, ctx->in and ctx->busy needs to be cleared to avoid similar
problems with buffered request body fixed in cd12dc4f1.
Previously, ctx->out was not cleared in ngx_http_proxy_v2_reinit_request(),
which could cause stale HTTP/2 control frames (SETTINGS ACK, PING ACK,
WINDOW_UPDATE) queued for the old upstream connection to be sent to a new
upstream connection during a retry.
Previously, the expiration timer for learned session was not started
until a new session is created. This could lead to the sessions being
active past the expiration time.
RFC 6265 defines "Max-Age" cookie attribute in section 5.2.2.
If the "expires" option is passed to the "sticky" directive, "max-age"
attribute will appear in cookies set by the module with corresponding
value in seconds.
For the special "max" value of the "expires" option, corresponding "max-age"
attribute value will be set to 315360000 seconds (10 years, similar to
how its done in headers_filter module for the "Cache-Control" header).
While peer is draining, only sticky requests are served and the peer is never
selected to process new requests.
Co-authored-by: Ruslan Ermilov <ru@nginx.com>
In this mode, nginx "learns" which client uses which proxied server by
analyzing headers of client requests and proxied server responses.
For example, a proxied server may start sessions by issuing the "Set-Cookie"
header field to set cookie 'sid' and returning clients will bring the cookie
with the same name.
The following configuration may be used to handle this case:
upstream u1 {
server 127.0.0.1:8080;
server 127.0.0.1:8081;
sticky learn timeout=10m zone=sess:1m
create=$upstream_cookie_sid
lookup=$cookie_sid;
}
Co-authored-by: Ruslan Ermilov <ru@nginx.com>
Co-authored-by: Maxim Dounin <mdounin@mdounin.ru>
Sticky sessions allow to route the same client to the same upstream server.
- upstream structures are extended to keep session-related information
- existing balancing modules are updated to provide an id of the selected
server (SID) in pc->sid, and to select the server, given it's SID.
- other balancing modules are allowed to set the pc->hint value to choose
the desired peer. The sticky module will not change the hint if it's
already set.
- the feature is enabled by default and can be disabled with the
"--without-http_upstream_sticky" switch of the configure script.
The following configuration can be used to enable sticky sessions for
supported balancing modules:
upstream u1 {
server 127.0.0.1:8080;
server 127.0.0.1:8081;
sticky cookie server_id expires=1h domain=.example.com path=/;
}
Co-authored-by: Ruslan Ermilov <ru@nginx.com>
Co-authored-by: Roman Arutyunyan <arut@nginx.com>
Co-authored-by: Maxim Dounin <mdounin@mdounin.ru>
Previously ngx_quic_compat_create_record() could try to encrypt a TLS
record even if encryption context was missing, which resulted in a NULL
pointer dereference.
The context is created by ngx_quic_compat_set_encryption_secret() called
from the OpenSSL keylog callback. If an error occurred in that function,
the context could remain missing. This could happen under memory pressure,
if an allocation failed inside this function.
The fix is to handle errors from ngx_quic_compat_set_encryption_secret()
and set qc->error to trigger an error after SSL_do_handshake() return.
Also, a check for context is added to ngx_quic_compat_create_record()
to avoid other similar issues.
Previously, it was possible to obtain a stateless reset token for a
connection by routing its packet to a wrong worker. This allowed to
terminate the connection.
The fix is to bind stateless reset token to the worker number.
QUIC bpf program previously used struct bpf_map_def which was
deprecated in [1] (kernel version 5.18) and removed in [2] (kernel 6.1).
New-style BTF map definitions were added in [3] (linux kernel 5.3).
Switching the program to BTF is however not necessary since nginx has
its own relocation procedure which allows referencing the real map
structure by its file descriptor allocated earlier. In particular,
bpf instruction BPF_LD_IMM64 (0x18/0x0) is substituted with instruction
BPF_LD_MAP_FD (0x18/0x1) and map_fd is stored in the imm field, see [4]
and [5] for details.
To fix compilation, struct bpf_map_def is changed to a known type (int)
and "extern" is added to indicate external linkage and reduce object
file size.
[1] 93b8952d22
[2] dc567045f1
[3] abd29c9314
[4] https://github.com/torvalds/linux/blob/master/include/linux/filter.h
[5] https://datatracker.ietf.org/doc/rfc9669/
It uses a bloom filter to limit sending Stateless Reset packets no more
than once per second in average for the given address. This allows to
address resource asymmetry from precomputed packets, as well as to limit
potential Stateless Reset exchange.
Per ISO 14496-12 Section 8.6.2, sync sample numbers must be 1-based.
A zero-valued stss entry caused ngx_http_mp4_seek_key_frame() to
return a key_prefix exceeding the samples consumed in the forward
stts pass, which led the backward loop in ngx_http_mp4_crop_stts_data()
to walk past the beginning of the stts data buffer.
The fix validates each stss entry in ngx_http_mp4_seek_key_frame()
and returns an error if a zero sync sample is encountered. The
function signature is changed to return ngx_int_t so it can signal
errors to the caller.
Passing requests to SCGI uses a recalculated size of a request body
as per changes made in d60b8d10f (1.3.9) to support CONTENT_LENGTH
with chunked body requests. This, however, is not compatible with
unbuffered mode introduced later in 7ec559df5 (1.7.11), where such
an approach may not always represent complete request body.
The fix is to use r->headers_in.content_length_n representing either
original Content-Length, if any, or a recalculated value from request
body filters, such as chunked body filter.
Reported by Mufeed VH.
In case "Cookie" header is sent by client, multiple cookie pairs were
incorrectly split by a semicolon and comma.
Now they are split by a semicolon only.
For example, next variables will be found for "Cookie: a=b, c=d; e=f":
- $cookie_a: "b, c=d"
- $cookie_e: "f"
Closes#1042 on GitHub.
Previously, when proxy_cache and keepalive were both enabled with an
HTTP/2 upstream, the second request for a cached resource could fail with
"upstream sent frame for unknown stream" error followed by "cache file
contains invalid header".
This happened because ctx->id was set to 1 in the case when no upstream
connection exists (e.g. cache hit), making the stream id check fail when
the cached response contained frames from a different stream.
The fix is to set ctx->id to 0 when there is no upstream connection,
indicating that no real stream exists, and skip the stream id validation
in this case. Also, ctx->id = 1 is now set only for new connections,
not in the shared done label.
Closes: https://github.com/nginx/nginx/issues/1101
Previously, when connecting to a backend, if the read event handler was
called before the write event handler, and the received response triggered
a next upstream condition, then ngx_http_upstream_reinit() was not called
to clean up the old upstream context. This had multiple implications.
For all proxy modules, since the last upstream response was not cleaned up,
it was mixed with the next upstream response. This could result in ignoring
the second response status code, duplicate response headers or reporting
old upstream header errors.
With ngx_http_grpc_module and ngx_http_proxy_v2_module, ctx->connection
was left dangling since the object it referenced was allocated from the
last upstream connection pool, which was deleted when freeing last upstream.
This lead to use-after-free when trying to reuse this object for the next
upstream.
When connecting to a backend, the connection write event is triggered
first in most cases. However if a response arrives quickly enough, both
read and write events can be triggered together within the same event loop
iteration. In this case the read event handler is called first and the
write event handler is called after it.
SSL initialization for backend connections happens only in the write event
handler since SSL handshake starts with sending Client Hello. Previously,
if a backend sent a quick plain text response, it could be parsed by the
read event handler prior to starting SSL handshake on the connection.
The change adds protection against parsing such responses on SSL-enabled
connections.
For instance, the last_buf flag is used in the http proxy module when
creating HTTP/2 requests to indicate the output is closed. The flag
is inherited in ngx_output_chain() to a destination buffer when reading
the buffered request body. Then it is used in the proxy output filter
to mark the last HTTP/2 DATA frame with END_STREAM.
The problem happens when reusing the destination buffer, such as to
re-read the buffered request body on next upstream, because this buffer
may contain a dirty last_buf value, which breaks sending HTTP/2 request
body in multiple output filter calls.
The flush and last_in_chain flags are cleared for consistency.
If a buffered request body wasn't fully sent, such as on early upstream
response or limited by flow control, unsent buffers could remain in the
input or busy chains when switching to the next upstream server. This
resulted either in the invalid request sent or a stalled connection.
The fix is to reset chains similar to ngx_http_upstream_reinit().
