SIGN IN SIGN UP

[Bug] Hold the full-duplex header reply until the body is routed (#4266)

* [Bug] Hold the full-duplex header reply until the body is routed

Under FULL_DUPLEX_STREAMED the Router answered the request headers at
once, chose the model at end of stream, and then dropped the routing
header mutations from the body reply, where the gateway ignores them.
Raw Envoy therefore routed on the original headers: the request reached
its default route with the original path and the client's own
Authorization header, even when the body arrived in one write. A request
with trailers got a 500.

When the Router accumulates a full-duplex body, hold the header reply
while the body streams. At end of stream, move the body stage's header
mutations and route-cache clear onto the held reply and send it before
the body reply. Envoy and agentgateway both stream the body while the
header reply is pending in this mode. An immediate response replaces the
held reply. If the Router ends the stream with an error while the reply
is held, send it first, so the header stage's own mutations still apply
under the gateway's failure policy.

Full-duplex request trailers now end the body and, unless the body stage
answers with an immediate response, are answered last; a request that
ends with trailers and no body data still goes through the body stage.
Full-duplex requests with trailers that the Router passes through
(streamed_body off, or skip-processing), which got a 500, now get a
trailers reply. Header-only requests and other body modes reply as
before.

Fixes #3892

Signed-off-by: Colin McNamara <colin@2cups.com>

* Drop the held header reply after a receive error

grpc-go sends the error status itself when a receive fails, for example on
a body chunk over the Router's gRPC message limit, so a held header reply
can no longer reach the gateway; sending it only logged a Canceled error.
Process now drops it on that path, and still sends it before an error the
Router raises itself, such as a recovered panic.

The stream-error test is now table-driven: a panic in the Router sends the
header stage's removals before the error, and nothing is sent after a
receive error. The old test modeled a receive error that a real stream never
lets the Router answer.

Signed-off-by: Colin McNamara <colin@2cups.com>

* [Bug] Add full-duplex E2E for Envoy and agentgateway routing

Adds envoy-full-duplex-routing (streaming profile) and
agentgateway-full-duplex-routing (agentgateway profile): a second
gateway per profile with two provider-mocker backends. One-write,
delayed and trailer requests must reach the backend named by
x-selected-model with the provider path and credential, and an
end-of-stream rejection must reach no backend. provider-mocker now
reports the request path and a digest of the Authorization header.
The doc notes that agentgateway routes on the Router's headers only
with its ExtProc policy in PreRouting.

Signed-off-by: Colin McNamara <colin@2cups.com>

---------

Signed-off-by: Colin McNamara <colin@2cups.com>
Co-authored-by: Binbin Zhang <binbin36520@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
C
Colin McNamara committed
445cc87491dca488a58f3cfeb425b9b2e9af4787
Parent: 3e8f747
Committed by GitHub <noreply@github.com> on 9/28/2026, 11:18:37 AM