ProxyRouterEndpoints should have retry capability for certain categories of errors, mainly connection errors.
For example, there is the possibility that a keep-alive pooled connection for talking to a downstream service is closed in the short time between pulling the connection from the pool and executing the call, for example due to idleness if the connection was sitting unused in the pool for a while. The downstream service is not doing anything wrong by closing the connection, but there's nothing we can do to prevent this case and a simple retry of the call on a different connection should work fine (the likelihood of getting 2-3 of these errors in a row for a single request should be very small).
We need to identify the exceptions that should trigger retry, implement the retry feature, and expose knobs for the retry capability in ServerConfig (e.g. number of retries allowed before giving up and bubbling up the exception).
Note that this retry feature can only work if the exception occurs while trying to send the first chunk downstream (the non-payload chunk that contains headers, etc) - once even a single chunk of payload is sent downstream then Riposte releases it and it is garbage collected, so there's no way to replay any of the payload.
ProxyRouterEndpoints should have retry capability for certain categories of errors, mainly connection errors.
For example, there is the possibility that a keep-alive pooled connection for talking to a downstream service is closed in the short time between pulling the connection from the pool and executing the call, for example due to idleness if the connection was sitting unused in the pool for a while. The downstream service is not doing anything wrong by closing the connection, but there's nothing we can do to prevent this case and a simple retry of the call on a different connection should work fine (the likelihood of getting 2-3 of these errors in a row for a single request should be very small).
We need to identify the exceptions that should trigger retry, implement the retry feature, and expose knobs for the retry capability in
ServerConfig(e.g. number of retries allowed before giving up and bubbling up the exception).Note that this retry feature can only work if the exception occurs while trying to send the first chunk downstream (the non-payload chunk that contains headers, etc) - once even a single chunk of payload is sent downstream then Riposte releases it and it is garbage collected, so there's no way to replay any of the payload.