bzm HTTP2 Sampler sends only HTTP/1 requests on some servers

[gh-andre]

This issue was created in response to this comment:

#67 (comment)

The issue is in how BlazeMeter sends HTTP/2 requests, in that it lists preferred protocols in the ALPN data in the reverse order. In addition to this, an invalid protocol h2c is included, which is not supposed to be negotiated over TLS.

Extension: application_layer_protocol_negotiation (len=18)
    Type: application_layer_protocol_negotiation (16)
    Length: 18
    ALPN Extension Length: 16
    ALPN Protocol
        ALPN string length: 8
        ALPN Next Protocol: http/1.1
        ALPN string length: 3
        ALPN Next Protocol: h2c
        ALPN string length: 2
        ALPN Next Protocol: h2

RFC 7301 describes that protocols are supposed to be listed in the descending order of preference, so in this case http/1.1 is listed as the preferred protocol.

https://datatracker.ietf.org/doc/html/rfc7301#section-3.1

RFC 7540 describes that h2c must not be sent over TLS.

https://datatracker.ietf.org/doc/html/rfc7540#section-3.3

Web servers that ignore the invalid h2c and respect the order in ALPN, serve the response in HTTP/1, which is what I'm observing.

Here's the code that can be used to reproduce this behavior. The code is largely taken from HTTP2JettyClient.java.

import java.util.List;
import com.blazemeter.jmeter.http2.core.JMeterJettySslContextFactory;
import org.eclipse.jetty.client.api.ContentResponse;
import org.eclipse.jetty.client.dynamic.HttpClientTransportDynamic;
import org.eclipse.jetty.client.http.HttpClientConnectionFactory;
import org.eclipse.jetty.client.http.HttpClientTransportOverHTTP;
import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.client.HttpClientTransport;
import org.eclipse.jetty.client.MultiplexConnectionPool;
import org.eclipse.jetty.http2.client.http.ClientConnectionFactoryOverHTTP2;
import org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2;
import org.eclipse.jetty.http2.client.HTTP2Client;
import org.eclipse.jetty.io.ClientConnectionFactory;
import org.eclipse.jetty.io.ClientConnector;
import org.eclipse.jetty.io.MappedByteBufferPool;
import org.eclipse.jetty.util.Fields;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.eclipse.jetty.util.thread.QueuedThreadPool;

public class Jetty {
    public static void main(String[] args) throws Exception {
        SslContextFactory.Client sslContextFactory = new JMeterJettySslContextFactory();

        ClientConnector clientConnector = new ClientConnector();
        clientConnector.setSelectors(1);
        clientConnector.setConnectBlocking(false);
        clientConnector.setSslContextFactory(sslContextFactory);

        ClientConnectionFactory.Info http11 = HttpClientConnectionFactory.HTTP11;

        HTTP2Client http2client = new HTTP2Client(clientConnector);
        ClientConnectionFactoryOverHTTP2.HTTP2 http2 = new ClientConnectionFactoryOverHTTP2.HTTP2(
            http2client);

        http2client.setMaxConcurrentPushedStreams(100);
        http2client.setUseALPN(true);
        http2client.setProtocols(List.of("h2", "h2c", "http/1.1"));

        HttpClientTransport transport = new HttpClientTransportDynamic(clientConnector, http11, http2);

        HttpClient client = new HttpClient(transport);
        client.setStrictEventOrdering(false);

        client.start();

        ContentResponse response = client.GET("https://server/");

        System.out.println("Protocol: " + response.getRequest().getVersion());
        System.out.println("Status: " + response.getStatus());
        //System.out.println("Body: " + response.getContentAsString());

        client.stop();
    }
}

If the URL in this test is changed to microsoft.com, you can see in Wireshark how the web server rejects the first Client Hello with http/1.1, 'h2c and 'h2, in this order, and then the next Client Hello is issued with just h2, which succeeds. Other web servers seem to just ignore the order of protocols and pick h2 without failing the Client Hello.

Also, I suspect that h2c is actually injected by Jetty, because when I comment out setProtocols above, I can still see h2c in ALPN.

I compiled this sample against the JARs in the JMeter lib directory, like shown below. Same for running., except for the local test web server it needs a local trust store because BlazeMeter doesn't ignore certificate validation, unlike the native JMeter HTTP Request behaves.

javac -proc:none -cp c:\tools\apache-jmeter-5.6.3\lib\*;c:\tools\apache-jmeter-5.6.3\lib\ext\* Jetty.java

java -cp .\;c:\tools\apache-jmeter-5.6.3\lib\*;c:\tools\apache-jmeter-5.6.3\lib\ext\* Jetty

In my specific case, I was able to build the local version of the plugin with this command, so I can successfully run JMeter, and will understand if this issue is closed because it's tricky to reproduce. I still think that the order of http11 and http2 should be reversed, simply because it corrects the order of protocols in ALPN.

For anyone who runs into the same issue, you can correct this line, like so:

HttpClientTransport transport = new HttpClientTransportDynamic(clientConnector, http2, http11);

, and run this command to build the plugin, which can be dropped into lib/ext in JMeter.

mvn clean install -Dcheckstyle.skip=true -DskipTests

Worth noting that I built this in Java 21, so there's a few errors generated by the style check, mostly because of missing trailing line feeds. Also, some of the tests start failing, probably because of the swapped protocol order. I didn't chase this down because the plugin is working for me in JMeter.

WRT the test environment, BlazeMeter v2.0.6 running in JMeter v5.6.3 on Windows 11 in Java 21, and it sends only HTTP/1 requests to lighttpd v1.4.82.