Skip to content

Add option to not pass HTTP body to lua script #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
t-lo merged 7 commits into from
Jun 9, 2020
Merged

Add option to not pass HTTP body to lua script #4

t-lo merged 7 commits into from
Jun 9, 2020

Conversation

@t-lo
Copy link
Member

@t-lo t-lo commented Jun 9, 2020

This PR adds a command line option to not pass HTTP response bodies to a LUA script. The new option, --lua-dont-pass-body, makes wrk2 perform significantly better when benchmarking endpoints that return large HTTP bodies, like e.g. CDN / caching servers.

The PR is based on the multi-endpoint / multi-server benchmark branch and is a clean implementation of the experimental cache stresser branch, which it obsoletes.

The PR aims to be merged into master so as to not pollute the multi-endpoint branch, which is used in an upstream PR.

Also, a docker file is provided to build a wrk2 container image particularly aimed at running CDN cache stress benchmarks.

how to use

As per the readme:

$ docker build -t cache-stresser -f docker/Dockerfile.cache-stresser .
$ docker run -ti cache-stresser \
            [-c <overall-num-of-concurrent-connections>] \
            [-r <overall-num-of-requests-per-second>] \
            [-d <duration>] \
            http[s]://<hostname>/<cacheserver-ip>/path \
            [ http[s]://<hostname>/<cacheserver-ip>/path ] \
            ...

t-lo added 6 commits August 11, 2019 16:22
@t-lo
Generic multi-endpoint support for upstreaming
b85ddb7 
This change makes multi-endpoint support more generic, with
the motivation of making this feature useful for upstream.

The LUA script 'multiple-endpoints.lua' allows for specifying
an arbitrary number of HTTP(S) endpoints to include in the
benchmark. Endpoints will be connected to in a random, evenly
distributed fashion. After a run finished, the overall latency
will be reported (i.e. there's currently no break-down per
endpoint).

The main purpose of running a benchmark over multiple endpoints
is to allow benchmarking of e.g. a whole web application instead
of the pages and/or restful resources that make up said
application individually.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
@t-lo
cache-stresser: modified wrk2 to stress CDN caches
280f7f3 
This change removes passing HTTP response bodies to LUA scripts
(the body is read and then discarded) and adds a script designed
to stress CDN caches.

Note that the cache stresser needs a high-throughput NIC to actually
stress CDN cahces.
@t-lo
cache-stresser: updated README.md, SCRIPTING
fca36e5
@t-lo
scripts/cache-stresser.lua: support https default port
7bf4301
@t-lo
Merge remote-tracking branch 'upstream/master' into t-lo/upstream-mul…
82c1642 
…ti-endpoint-support
@t-lo
Merge branch 't-lo/upstream-multi-endpoint-support' into nginx-cache-…
bd1ac20 
…stresser
@t-lo t-lo requested a review from margamanterola June 9, 2020 10:40
margamanterola
margamanterola approved these changes Jun 9, 2020
View reviewed changes
Copy link

@margamanterola margamanterola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few nitpicks with regards to the documentation. Otherwise, LGTM

SCRIPTING Outdated
global response -- called with HTTP response data
global response -- called with HTTP response header and body
(with --lua-dont-pass-body command line option
the body is omitted)
Copy link

@margamanterola margamanterola Jun 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this phrasing a bit confusing. How about "(only the response header is passed if --lua-dont-pass-body is enabled)? Or maybe "(the body is omitted if --lua-dont-pass-body is enabled)"?

Copy link
Member Author

@t-lo t-lo Jun 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll rephrase to "(body does not contain actual response data if the command line option --lua-dont-pass-body is used)".

SCRIPTING Outdated
nil after the call to init() wrk will ignore the headers and body.
Using wrk's --lua-dont-pass-body command line option will cause a "dummy"
body to be parsed instead of the actual body. The option is useful e.g. for
benchmarking endpoints which pass large response bodies, like caching
Copy link

@margamanterola margamanterola Jun 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "like caching server" part seems out of place. You could be benchmarking a caching server that caches small responses. I suggest deleting that last part.

Copy link
Member Author

@t-lo t-lo Jun 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, will remove the caching server reference.

docker/README.md Outdated
$ docker build -t cache-stresser -f docker/Dockerfile.cache-stresser .
```

After the build concluded, you may push the image to the registry.
Copy link

@margamanterola margamanterola Jun 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "concluded" sounds weird, it's past tense for something that's actually in the future... I think "is done" would sound better.

Copy link
Member Author

@t-lo t-lo Jun 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole sentence feels out of place; I'll remove the registry reference altogether.

docker/README.md Outdated
cache server in the above example), with an overall of 10 requests per second -
that's 1 request per second, per connection. The stress test duration is 60 seconds by default.

While the test is running it will display throughput per interface via the `nload` utility. After the test concluded,
Copy link

@margamanterola margamanterola Jun 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment about "concluded" here.

Copy link
Member Author

@t-lo t-lo Jun 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, will change the wording.

@t-lo t-lo force-pushed the t-lo/optional-no-passing-of-body-to-lua branch from 14d0bcc to 661217d Compare June 9, 2020 14:19
@t-lo t-lo merged commit 93faa2b into master Jun 9, 2020
@t-lo t-lo deleted the t-lo/optional-no-passing-of-body-to-lua branch June 9, 2020 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@margamanterola margamanterola margamanterola approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@t-lo @margamanterola