Draft
Conversation
… misiug/slurmscripts
…ox-testing into misiug/slurmscripts
… misiug/slurmscripts
… misiug/slurmscripts
… misiug/slurmscripts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Set up a system to record profiles in a slurm cluster environment.
When the profiling option is used for
run_benchmark.sh(-p|--profile), each worker container will run nsys start, wrapping the presto executable. Whenrun_benchmark.shis finished running, it will send a signal to each worker by writing to a file based on that worker_id. Each worker will periodically check it's respective file and will run nsys stop when the signal is received.This means that we currently profile the entire query suite, rather than individual queries - and all profiles are only for one worker. If we want to profile an individual query, it is best to run the
run_benchmark.shwith only that query.This PR is also based on the as-of-yet unmerged slurmscripts PR: #202 so it will need to wait for that to merge first.