pyrex.ini: bind-mount /tmp to /tmp #281
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It was observed that the root filesystem on my dev machine had an unusually large amount of I/O during NILRT builds, sometimes sustaining over >50MB/s. This is not expected, because with the exception of the pyrex containers (which one would naively consider to be relatively static beasts), my NILRT build does not use the root filesystem in any way. Some judicious use of `fatrace` indicated that the vast majority of the I/O was coming from e.g. /var/lib/docker/overlay2/dc702bbbb061fbbf13b9d06c36d8ccee398257070e6c0455ff209b656b2db2f3/diff/tmp/ccuwDoBf.o. What's going on is that the tasks are using the container /tmp instead of the system /tmp. On many machines, including mine, /tmp is a tmpfs, and writing instead to the container /tmp causes needless wear on the root filesystem. The fix is to simply always bind-mount the container /tmp to the system /tmp. This change cannot induce any additional build system instability due to to tmpfs-related OOMs, because to a first order, all of our upstream recipes have to deal with this anyway: presumably, not all Yocto developers use pyrex nowadays. Signed-off-by: Rich Tollerton <rich.tollerton@ni.com>
Contributor
Author
|
https://docs.yoctoproject.org/singleindex.html#speeding-up-a-build
My experience at present is that gcc is quite unambiguously not using -pipe. Did we break that configuration somehow? Or did upstream break it recently? |
Contributor
|
I think this change should go to upstream-pyrex to see what they say. But before you do that, does the |
Contributor
Author
|
Opened upstream garmin/pyrex#95. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It was observed that the root filesystem on my dev machine had an unusually large amount of I/O during NILRT builds, sometimes sustaining over >50MB/s. This is not expected, because with the exception of the pyrex containers (which one would naively consider to be relatively static beasts), my NILRT build does not use the root filesystem in any way.
Some judicious use of
fatraceindicated that the vast majority of the I/O was coming from e.g./var/lib/docker/overlay2/dc702bbbb061fbbf13b9d06c36d8ccee398257070e6c0455ff209b656b2db2f3/diff/tmp/ccuwDoBf.o. What's going on is that the tasks are using the container /tmp instead of the system /tmp. On many machines, including mine, /tmp is a tmpfs, and writing instead to the container /tmp causes needless wear on the root filesystem.
The fix is to simply always bind-mount the container /tmp to the system /tmp.
This change cannot induce any additional build system instability due to to tmpfs-related OOMs, because to a first order, all of our upstream recipes have to deal with this anyway: presumably, not all Yocto developers use pyrex nowadays.