example/lava-docker: example job for using a docker shell for testing#111
Draft
galak wants to merge 140 commits intoLinaro:litefrom
Draft
example/lava-docker: example job for using a docker shell for testing#111galak wants to merge 140 commits intoLinaro:litefrom
galak wants to merge 140 commits intoLinaro:litefrom
Conversation
libudev which is utilized by pyudev expects /run/udev/control to exist for udev class events to be available. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This script is meant to be called via udev to passthrough the usb
devices to a container that needs to use them. The script will update
the cgroup file for the container to give (or remove) permissions to the
device, create device nodes in the container, and udev symlinks in the
container. The udev rule is expected to key off a USB serial number for
the device in question.
An example udev rule to utilize this script would look like:
ACTION=="add", ENV{ID_SERIAL_SHORT}=="E00A1029", RUN+="/usr/local/bin/usb-passthrough -a -d %E{ID_SERIAL_SHORT} -i lava-dispatcher"
Where "E00A1029" is the serial number of the device in question, and
"lava-dispatcher" is the container that any associated device nodes will
get setup in.
This script is based on:
https://git.linaro.org/lava/lava-lab.git/tree/shared/lab-scripts/docker/passthrough
The major additions are:
* Handle device removal
* Dynamically create/remove device nodes in container
* Create symlinks in container for udev friend names
The removal logic creates on the fly udev rules to handle the removal
since we can't key off of the serial number.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add a LITE specific dispatcher with pyOCD installed in it. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We aren't using QEMU right now so comment out /dev/kvm Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Update version to get fixes so udev bind/unbind events are seen correctly. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
https://github.com/eiz/udevfw Note: we workaround error on older linux kernels Kernel version 4.15 on Ubuntu shows a behavior of reporting ECONNREFUSED from the send. The data is still sent, so we workaround this by ignoring the ConnectionRefusedError exception we get in python. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add example of how you can setup the docker-compose.yaml to point to a local git checkout of lava for development purposes. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The commit changes the boot action method to print out the version of the pyocd-flashtool utility used by the method. It is non-Debian so it was previously failing with the previous methodology of querying the version number. The changed file is provided as a lava-dispatcher overlay. Signed-off-by: Vincent Wan <vincent.wan@linaro.org>
As a convenience for the developer Signed-off-by: Vincent Wan <vincent.wan@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Move the init of the pyudev.MonitorObserver up before handling of packet thread Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Move to using a queue so it handles the syncronization, etc for us. Also make a subtle change to build the packet in the udev thread so if we support multicasting we only build the packet once and have multiple send threads for each container we would send to. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This change supports dealing with the lifecycle of the container and know how to setup the sendMsgThread for this lifecycle. We handle three cases: 1. Container is up already 2. Container comes up after we are running 3. Container goes down after we are running For case 2/3 we loop so we can handle multiple up/downs with having to restart the script. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Just handle all the open/close for the netfs in the thread. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Make changes to support multiplexing data to multiple containers. We either pass some per thread information as args to the thread (netns_file) or we make the thread and workqueue tracked on a per container basis (introduction of containerClass and containers dict) you can now pass multiple -i options for the set of containers you want events sent to. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Found a pure python implementation of murmur2 hash that we can use to replace the need to pull in pyhash. pyhash is always an option if the performance of the python implementation is an issue, but seems unlikely as what we hash are pretty small strings. Grabbed the python implementation from here: https://github.com/Orochimarufan/cdev/blob/master/cdev/murmurhash2.py Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We forgot to import get_errno from ctypes so we get a NameError if this code path gets executed. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add fetching a version of segger jlink into the dispatcher. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Pull in the host-tools from Zephyr SDK. This also gets us QEMU for ARM as well (plus some other tools from Zephyr that might be useful in the future). For now we symlink openocd into /usr/local/bin to expose it. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We need access to /run/udev for the docker test action. This allows the lava dispatcher to see the full udev info for a device. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The lava-dispatcher code imports somethings from the lava_dispatcher_host dir, so add it in as part of the dirs we would mount of we are pulling in a development version of the lava-dispatcher. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We need device_info[].board_id set for use by docker test action. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This is similar to a change done previously qemu-01.jinja2, to make this device compatible with qemu-from-docker usage. (This device isn't intended for such a usage, but maybe randomly selected by scheduler nonetheless. We'll need to use explicit tags going forward, but for now, this is apparently a least-effort workaround.) Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
For some docker-in-docker scenarios (e.g. qemu-in-docker) sub-containers needs to access downloaded images which are stored in tmp dir. Due to the way docker-in-docker functionality works, it should be the same bind mount across all containers (i.e. dispatcher container and subcontainers spawned by it). And the way to achieve that is to mount the same host path into the containers. For more details, see upstream ticket: https://git.lavasoftware.org/lava/pkg/docker-compose/-/issues/8 This dir appears to be created on the host if it doesn't exist, but for clarity, we also create it in the Makefile "install" target. Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Inform user that script will continue after a key is pressed Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
This was intended to be like that from the beginning, but instead turned out to be duplicated, copy-pasted comment from the previous section in the Makefile. Fix that now. Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Signed-off-by: Rémi Duraffort <remi.duraffort@linaro.org>
Update lava-boards target to configure disco-l475-iot1 device. Signed-off-by: Erwan GOURIOU <erwan.gouriou@linaro.org>
This example is based on existing micropython-interactive.job, but utilizes a recent LAVA feature - using QEMU binary from a Docker image (instead of whatever binary is installed in the dispatcher). (Besides pulling QEMU from Docker image, this sample also downloads test binary from URL by default, to simplify testing against production LAVA, and adjusts various timeouts, as pulling an image from Dockerhub takes additional time). Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
docker-compose and docker somehow use different namespaces for the names, where docker's name is exact and concrete, while docker-compose name is a logical name, from which a conrete is normally formed by namespacing the logical name using a docker-compose project directory (so, there potentially can be different deployments of the same project, residing in different filesystem directories). It works that way, unless exact docker name is explicitly overriden. When we forked lava-docker-compose, it exactly used overriden names. But later they were dropped. We however didn't (cherry-)pick that name, for the fear that it may break our setup infra we built on top of the original project. So, if anything, this change brings us closer to the setup used in upstream master, with the intention that we could pick that upstream change later (and so reduce delta to upstream). Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Different QEMU machines, and applications running on them, have different requirements regarding memory. So, don't hardcode a value in a device config, and instead allow to specify a particular value in a job, in "context:" dictionary. If none is specified, LAVA default of "512" will be used. Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
This commit adds a new Debian-based docker image that can be used for testing TF-M on QEMU using the `mps2_an521` machine target. It also adds a new qemu_mps2_an521/tfm application image for testing purposes, which can be run with QEMU. Signed-off-by: Kevin Townsend <kevin.townsend@linaro.org>
This commit adds a new test job for TF-M regression tests running in QEMU. Signed-off-by: Kevin Townsend <kevin.townsend@linaro.org>
This commit updates the README files to point to the QEMU container on DockerHub as an alternative source. Signed-off-by: Kevin Townsend <kevin.townsend@linaro.org>
This commit sets the amount of SRAM for the AN521 to 16M, and sets the `-device` field when running the TF-M binary. Signed-off-by: Kevin Townsend <kevin.townsend@linaro.org>
This is a simple example job which shows how to use multinode synchronization actions like "lava-send", "lava-wait" with "interactive" tests, to coordinate actions between 2 devices. This job requires just 2 "docker" virtual devices, so runs without real hardware. Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Instead of having adhoc "sleep XX" in the test, switch to the proper multinode synchronization: first, the "host" role waits for the "device" role to boot, before starting to ping it. Then, "device" role waits for "host" to complete its testing (before e.g. go offline). Unfortunately, there's still a need to intersperse adhoc delays for longer multinode syn operation, because default multinode operation timeout is pretty small (30s), and there's no visible way to adjust it. Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Following similar conversion done to zephyr-net-ping-frdm_k64f.job. Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
When we set a template for custom device type with "lavacli device-types template set" command, it ends up in one container on docker-compose system, but actually also expected in others. This is a known upstream issue tracked as https://git.lavasoftware.org/lava/pkg/docker-compose/-/issues/4. We work it around for now by pushing the file manually to "another" container. Currently, this applies to "musca_a" device type, but care should be taken to apply the same workaround to other introduced custom device types. Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Similar to a change previously in another place. Allows us to rely on docker-compose's logical names (and ultimately, get close to upstream and run multiple instances of the setup). Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
*** WIP *** Hacky wip job that uses a docker shell w/a MCU board (frdm-k64f) to run a slimmed down zephyr testsuite. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Contributor
|
Thanks for posting! I assume this depends on #110 ? |
Contributor
Author
It kinda does. unfortunately the latest lava release 2020.07 has issues w/using docker shell + having the lava-dispatcher in a docker container as well. This is being worked on based on discussion with LAVA guys on IRC. |
galak
commented
Aug 13, 2020
|
|
||
| - test: | ||
| docker: | ||
| image: kumargala/ci-run |
Contributor
Author
There was a problem hiding this comment.
I've published the docker image sources here:
https://github.com/galak/docker-ci-run
Meant to be minimal env to be able to run zephyr sanitycheck.
galak
commented
Aug 13, 2020
| - cd zephyr | ||
| - west init -l | ||
| - west config zephyr.base zephyr | ||
| - ./scripts/sanitycheck -i -p frdm_k64f --device-testing --device-serial /dev/ttyACM0 --test-only --west-flash |
Contributor
Author
There was a problem hiding this comment.
Assumption here about the serial device being /dev/ttyACM0. Need to go from something like ANDROID_SERIAL to the device. Here's a quick python script w/hardcoded serial number.
import pyudev
context = pyudev.Context()
board_id = "02400b0131754e45001c0019948500046461000097969900"
for device in context.list_devices(subsystem='tty', ):
if device.properties.get("ID_SERIAL_SHORT"):
if board_id == device.properties["ID_SERIAL_SHORT"]:
print(device.device_node)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
*** WIP ***
Hacky wip job that uses a docker shell w/a MCU board (frdm-k64f) to run
a slimmed down zephyr testsuite.
Signed-off-by: Kumar Gala kumar.gala@linaro.org