Robot libraries cause too much traffic in the network

[miczyg1]

I noticed it when running test on VP46xx when cleaning up the test stands. I was getting inconsistent results when running BPS suite on the platforms, failing sometimes in the suite setup and sometimes in the middle of the tests. Sometimes tests passed sometimes not on multiple invocations of the suite with run.sh script. The cause was the timeout on the HTTP requests to sniepit. After a quick investigation it occurs that every robot library invocation from this repo causes the lib to call its __init__. The __init__ then needs to query snipeit for IPs, etc. In our test repository, there are may calls to the libraries, mainly for power management (these are the most frequent and common probably). Relevant logs:

Initializing library 'osfv.rf.rte_robot.RobotRTE' with arguments [ 192.168.10.244 | True ] failed: ReadTimeout: HTTPConnectionPool(host='192.168.4.202', port=80): Read timed out. (read timeout=10)
Traceback (most recent call last):
  File "/home/miczyg/Projects/Dasharo/open-source-firmware-validation/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
  File "/home/miczyg/Projects/Dasharo/open-source-firmware-validation/venv/lib/python3.13/site-packages/urllib3/connection.py", line 466, in getresponse
    httplib_response = super().getresponse()
  File "/usr/lib64/python3.13/http/client.py", line 1430, in getresponse
    response.begin()
    ~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.13/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.13/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
    [ Message content over the limit has been removed. ]
        timeout=10,
    )
  File "/home/miczyg/Projects/Dasharo/open-source-firmware-validation/venv/lib/python3.13/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/home/miczyg/Projects/Dasharo/open-source-firmware-validation/venv/lib/python3.13/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miczyg/Projects/Dasharo/open-source-firmware-validation/venv/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/miczyg/Projects/Dasharo/open-source-firmware-validation/venv/lib/python3.13/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/miczyg/Projects/Dasharo/open-source-firmware-validation/venv/lib/python3.13/site-packages/requests/adapters.py", line 532, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='192.168.4.202', port=80): Read timed out. (read timeout=10)

Parent suite setup failed:
ReadTimeout: HTTPConnectionPool(host='192.168.4.202', port=80): Read timed out. (read timeout=10)

[miczyg1]

The solution seems to be to init the library once in suite setup and keep the state for the rest of the suite. How to achieve it though, I am not sure yet. But it is kind of unacceptable redundancy to go through init in each keyword call from any lib in this repo. it also causes undesired delays which may end up with test missing some strings in serial output, etc.

[macpijan]

I agree that we should reduce number of the call during testing. But I was able to reproduce it even in much simpler scenarios when checking out devices for monitoring purposes. I would expect the SnipeIT API to handle such throughput well.

There also seems to be API throttling. It is not defined whether it is global or per user - probably the first one, but may need to be verified.

I can see following actions here:

• reduce the number of redundant API calls in tests
• verify the resources of SnipeiT VM - maybe they need to be increased: increased VM resourced with the help of IT team
• verify the API throttling limits
• develop API stress test and make snipeit lib more robust in the event of timeout: Resilient snipeit requests #124

[miczyg1]

But I was able to reproduce it even in much simpler scenarios when checking out devices for monitoring purposes

Yes, if there are many people doing tests, it is as simple as using any test with the libs to trigger that.

[philipanda]

This PR should in theory make the library be instantiated only once per run of robot
#130

According to:
https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#library-scope
Used the decorators from
https://robot-framework.readthedocs.io/en/latest/_modules/robot/api/deco.html

I haven't tested it though

[macpijan]

1. osfv_cli PR: rte_robot.py: Make RobotRTE lib global #130 - merged
2. OSFV repo PR: requirements.txt: bump osfv_cli to v0.6.0 open-source-firmware-validation#907 - merged

I can confirm great improvement in terms of using CLI - e.g. <1.5s instead of 5s for psu get command.

[philipanda]

1. osfv_cli PR: rte_robot.py: Make RobotRTE lib global #130 - merged

   2. OSFV repo PR: requirements.txt: bump osfv_cli to v0.6.0 open-source-firmware-validation#907 - merged

I can confirm great improvement in terms of using CLI - e.g. <1.5s instead of 5s for psu get command.

That's not because of making the library global, but because of caching the items from snipeit instead of downloading all of our assets four times on every operation. By jsut making it global I wasn't able to see any significant improvements in OSFV