Tutorial
By Brett Stoddard, written 12/2/2020
One of the surprising impacts that COVID-19 has had on the world is the explosion of robot technologies. Among them are teleoperation robots, which allow a user to remotely interact with an environment. These are interesting because it allows a person to physically be somewhere without having to worry about germs.
This tutorial presents a technology stack for roboticists who are interested in being able to control their robot over the internet. This technology stack was developed in order to enable human-robot interaction studies in a quarantined world, but it is applicable to any project. Being able to control a robot from anywhere on Earth with an internet connection is both useful and awesome.
TLDR: This is what you will need and need to do
- BYOR (bring your own robot) that’s running ROS
- Setup a Server that runs ROS (using a Flask server node or a roslibjs socket)
- Build a Website UI that sends user input to the server (using HTML/CSS/JavaScript)
- Use a HTTP tunnel to access the server from anywhere on Earth (using Ngrok)
This tutorial combines software tools from robotics and web development to make a user interface that can control a robot remotely. Controlling a robot is fun, but controlling a robot over the internet opens the doors to exciting new applications. Such a system can be built with a website, a server, and, of course, robot(s).
Here’s a high-level networking diagram* that showing one implementation. It illistraits the main interactions between the systems. This tutorial uses Python but a similar approach may work for C++ ROS systems with a ROS node server written in that language. The server both serves the website to the end-user’s browser, and acts as an intermediary between the website and ROS.
.png)
*From a publication under submission.
The requirements for the robots to work with this technology stack is that it runs ROS and can connect to an internet router.
Setting up the server is the meat of this setup (or tofu if you’re vegetarian). It has the dual responsibility of translating the ROS commands from the website, and serving the website when requested by the user’s browser. There are two ways if implementing this: by writing custom Flask routes for HTTP requests, or setting up a socket and using roslibjs.
In my implementation, I added code to the Flask server which translated HTTP requests to the robots (github link here). However, using rosjslib could also be used in this setup which enables realtime bidirectional communication. The advantages and disadvantages are shown below.
| Advantages | Disadvantages | |
| HTTP | -Simplier to setup
-ChairBot example that can be used |
-Requires writing custom routes
-Website must query the server for updates |
| Rosjslib | -ROS can send
-Full access to ROS topics in the browser |
-Very difficult to setup
-May require additional networking debugging (Ngrok) |
Also, in our implementation we had a USB webcamera that was streamed to the web UI which is recommended to help with situational awareness. Here’s a quick tutorial on getting that setup with OpenCV.
Exactly how the website communicated with the server will change depending on how the server was setup (use a socket if socket, use http requests if ). Using web technologies for building . At this point, it should be possible to control the robot locally by connecting to the same router running ROS.
This is the secret sauce that allows the robot to truely be controlled from anywhere on Earth. Ngrok is a great free tool for setting this up. With Ngrok installed, ROS, the server, and the robots up and running, a simple command line argument can be used to generate a URL which can be used to access (note: Ngrok was only tested using HTTP commands and extra steps may be necessary to setup the socket).
This setup was used to build an iteration of the Oregon State University CHARISMA Lab’s ChairBot project. A publication is currently under review detailing the motivation behind developing this technology and HRI aspects of teleoperating a multi-robot furniture system. It is currently being used to evaluate the ChairBot UI safely during the COVID-19 pandemic.