[RSDK-12362] Provide Support for Orbbec Gemini 335LE Camera #58

SebastianMunozP · 2025-10-24T19:53:01Z

Providing support for Gemini 335LE cameras, until now, the Orbbec module was designed to exclusively support Astra2 cameras, so this PR involves a redesign of the Orbbec module to support for different Orbbec model cameras, adding support at this instance to Gemini 335LE.

Per documentation, the camera only supports MJPG color stream format natively, it supports RGB variants through the SDK. I'm disabling that option for now until thoroughly tested as well, so only supporting MJPG at this time.

Testing Performed:

Having a Astra2 and an Gemini335LE resources in the same robot, streamed continuously for a day
Successfully tested the firmware update of this camera to the recommended FW for our SDK version
Hot unplugging/replugging Gemini and Astra2 cameras
Tested the following color->depth resolutions using the supported MJPG, Y16 formats.

- Aspect Ratio 16:10:
	- 1280 X 800
		- 1280 X 1280: OK
		- 848 X 530: OK
		- 640 X 400: OK
		- 424 X 266: OK
		- 320 X 200: OK
	- 848 X 530
		- 848 X 530: OK
		- 640 X 400: OK
		- 424 X 266: OK
		- 320 X 200: OK
	- 640 X 400
		- 424 X 266: OK
		- 320 X 200: OK
- Aspect Ratio 4:3:
	- 640 X 480
		- 640 X 480: OK

Copilot

Pull Request Overview

This PR adds support for the Orbbec Gemini 335LE camera model in addition to the existing Astra 2, implementing a model-based configuration system to handle multiple camera types. The changes include refactoring model-specific configurations, adding firmware update support, Windows registry setup utilities, and improving device discovery for both USB and Ethernet-connected cameras.

Introduced model configuration system with separate configs for Astra 2 and Gemini 335LE
Added firmware update functionality and Windows registry setup as separate modules
Enhanced device discovery to support network-connected cameras

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/module/orbbec_windows_registry.hpp	Header for Windows registry setup utilities
src/module/orbbec_windows_registry.cpp	Implementation of Windows device registry configuration
src/module/orbbec_firmware.hpp	Header declaring firmware update functionality
src/module/orbbec_firmware.cpp	Implementation of firmware download and update process
src/module/orbbec.hpp	Added model configurations, multiple model support, and updated method signatures
src/module/orbbec.cpp	Refactored to use model configs, added validation helpers, improved error handling
src/module/main.cpp	Registered both Astra2 and Gemini335Le models
src/module/discovery.cpp	Enhanced discovery to support network devices and multiple models
CMakeLists.txt	Added new firmware and Windows registry source files to build

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/module/orbbec.cpp

src/module/orbbec_firmware.cpp

Copilot

Pull Request Overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/module/orbbec_firmware.cpp

src/module/orbbec.cpp

src/module/discovery.cpp

… the module

…, preventing reutilizing a camera by multiple resources by the if my_dev->started in startDevice

…ons are not supported instead of silently picking other ones

…until it has been tested, removing support for RGB color stream on Gemini as it didnt work with a 640X480 resolution -> need to investigate which are actually supported, removed raw pointers in favor of const references, adding supported and default formats to each model config

…n nowUs <= imageTimeUs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…ut just moved to its own file

…35LE camera

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

SebastianMunozP · 2025-10-27T18:53:31Z

Hello @williamjhyland , @IanWhalen

Please not that for the time being, we are supporting the following set of resolutions, and only MJPG/Y16 as formats for color/depth streams respectively, is this OK for the customer?

- Aspect Ratio 16:10:
	- 1280 X 800
		- 1280 X 1280: OK
		- 848 X 530: OK
		- 640 X 400: OK
		- 424 X 266: OK
		- 320 X 200: OK
	- 848 X 530
		- 848 X 530: OK
		- 640 X 400: OK
		- 424 X 266: OK
		- 320 X 200: OK
	- 640 X 400
		- 424 X 266: OK
		- 320 X 200: OK
- Aspect Ratio 4:3:
	- 640 X 480
		- 640 X 480: OK

williamjhyland · 2025-10-27T19:47:02Z

Checking regarding customer commentary.

src/module/orbbec.cpp

oliviamiller · 2025-10-28T14:21:59Z

src/module/discovery.cpp

+                deviceInfoString << "Device " << (i + 1) << " - Name: " << deviceName << ", Serial: " << serialNumber
+                                 << ", Connection: " << connectionType;
+                if (!ipAddress.empty()) {
+                    deviceInfoString << ", IP: " << ipAddress;


For ethernet cameras does the IP address need to be in the config?

Nope, the SDK takes care of that, it has its own "discovery service", it detects all Orbbec cameras reachable therough USB or the network

oliviamiller · 2025-10-28T14:23:00Z

src/module/discovery.cpp

+
+                VIAM_RESOURCE_LOG(info) << "Successfully configured device " << (i + 1) << " with serial: " << serialNumber;
+            } catch (ob::Error& deviceError) {
+                VIAM_RESOURCE_LOG(warn) << "Failed to get device info for device " << (i + 1) << ": " << deviceError.what();


Maybe this should be an error log?

Agreed, raised to error

oliviamiller · 2025-10-28T14:23:38Z

src/module/discovery.cpp

+        VIAM_RESOURCE_LOG(warn) << "Discovery failed - check network connectivity for Ethernet cameras or USB connection for USB cameras";
+    } catch (...) {
+        VIAM_RESOURCE_LOG(error) << "Failed to discover Orbbec devices: unknown error";
+        VIAM_RESOURCE_LOG(warn) << "Discovery failed - check network connectivity for Ethernet cameras or USB connection for USB cameras";


same with these, they feel more like errors than warnings

Agreed, raised to error

oliviamiller · 2025-10-28T14:25:13Z

src/module/orbbec.cpp

+const OrbbecModelConfig& OrbbecModelConfig::forDevice(const std::string& device_name) {
+    if (device_name.find("Astra2") != std::string::npos || device_name.find("Astra 2") != std::string::npos)
+        return ASTRA2_CONFIG;
+    if (device_name.find("Gemini 335Le") != std::string::npos)


Is it possible the device name could be "Gemini335Le" like the Astra2 check above?

I'm using the exact names that I get from the Device for simplicity.

Just bear in mind that for unexplicable reasons, Astra2 can return to different names depending on the way you query, either "Orbbec Astra2" or "Orbbec Astra 2", hence I added support for multiple names :/

oliviamiller · 2025-10-28T14:25:56Z

src/module/orbbec.cpp

+    throw std::runtime_error("Unsupported Orbbec camera model: " + device_name);
+}
+
+bool OrbbecModelConfig::isSupported(const std::string& device_name) {


Could we combine this with the function above since they pretty much do the same thing?

oliviamiller · 2025-10-28T14:30:07Z

src/module/orbbec.cpp

+    std::string const serial = attrs["serial_number"].get_unchecked<std::string>();
+    if (serial.empty()) {
+        throw std::invalid_argument("serial_number must be a non-empty string");
+    }


could me make a helper for these above checks since the code is repeated in the validateGemini function ?

Or is it possible to combine into only one validate function? Would be harder to mantain a validate function for every model if we add more in the future

Good point, done

oliviamiller · 2025-10-28T14:34:01Z

src/module/orbbec.cpp

+                        buffer << "Device firmware already on version " << model_config_->min_firmware_version;
                        resp.emplace(firmware_key, buffer.str());
-                        break;
+                        VIAM_RESOURCE_LOG(info) << buffer.str();


why do we log the buffer?

There's no harm in adding this to the logs as well, but we may want it to investigate potential issues, specially with customers. This log is not repetitive, so wont be intrusive.

oliviamiller · 2025-10-28T14:35:11Z

src/module/orbbec.cpp

                } else {
-                    VIAM_SDK_LOG(warn) << "VIAM_MODULE_DATA is set to " << viam_module_data
-                                       << " but is not a valid directory, using current working directory to store PCD file";
+                    VIAM_RESOURCE_LOG(warn) << "VIAM_MODULE_DATA is set to " << viam_module_data


did you run into this env var not having a valid directory?

Not really, the only change I'm doing in this part is to use VIAM_RESOURCE_LOG instead of VIAM_SDK_LOG for better clarity in the logs

oliviamiller · 2025-10-28T14:36:10Z

src/module/orbbec.cpp

-        VIAM_SDK_LOG(error) << "Current device does not support hardware depth-to-color "
-                               "alignment.";
-        return;
+        VIAM_SDK_LOG(warn) << "Device " << serialNumber << " does not support hardware depth-to-color alignment, trying software alignment";


why do we do this both in registerDevice and startDevice?

This was present before, which raises a really good point, I restructured this logic as follows:

registerDevice -> Only creates an unconfigured device, containing only its serialNumber and an empty ViamOBDevice.

configureDevice -> New function, takes care of actually configuring the device, i.e. taking the specs from the user and making all adjustments to the device to comply with them

startDevice -> Simply starts the device, nothing more

hexbabe · 2025-10-28T15:13:53Z

README.md

+## Model viam:orbbec:gemini_335le

-### Attributes
+Use [Orbbec cameras](https://www.orbbec.com/gemini-335le/).


wrong label for link?

Improved these messages to

Official Orbbec Astra 2 Camera Webpage.

and

Official Orbbec Gemini 335Le Camera Webpage.

README.md

hexbabe · 2025-10-28T15:16:30Z

src/module/discovery.cpp

+                try {
+                    const auto& modelConfig = orbbec::OrbbecModelConfig::forDevice(deviceName);
+                    viamModelSuffix = modelConfig.viam_model_suffix;
+                } catch (const std::runtime_error& e) {


we're sure all runtime errors = camera unsupported? any more specific error to catch?

hexbabe · 2025-10-28T15:19:43Z

src/module/orbbec.cpp

    int requiredMajor = 0, requiredMinor = 0, requiredPatch = 0;
+
    // ignore any trailing text in the case of a beta or RC version
    sscanf(version.c_str(), "%d.%d.%d*s", &major, &minor, &patch);


what is *s

It means to discard the string after that last integer, in particular

"*" -> means discard
"s" -> means string

put together is to discard the trailing string.

hexbabe · 2025-10-28T15:24:46Z

src/module/orbbec_windows_registry.hpp

+
+namespace orbbec {
+
+namespace windows_registry {


tested on windows?

I disabled this camera for all OSs other than Linux for now, I tested on Mac and the camera could not be reached.

src/module/orbbec_firmware.cpp

hexbabe · 2025-10-28T15:25:37Z

src/module/orbbec_firmware.hpp

+                    viam::sdk::LogSource& logger);
+
+}  // namespace firmware
+}  // namespace orbbec


[nit] newline eof missing

… std::optional when appropriate

… can be used to detect Orbbec cameras directly

…at here

…new function configureDevice and is the only place to handle the device configuration, as the resolutions config logic was duplicated

…README per feedback, fixing some nits

…odule.tar.gz rule

…the moment

SebastianMunozP requested a review from Copilot October 27, 2025 16:59

Copilot AI reviewed Oct 27, 2025

View reviewed changes

src/module/orbbec.cpp Outdated Show resolved Hide resolved

src/module/orbbec.cpp Outdated Show resolved Hide resolved

src/module/orbbec_firmware.cpp Outdated Show resolved Hide resolved

src/module/orbbec_firmware.cpp Outdated Show resolved Hide resolved

SebastianMunozP requested a review from Copilot October 27, 2025 18:28

Copilot AI reviewed Oct 27, 2025

View reviewed changes

SebastianMunozP requested review from hexbabe, oliviamiller and seanavery October 27, 2025 18:43

SebastianMunozP and others added 20 commits October 27, 2025 14:48

[RSDK-12362] Provide Support for Orbbec Gemini 335LE Camera

8d36ded

[[RSDK-12362] Add support for new Gemini in discovery and complete in…

18dcdaa

… the module

[[RSDK-12362] Supported devices are now not hardcoded but centralized…

540dd42

…, preventing reutilizing a camera by multiple resources by the if my_dev->started in startDevice

[[RSDK-12362] Improve handling for supported resolutions

40c5ac4

[[RSDK-12362] Throwing an error if the specified depth/color resoluti…

e6e0e43

…ons are not supported instead of silently picking other ones

[[RSDK-12362] Removing 16:9 support for now

e66a3ce

Returning timeSinceFrameUs logic to previous logic of returning 0 whe…

da15895

…n nowUs <= imageTimeUs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Reenabling checkFirmwareVersion in get_image()

f01134a

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

In Update Firmware making sure the zip name is at least 4 characters

9042e6b

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Actually performing the firmwareUpdate

362f7a4

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

[[RSDK-12362] Returning firmwareUpdate logic to its original state, b…

c2f7cf5

…ut just moved to its own file

[[RSDK-12362] Improving firmware_update logging and responses

5e0be35

[RSDK-12362] Enabled and positively tested firmware update on Gemini3…

0b3c31e

…35LE camera

[RSDK-12362] Updating README.md

260be47

Update src/module/orbbec_firmware.cpp

a745de5

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/module/orbbec_firmware.cpp

a1ed509

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/module/orbbec_firmware.cpp

0caa622

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/module/orbbec.cpp

162a193

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/module/discovery.cpp

767ce34

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

SebastianMunozP force-pushed the support-gemini350LE-camera branch from 544917e to 767ce34 Compare October 27, 2025 18:49

[RSDK-12362] Fixing Windows Build

e8daae1

[RSDK-12362] Creating function distortionTypeToString

4915a34

[RSDK-12362] Making lint happy

30ca885

seanavery reviewed Oct 27, 2025

View reviewed changes

src/module/orbbec.cpp Show resolved Hide resolved

src/module/orbbec.cpp Outdated Show resolved Hide resolved

src/module/orbbec.cpp Show resolved Hide resolved

oliviamiller reviewed Oct 28, 2025

View reviewed changes

hexbabe reviewed Oct 28, 2025

View reviewed changes

SebastianMunozP added 9 commits October 29, 2025 15:06

[RSDK-12362] some warn logs are now errors, removing raw pointers for…

d70a714

… std::optional when appropriate

[RSDK-12362] Take the fix the Orbbec name on the configs so that they…

6f978bb

… can be used to detect Orbbec cameras directly

[RSDK-12362] Orbbec devices can have mutiple names, accounting for th…

fa15ab4

…at here

[RSDK-12362] Making lint happy

7e9cbc7

[RSDK-12362] Improving logic of the code per feedback from PR

bb6720a

[RSDK-12362] Modularizing the creation of Alignment config for SW and HW

6ff60dd

[RSDK-12362] Modularizing startDevice and registerDevice by adding a …

ec79f87

…new function configureDevice and is the only place to handle the device configuration, as the resolutions config logic was duplicated

[RSDK-12362] Fixing a bug to handle when return a nullptr, improving …

605bb3a

…README per feedback, fixing some nits

[RSDK-12362] Making lint happy and adding the lint step to Makefile m…

32087d0

…odule.tar.gz rule

oliviamiller approved these changes Oct 30, 2025

View reviewed changes

seanavery approved these changes Oct 30, 2025

View reviewed changes

[RSDK-12362] Disabling Gemini cameras for hosts other than Linux for …

0bf051b

…the moment

SebastianMunozP merged commit 9a29244 into viam-modules:main Oct 30, 2025
5 checks passed

[RSDK-12362] Provide Support for Orbbec Gemini 335LE Camera #58

[RSDK-12362] Provide Support for Orbbec Gemini 335LE Camera #58

Uh oh!

Conversation

SebastianMunozP commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SebastianMunozP commented Oct 27, 2025

Uh oh!

williamjhyland commented Oct 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

SebastianMunozP commented Oct 24, 2025 •

edited

Loading