Hesai XT16 LiDAR

[deniwaki]

I want to use the XT16 in Nebula,　Is there a good way to do it?　
Or any update plan for use XT16 near future?

[mojomex]

Thank you for your interest in Nebula! Sadly, we are currently not supporting XT16 and do not currently have a model to test with.
However, seeing as the datasheets of XT32 and XT16 are (as far as I can tell) identical except for the 32 vs. 16 channels, it should be fairly quick to work off of the existing XT32 code to add support.
This would involve:

• Adding XT16 as a recognized model name in a few places (basically search and copy XT32)
• Adding an XT16 sensor definition (copy XT32)
• Perform testing (connect to the sensor, show pointcloud output)
• If everything looks good, record /pandar_packets while XT16 is running for us to confirm it is working.
• Add a unit test case based on the recorded packets

If you specifically want to use Nebula with XT16 and are willing to create a PR along those lines, we would be very happy to work with you on this.

[jemmmel]

We are also using XT-16, I may look into this

[drwnz]

@deniwaki @jemmmel How did you get on with testing XT-16? If you managed to implement in Nebula, we'd be happy to review a PR. Otherwise, I'll close this issue as we have do not have access to a device to test at TIER IV.

[jemmmel]

Sorry, I have not had time to look at this lately. If I do, I will request to re-open it

[imeshsps]

@mojomex I was able to do some modifications on XT32 implementation and get following outputs from XT16.

$ ros2 launch nebula_ros hesai_launch_all_hw.xml sensor_model:=PandarXT16

[INFO] [launch]: All log files can be found below /home/user/.ros/log/2024-11-28-10-38-10-647809-user-agx-orin-211950
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [hesai_ros_wrapper_node-1]: process started with pid [211951]
[hesai_ros_wrapper_node-1] [INFO] [1732770490.919271248] [hesai_ros_wrapper_node]: Sensor Configuration: Hesai Sensor Configuration:
[hesai_ros_wrapper_node-1] Sensor Model: PandarXT16
[hesai_ros_wrapper_node-1] Frame ID: hesai
[hesai_ros_wrapper_node-1] Host IP: 192.168.123.50
[hesai_ros_wrapper_node-1] Sensor IP: 192.168.123.20
[hesai_ros_wrapper_node-1] Data Port: 2368
[hesai_ros_wrapper_node-1] Return Mode: LastStrongest
[hesai_ros_wrapper_node-1] Frequency: 65535
[hesai_ros_wrapper_node-1] MTU: 1500
[hesai_ros_wrapper_node-1] Use Sensor Time: 0
[hesai_ros_wrapper_node-1] Multicast: disabled
[hesai_ros_wrapper_node-1] GNSS Port: 10110
[hesai_ros_wrapper_node-1] Rotation Speed: 600
[hesai_ros_wrapper_node-1] Sync Angle: 0
[hesai_ros_wrapper_node-1] Cut Angle: 0
[hesai_ros_wrapper_node-1] FoV Start: 0
[hesai_ros_wrapper_node-1] FoV End: 360
[hesai_ros_wrapper_node-1] Dual Return Distance Threshold: 0.1
[hesai_ros_wrapper_node-1] Calibration Path: /home/user/rosws/install/nebula_decoders/share/nebula_decoders/calibration/hesai/PandarXT16.csv
[hesai_ros_wrapper_node-1] PTP Profile: IEEE_1588v2
[hesai_ros_wrapper_node-1] PTP Domain: 0
[hesai_ros_wrapper_node-1] PTP Transport Type: UDP/IP
[hesai_ros_wrapper_node-1] PTP Switch Type: TSN
[hesai_ros_wrapper_node-1] [INFO] [1732770490.919631861] [hesai_ros_wrapper_node]: UDP-only mode is enabled. Settings checks, synchronization, and diagnostics publishing are disabled.
[hesai_ros_wrapper_node-1] [INFO] [1732770490.920580897] [IoContext::IoContext]: Thread(s) Created: 1
[hesai_ros_wrapper_node-1] [INFO] [1732770490.922213526] [hesai_ros_wrapper_node.HesaiDecoder]: Using calibration data from /home/user/rosws/install/nebula_decoders/share/nebula_decoders/calibration/hesai/PandarXT16.csv
[hesai_ros_wrapper_node-1] [INFO] [1732770490.922249846] [hesai_ros_wrapper_node.HesaiDecoder]: Starting Decoder
[hesai_ros_wrapper_node-1] [INFO] [1732770490.970740425] [hesai_ros_wrapper_node.HesaiDecoder]: . Wrapper=OK
[hesai_ros_wrapper_node-1] [INFO] [1732770490.971526867] [hesai_ros_wrapper_node.HwInterface]: Starting UDP receiver
[hesai_ros_wrapper_node-1] [WARN] [1732770492.015865025] [hesai_ros_wrapper_node]: New subscription discovered on topic '/pandar_points', requesting incompatible QoS. No messages will be sent to it. Last incompatible policy: RELIABILITY_QOS_POLICY

The following output is from the /pandar_points topic.

$ros2 topic echo /pandar_points

---
header:
  stamp:
    sec: 1564025028
    nanosec: 99723008
  frame_id: hesai
height: 1
width: 16795
fields:
- name: x
  offset: 0
  datatype: 7
  count: 1
- name: y
  offset: 4
  datatype: 7
  count: 1
- name: z
  offset: 8
  datatype: 7
  count: 1
- name: intensity
  offset: 12
  datatype: 2
  count: 1
- name: return_type
  offset: 13
  datatype: 2
  count: 1
- name: channel
  offset: 14
  datatype: 4
  count: 1
- name: azimuth
  offset: 16
  datatype: 7
  count: 1
- name: elevation
  offset: 20
  datatype: 7
  count: 1
- name: distance
  offset: 24
  datatype: 7
  count: 1
- name: time_stamp
  offset: 28
  datatype: 6
  count: 1
is_bigendian: false
point_step: 32
row_step: 537440
data:
- 179
- 7
- 154
- '...'
is_dense: true
---

I'm trying to visualize this in rviz but initially it was saying no actual frame map. I tried publishing static tf from map to hesai but still can't see any data on rviz. Can you please help with this?

[mojomex]

Hi @imeshsps, it's super nice to hear you're working on XT16 support!

The ros2 topic echo log looks good, so changing the Fixed Frame field in RViz from map to hesai should be all that's needed!
Nebula's default frame ID for Hesai sensors is hesai (see line 8 of your log above). You can change it in the sensor's .param.yaml file 🙌

[jemmmel]

Hey all, I have also had some success with the XT16 and should have something ready next week. Looks like good progress already though

[jemmmel]

Currently getting one warning related to payload size

[INFO] [launch]: All log files can be found below /home/nagoya/.ros/log/2024-11-28-16-42-27-776466-nagoya-ASUS-TUF-Gaming-F15-FX507VV4-FX507VV4-14598
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [hesai_ros_wrapper_node-1]: process started with pid [14599]
[hesai_ros_wrapper_node-1] [INFO 1732779747.839804660] [hesai_ros_wrapper_node]: Sensor Configuration: Hesai Sensor Configuration:
[hesai_ros_wrapper_node-1] Sensor Model: PandarXT16
[hesai_ros_wrapper_node-1] Frame ID: hesai
[hesai_ros_wrapper_node-1] Host IP: 255.255.255.255
[hesai_ros_wrapper_node-1] Sensor IP: 192.168.1.201
[hesai_ros_wrapper_node-1] Data Port: 2368
[hesai_ros_wrapper_node-1] Return Mode: LastStrongest
[hesai_ros_wrapper_node-1] Frequency: 0
[hesai_ros_wrapper_node-1] MTU: 1500
[hesai_ros_wrapper_node-1] Use Sensor Time: 0
[hesai_ros_wrapper_node-1] Multicast: disabled
[hesai_ros_wrapper_node-1] GNSS Port: 10110
[hesai_ros_wrapper_node-1] Rotation Speed: 600
[hesai_ros_wrapper_node-1] Sync Angle: 0
[hesai_ros_wrapper_node-1] Cut Angle: 0
[hesai_ros_wrapper_node-1] FoV Start: 0
[hesai_ros_wrapper_node-1] FoV End: 360
[hesai_ros_wrapper_node-1] Dual Return Distance Threshold: 0.1
[hesai_ros_wrapper_node-1] Calibration Path: /home/nagoya/nebula_ws/install/nebula_decoders/share/nebula_decoders/calibration/hesai/PandarXT16.csv
[hesai_ros_wrapper_node-1] PTP Profile: IEEE_1588v2
[hesai_ros_wrapper_node-1] PTP Domain: 0
[hesai_ros_wrapper_node-1] PTP Transport Type: UDP/IP
[hesai_ros_wrapper_node-1] PTP Switch Type: TSN (HesaiRosWrapper() at /home/nagoya/nebula_ws/src/nebula/nebula_ros/src/hesai/hesai_ros_wrapper.cpp:40)
[hesai_ros_wrapper_node-1] [INFO 1732779747.839965434] [IoContext::IoContext]: Thread(s) Created: 1 (IoContext() at /home/nagoya/nebula_ws/src/transport_drivers/io_context/src/io_context.cpp:44)
[hesai_ros_wrapper_node-1] [INFO 1732779747.840244039] [TcpSocket::open]: connected (open() at /home/nagoya/nebula_ws/src/transport_drivers/tcp_driver/src/tcp_socket.cpp:531)
[hesai_ros_wrapper_node-1] 192.168.1.201:9347
[hesai_ros_wrapper_node-1] [WARN 1732779747.841259869] [hesai_ros_wrapper_node]: Sensor returned longer payload than expected. Truncating and parsing anyway. (CheckSizeAndParse() at /home/nagoya/nebula_ws/src/nebula/nebula_hw_interfaces/src/nebula_hesai_hw_interfaces/hesai_hw_interface.cpp:1439)
[hesai_ros_wrapper_node-1] [INFO 1732779747.861800554] [hesai_ros_wrapper_node]: current lidar sync: 1 (PrintInfo() at /home/nagoya/nebula_ws/src/nebula/nebula_hw_interfaces/src/nebula_hesai_hw_interfaces/hesai_hw_interface.cpp:1324)
[hesai_ros_wrapper_node-1] [INFO 1732779747.861882661] [hesai_ros_wrapper_node]: current lidar sync_angle: 0 (PrintInfo() at /home/nagoya/nebula_ws/src/nebula/nebula_hw_interfaces/src/nebula_hesai_hw_interfaces/hesai_hw_interface.cpp:1324)
[hesai_ros_wrapper_node-1] [INFO 1732779747.861916518] [hesai_ros_wrapper_node]: current configuration sync_angle: 0 (PrintInfo() at /home/nagoya/nebula_ws/src/nebula/nebula_hw_interfaces/src/nebula_hesai_hw_interfaces/hesai_hw_interface.cpp:1324)
[hesai_ros_wrapper_node-1] [INFO 1732779748.005295839] [hesai_ros_wrapper_node]: Trying to set Clock source to PTP (PrintInfo() at /home/nagoya/nebula_ws/src/nebula/nebula_hw_interfaces/src/nebula_hesai_hw_interfaces/hesai_hw_interface.cpp:1324)
[hesai_ros_wrapper_node-1] [INFO 1732779748.106083371] [hesai_ros_wrapper_node]: Trying to set PTP Config: IEEE_1588v2, Domain: 0, Transport: UDP/IP, Switch Type: TSN via TCP (PrintInfo() at /home/nagoya/nebula_ws/src/nebula/nebula_hw_interfaces/src/nebula_hesai_hw_interfaces/hesai_hw_interface.cpp:1324)
[hesai_ros_wrapper_node-1] [INFO 1732779748.213214034] [hesai_ros_wrapper_node.HwMonitor]: Inventory info: sn: XT38C9519038CB53, date_of_manufacture: 2023-08-06, mac: ec:9f:0d:01:6a:91, sw_ver: 1.0.10, hw_ver: 1.0.0, control_fw_ver: 1.2.7, sensor_fw_ver:    1.5.7, zero_angle_offset: 8794, model: PandarXT16, motor_type: unidirectional, num_of_lines: 16 (HesaiHwMonitorWrapper() at /home/nagoya/nebula_ws/src/nebula/nebula_ros/src/hesai/hw_monitor_wrapper.cpp:55)
[hesai_ros_wrapper_node-1] [INFO 1732779748.213312324] [hesai_ros_wrapper_node.HwMonitor]: Hardware ID: PandarXT16: XT38C9519038CB53 (HesaiHwMonitorWrapper() at /home/nagoya/nebula_ws/src/nebula/nebula_ros/src/hesai/hw_monitor_wrapper.cpp:62)
[hesai_ros_wrapper_node-1] [INFO 1732779748.215094648] [hesai_ros_wrapper_node]: Downloaded calibration data from sensor. (get_calibration_data() at /home/nagoya/nebula_ws/src/nebula/nebula_ros/src/hesai/hesai_ros_wrapper.cpp:471)
[hesai_ros_wrapper_node-1] [INFO 1732779748.215409746] [hesai_ros_wrapper_node]: Saved downloaded data to /home/nagoya/nebula_ws/install/nebula_decoders/share/nebula_decoders/calibration/hesai/PandarXT16_from_sensor_192.168.1.201.csv (get_calibration_data() at /home/nagoya/nebula_ws/src/nebula/nebula_ros/src/hesai/hesai_ros_wrapper.cpp:476)
[hesai_ros_wrapper_node-1] [INFO 1732779748.499850349] [hesai_ros_wrapper_node.HesaiDecoder]: Using calibration data from /home/nagoya/nebula_ws/install/nebula_decoders/share/nebula_decoders/calibration/hesai/PandarXT16_from_sensor_192.168.1.201.csv (HesaiDecoderWrapper() at /home/nagoya/nebula_ws/src/nebula/nebula_ros/src/hesai/decoder_wrapper.cpp:36)
[hesai_ros_wrapper_node-1] [INFO 1732779748.499931152] [hesai_ros_wrapper_node.HesaiDecoder]: Starting Decoder (HesaiDecoderWrapper() at /home/nagoya/nebula_ws/src/nebula/nebula_ros/src/hesai/decoder_wrapper.cpp:39)
[hesai_ros_wrapper_node-1] [INFO 1732779748.512553625] [hesai_ros_wrapper_node.HesaiDecoder]: . Wrapper=OK (HesaiDecoderWrapper() at /home/nagoya/nebula_ws/src/nebula/nebula_ros/src/hesai/decoder_wrapper.cpp:67)
[hesai_ros_wrapper_node-1] [INFO 1732779748.512756871] [hesai_ros_wrapper_node]: Starting UDP receiver (PrintInfo() at /home/nagoya/nebula_ws/src/nebula/nebula_hw_interfaces/src/nebula_hesai_hw_interfaces/hesai_hw_interface.cpp:1324)

I believe this might be related to max_scan_buffer_points which I changed to 128000 (Max data points x2?). I will keep looking into it. Otherwise, pointcloud looks good.

Edit: Just found this in the docs - "max_scan_buffer_points = bytes_per_second / lowest_supported_frequency", I will fix this

[imeshsps]

@mojomex The issue was with the Reliability Policy setting. I was able to get the data on Rviz.

[mojomex]

@jemmmel @imeshsps Thanks for your guys' good work!
Are you collaborating or did you come up with contributions independently?

@jemmmel For the payload size warning, it is related to parsing of TCP metadata, such as diagnostics and sensor config. Hesai has documents for that, but they're not public and I don't have any for XT16.

For now, if diagnostics look all good and the printed responses from the sensor look reasonable (they do), you can go ahead and create a PR :D

Please make sure to follow our contribution guidelines for a smooth process. Thank you!

[jemmmel]

@mojomex Ok thanks for the info! We are working independently, just good timing I guess.
I was wondering about the tests, I am yet to add one for this sensor or should I just do the PR ASAP?

[mojomex]

@jemmmel Yes, please add tests. You can do something similar to the other Hesai sensors in nebula_tests.
They test the decode functionality, and ground-truth data can be generated by recording a rosbag of scan messages from /pandar_packets and then converting one of them to PCD. The test suite then compares the ground-truth pointcloud with the one Nebula decodes from the rosbag.

nebula_examples can be used for the conversion, but the script is not up-to-date. See #236 for a working version 🙇

Also, if you could record a PCAP (with Wireshark for example) with all TCP, UDP, HTTP data (no filters), where you start Nebula and let it run for a few seconds, that would help our in-progress TCP protocol unit tests (#234) greatly!