Unusable timestamps in D435i/D455 on Tinkerboard

Hi everyone, I am not sure if this issue should be posted here or in the librealsense repo, since I don't really know the origin of the issue. My platform:

Required Info | &nbsp;
-- | --
Camera Model | D435i, D455
Firmware Version | 5.12.13.50
Operating System &amp; Version | Armbian 20.05.7 Focal
Kernel Version (Linux Only) | 5.4.123-rockchip
Platform | ARM
SDK Version | 2.45
Language | C++, Python
Segment | others

<h3>Issue Description</h3>

I am trying to make a D435i work on the Tinkerboard S with Focal, which only offers USB2. I compiled both SDK v2.45 and RealSense ROS v2.3.0 from source. I also patched the kernel as indicated by the linux instructions. When launching rs_camera.launch, I can get the depth image as well as the RGB image. However, the timestamps look to me very odd and with a huge offset. 
I have already read almost every issue posted in this forum and other ones as well, including:

- https://answers.ros.org/question/350467/intel-realsense-time-is-out-of-dual-32-bit-range
- https://github.com/IntelRealSense/realsense-ros/issues/1510
- https://github.com/IntelRealSense/librealsense/issues/5319
- https://github.com/IntelRealSense/librealsense/issues/4505
- https://github.com/IntelRealSense/librealsense/issues/5067


To better observe the issue, consider the following Python script:

```
#!/usr/bin/env python

import rospy
from sensor_msgs.msg import Image

def depth_cb(msg):
    diff = (rospy.Time.now() - msg.header.stamp).to_sec()
    print("Time diff is : " + str(diff))

rospy.init_node("header_publisher")

depth_sub = rospy.Subscriber("/camera/depth/image_rect_raw", Image, depth_cb)

rospy.spin()
```

I first tried this script in my Intel workstation. I actually have tried with 2 different computers, both with USB3.2:

- Ubuntu 18 + Kernel 5.4.0-56-generic + ROS Melodic  (kernel **unpatched**)
- Ubuntu 20 + Kernel 5.8.0-44-generic + ROS Noetic (kernel **unpatched**)
In both cases, I get all streams (depth, rgb, infra1, infra2, etc...) with the expected 30Hz, and the scripts outputs:
```
Time diff is : 0.004414319
Time diff is : -0.0028548250000000053
Time diff is : -0.0036184799999999795
Time diff is : -0.004709960000000013
Time diff is : -0.0039980419999999794
Time diff is : -0.0026743420000000517
Time diff is : -0.0009248259999999897
Time diff is : 0.002310276
Time diff is : 0.003344535
Time diff is : 0.005857705
Time diff is : 0.007889747
Time diff is : 0.01012206
.......
```
And then quickly stabilizes to a difference of around 10ms, which is a very reasonable value.

However, when I run this same script on the Tinkerboard, I get this very different output:

```
Time diff is : 13161.960861921
Time diff is : 13161.956075191
Time diff is : 13161.952967882
Time diff is : 13161.950196981
Time diff is : 13161.946646213
Time diff is : 13161.943586111
Time diff is : 13161.94018054
Time diff is : 13161.937459945
....
Time diff is : 13197.146675586
Time diff is : 13203.94861412
```
And after 15-20seconds, it stabilizes to a value of 13200secs = 3.7hours! It doesn't really stabilize, since the value keeps fluctuating. I am launching the ROS node with the following command, as suggested in other posts:

```
roslaunch realsense2_camera rs_camera.launch initial_reset:=True enable_sync:=True
```
This is the full output when launched above commands:

```
SUMMARY
========

PARAMETERS
 * /camera/realsense2_camera/accel_fps: -1
 * /camera/realsense2_camera/accel_frame_id: camera_accel_frame
 * /camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti...
 * /camera/realsense2_camera/align_depth: False
 * /camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/allow_no_texture_points: False
 * /camera/realsense2_camera/base_frame_id: camera_link
 * /camera/realsense2_camera/calib_odom_file: 
 * /camera/realsense2_camera/clip_distance: -2.0
 * /camera/realsense2_camera/color_fps: -1
 * /camera/realsense2_camera/color_frame_id: camera_color_frame
 * /camera/realsense2_camera/color_height: -1
 * /camera/realsense2_camera/color_optical_frame_id: camera_color_opti...
 * /camera/realsense2_camera/color_width: -1
 * /camera/realsense2_camera/confidence_fps: -1
 * /camera/realsense2_camera/confidence_height: -1
 * /camera/realsense2_camera/confidence_width: -1
 * /camera/realsense2_camera/depth_fps: -1
 * /camera/realsense2_camera/depth_frame_id: camera_depth_frame
 * /camera/realsense2_camera/depth_height: -1
 * /camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti...
 * /camera/realsense2_camera/depth_width: -1
 * /camera/realsense2_camera/device_type: 
 * /camera/realsense2_camera/enable_accel: False
 * /camera/realsense2_camera/enable_color: True
 * /camera/realsense2_camera/enable_confidence: True
 * /camera/realsense2_camera/enable_depth: True
 * /camera/realsense2_camera/enable_fisheye1: False
 * /camera/realsense2_camera/enable_fisheye2: False
 * /camera/realsense2_camera/enable_fisheye: False
 * /camera/realsense2_camera/enable_gyro: False
 * /camera/realsense2_camera/enable_infra1: False
 * /camera/realsense2_camera/enable_infra2: False
 * /camera/realsense2_camera/enable_infra: False
 * /camera/realsense2_camera/enable_pointcloud: False
 * /camera/realsense2_camera/enable_pose: False
 * /camera/realsense2_camera/enable_sync: True
 * /camera/realsense2_camera/filters: 
 * /camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f...
 * /camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o...
 * /camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f...
 * /camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o...
 * /camera/realsense2_camera/fisheye_fps: -1
 * /camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame
 * /camera/realsense2_camera/fisheye_height: -1
 * /camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op...
 * /camera/realsense2_camera/fisheye_width: -1
 * /camera/realsense2_camera/gyro_fps: -1
 * /camera/realsense2_camera/gyro_frame_id: camera_gyro_frame
 * /camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic...
 * /camera/realsense2_camera/imu_optical_frame_id: camera_imu_optica...
 * /camera/realsense2_camera/infra1_frame_id: camera_infra1_frame
 * /camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt...
 * /camera/realsense2_camera/infra2_frame_id: camera_infra2_frame
 * /camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt...
 * /camera/realsense2_camera/infra_fps: 30
 * /camera/realsense2_camera/infra_height: 480
 * /camera/realsense2_camera/infra_rgb: False
 * /camera/realsense2_camera/infra_width: 848
 * /camera/realsense2_camera/initial_reset: True
 * /camera/realsense2_camera/json_file_path: 
 * /camera/realsense2_camera/linear_accel_cov: 0.01
 * /camera/realsense2_camera/odom_frame_id: camera_odom_frame
 * /camera/realsense2_camera/ordered_pc: False
 * /camera/realsense2_camera/pointcloud_texture_index: 0
 * /camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
 * /camera/realsense2_camera/pose_frame_id: camera_pose_frame
 * /camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic...
 * /camera/realsense2_camera/publish_odom_tf: True
 * /camera/realsense2_camera/publish_tf: True
 * /camera/realsense2_camera/rosbag_filename: 
 * /camera/realsense2_camera/serial_no: 
 * /camera/realsense2_camera/stereo_module/exposure/1: 7500
 * /camera/realsense2_camera/stereo_module/exposure/2: 1
 * /camera/realsense2_camera/stereo_module/gain/1: 16
 * /camera/realsense2_camera/stereo_module/gain/2: 16
 * /camera/realsense2_camera/tf_publish_rate: 0.0
 * /camera/realsense2_camera/topic_odom_in: odom_in
 * /camera/realsense2_camera/unite_imu_method: 
 * /camera/realsense2_camera/usb_port_id: 
 * /rosdistro: noetic
 * /rosversion: 1.15.9

NODES
  /camera/
    realsense2_camera (nodelet/nodelet)
    realsense2_camera_manager (nodelet/nodelet)

ROS_MASTER_URI=https://localhost:11311

process[camera/realsense2_camera_manager-1]: started with pid [7210]
process[camera/realsense2_camera-2]: started with pid [7211]
[ INFO] [1622463313.044422583]: Initializing nodelet with 4 worker threads.
[ INFO] [1622463313.403502744]: RealSense ROS v2.3.0
[ INFO] [1622463313.403814245]: Built with LibRealSense v2.45.0
[ INFO] [1622463313.404075580]: Running with LibRealSense v2.45.0
[ INFO] [1622463313.464884925]:  
 31/05 14:15:13,556 WARNING [2756690960] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: No data available, number: 3d
 31/05 14:15:13,607 WARNING [2756690960] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: No data available, number: 3d
[ INFO] [1622463313.804460933]: Device with serial number 049XXXXXXXXX was found.

[ INFO] [1622463313.804861685]: Device with physical ID 1-1.2-20 was found.
[ INFO] [1622463313.805124187]: Device with name Intel RealSense D455 was found.
[ INFO] [1622463313.806747029]: Device with port number 1-1.2 was found.
[ INFO] [1622463313.807013031]: Device USB type: 2.1
[ WARN] [1622463313.807231782]: Device 049XXXXXXXXX is connected using a 2.1 port. Reduced performance is expected.
[ INFO] [1622463313.807458992]: Resetting device...
[ INFO] [1622463319.919641342]:  
[ INFO] [1622463320.203137991]: Device with serial number 049XXXXXXXXX was found.

[ INFO] [1622463320.203331950]: Device with physical ID 1-1.2-21 was found.
[ INFO] [1622463320.203455909]: Device with name Intel RealSense D455 was found.
[ INFO] [1622463320.204693749]: Device with port number 1-1.2 was found.
[ INFO] [1622463320.204826167]: Device USB type: 2.1
[ WARN] [1622463320.204926792]: Device 049XXXXXXXXX is connected using a 2.1 port. Reduced performance is expected.
[ INFO] [1622463320.209113691]: getParameters...
[ INFO] [1622463320.313030155]: setupDevice...
[ INFO] [1622463320.313147989]: JSON file is not provided
[ INFO] [1622463320.313771576]: ROS Node Namespace: camera
[ INFO] [1622463320.314055661]: Device Name: Intel RealSense D455
[ INFO] [1622463320.314316121]: Device Serial No: 049XXXXXXXXX
[ INFO] [1622463320.314566372]: Device physical port: 1-1.2-21
[ INFO] [1622463320.314813999]: Device FW version: 05.12.13.50
[ INFO] [1622463320.315057250]: Device Product ID: 0x0B5C
[ INFO] [1622463320.315312460]: Enable PointCloud: Off
[ INFO] [1622463320.315580795]: Align Depth: Off
[ INFO] [1622463320.315759004]: Sync Mode: On
[ INFO] [1622463320.316274090]: Device Sensors: 
[ INFO] [1622463320.344475208]: Stereo Module was found.
[ INFO] [1622463320.365591995]: RGB Camera was found.
[ INFO] [1622463320.366235707]: Motion Module was found.
[ INFO] [1622463320.366376874]: (Confidence, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1622463320.366482166]: num_filters: 0
[ INFO] [1622463320.366550125]: Setting Dynamic reconfig parameters.
[ WARN] [1622463320.770441914]: Param '/camera/stereo_module/auto_exposure_limit' has value 0 that is not in range [1, 165000]. Removing this parameter from dynamic reconfigure options.
[ WARN] [1622463320.782879235]: Param '/camera/stereo_module/auto_gain_limit' has value 0 that is not in range [16, 248]. Removing this parameter from dynamic reconfigure options.
[ INFO] [1622463320.865453744]: Done Setting Dynamic reconfig parameters.
[ INFO] [1622463320.866722793]: depth stream is enabled - width: 640, height: 480, fps: 15, Format: Z16
[ INFO] [1622463320.868016051]: color stream is enabled - width: 640, height: 480, fps: 15, Format: RGB8
[ INFO] [1622463320.868177343]: setupPublishers...
[ INFO] [1622463320.874226836]: Expected frequency for depth = 15.00000
[ INFO] [1622463320.882238965]: Expected frequency for color = 15.00000
[ INFO] [1622463320.887432994]: setupStreams...
[ INFO] [1622463320.909705662]: insert Depth to Stereo Module
[ INFO] [1622463320.910048955]: insert Color to RGB Camera
 31/05 14:15:21,012 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,062 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,113 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
[ INFO] [1622463321.164020020]: SELECTED BASE:Depth, 0
[ INFO] [1622463321.183567047]: RealSense Node Is Up!
[ WARN] [1622463321.347595477]: 
[ERROR] [1622463321.347994479]: An error has occurred during frame callback: Time is out of dual 32-bit range
[ERROR] [1622463321.413401100]: An error has occurred during frame callback: Time is out of dual 32-bit range
 31/05 14:15:21,478 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,529 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,580 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,630 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,681 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,731 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,782 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 31/05 14:15:21,832 WARNING [2681156624] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
```

The issue happens with both D435i and D455 cameras. I am hoping there is some way to fix the issue, considering that other users have reported to being able to make it work with Raspberry Pi 3 and Odroid platforms.

I have also tried with kernel 5.10.xx, with identical results. Anybody has any suggestion on how should I proceed? Suggestions are very welcome, thanks!



Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Unusable timestamps in D435i/D455 on Tinkerboard #1906

Issue Description

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Required Info
Camera Model	D435i, D455
Firmware Version	5.12.13.50
Operating System & Version	Armbian 20.05.7 Focal
Kernel Version (Linux Only)	5.4.123-rockchip
Platform	ARM
SDK Version	2.45
Language	C++, Python
Segment	others

Unusable timestamps in D435i/D455 on Tinkerboard #1906

Description

Issue Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions