Autonomous Object Tracking and Obstacle Avoidance on a Gobilda Robot

Abstract

This project developed an autonomous control system for a Gobilda robot equipped with an OAK-D Pro camera and RPLIDAR A1 sensor. It uses a finite state machine (FSM) approach to allow the robot to track people while avoiding obstacles. The system integrates computer vision and LiDAR for real-time navigation using ROS2 middleware. The robot demonstrates smooth behavioral transitions and performance suitable for dynamic environments, validating its capability for applications requiring both environment awareness and target following.

Introduction

• Combines person/object detection with obstacle avoidance.
• Uses OAK-D Pro spatial AI camera, RPLIDAR A1 2D lidar, and NVIDIA Jetson Nano Orin running ROS2.
• FSM manages transitions between tracking and avoidance based on sensor input.
• Main goal: Reliable, seamless behavior switching between following a tracked object and safely navigating obstacles.

Software Architecture

Built on ROS2 and a node-based structure, the Object Tracking Node subscribes to:

• /oak/nn/spatial_detections (camera object detections)
• /scan (LiDAR data)
• /oak/imu/data (IMU; not used in final implementation)

Finite State Machine Design

• Check LiDAR – Prioritizes obstacle detection.
• Check Detection – Processes camera data for targeting.
• Follow Object – Proportional control tracking a detected person.
• Stop Following – Halts when close to target.
• Roomba Move Forward – Default exploration when no detections.
• Roomba Backup – Reverse movement in response to nearby obstacles.
• Roomba Spin – Rotational search after backup.
• Roomba Recovery – Default safety when sensor data unavailable.

Implementation Details

Object Detection & Tracking

The camera uses MobileNet-SSD for real-time person detection. Upon detection, positional data (x, y, z) is extracted and processed to determine proximity and direction relative to the robot.

Obstacle Avoidance

LiDAR data (ranges 340–800) monitors the area ahead. Obstacles within 0.6 m trigger avoidance states. If no LiDAR data, the robot enters recovery.

Motion Control

Proportional control adjusts linear and angular velocities based on the detected target’s vector position. Separate gains are used for right and left turns to counteract hardware asymmetry. The logic reduces speed as the robot approaches its target to minimize oscillations for smooth tracking.

Challenges & Solutions

• State Machine Complexity: Race conditions mitigated by clear priorities—obstacle avoidance always overrides tracking.
• Detection Reliability: MobileNet-SSD tuned for confidence and robustness; fallback behaviors included.
• Velocity Tuning: Different proportional gains and normalization improve smoothness.
• Camera Configuration: Overcame documentation gaps with extensive testing and use of Foxglove for debugging.
• Integration: Ensured states set explicit velocities before transitions to avoid unsafe stops or direction changes.

Demonstration Results

• Showed consistent person following at distances >1 m.
• Reliable, safe obstacle avoidance within LiDAR range (~0.6 m).
• Quick, correct state transitions validated FSM design and safety prioritization strategy.
• See demo: Watch Video Demo

Future Improvements

• Experiment with additional neural network models or custom training for more robust detection.
• Investigate object tracking filters available in DepthAI ROS for improved performance.
• Integrate localization by fusing IMU and odometry data for advanced navigation tasks.
• Explore advanced object tracking and following algorithms to handle occlusions and quick movements.

Conclusion

This project validated a multi-modal sensor fusion approach for real-time autonomous object tracking and obstacle avoidance, leveraging effective state logic and proportional control to reliably navigate dynamic environments.

References

• Luxonis. "DepthAI ROS Driver."
• Luxonis. "depthai-ros." GitHub repository.
• [Add other relevant citations or links as appropriate]