TrajMamba: predicting where people will walk
A Mamba model that predicts the next three seconds of a person’s path from an RGB-D camera, in real time on a Jetson Orin.
- When
- 2026
- Origin
- PhD research, paper accepted at ICCMA 2026
- Built with
- ROS 2 Humble and Jazzy
- PyTorch
- Mamba
- MediaPipe Pose
- Jetson Orin
- TIAGo
- Paper
- ICCMA 2026, 2026 (accepted)
A robot that plans around people needs to know where they will be, not only where they are. This ROS 2 package tracks up to four people and predicts each one’s path for the next three seconds.
Pipeline
- MediaPipe Pose finds the hip centre of each person in the RGB image.
- The depth image and camera intrinsics turn that pixel into a 3D point.
- A nearest-neighbour tracker links detections over time.
- After about two seconds of observation, TrajMamba predicts the next three.
TrajMamba is a bidirectional Mamba (selective state space) encoder with a GRU decoder. It is small enough to run on board an NVIDIA Jetson Orin beside the rest of the navigation stack.
Output
The node publishes an annotated image, RViz markers, and a PoseArray of predicted paths. A live window shows skeletons, the observed trail, the prediction with its uncertainty cone, and a depth gauge.