Holistic Fusion: Task and Setup-agnostic Robot Localization and State Estimation with Factor Graphs

Authors: Julian Nubert (nubertj@gmail.com), Turcan Tuna, Jonas Frey, Cesar Cadena, Katherine J. Kuchenbecker, Shehryar Khattak, Marco Hutter

Homepage | Paper (T-RO) | arXiv | Video | Datasets | Docs | Doxygen | Examples

Holistic Fusion (HF) is an open-source library for flexible task and setup-agnostic robot localization and state estimation. It provides a wide range of features that are very useful in common robotic workflows, including online sensor fusion, offline batch optimization and calibration. While it already supports a set of measurement factors quite common in field robotic applications, a core purpose of HF is to simplify the process of integrating new measurement types without manually deriving Jacobians by i) utilizing GTSAM expression factors and ii) following a specific structure to follow a common parent class interface, depending on the measurement type. Currently, HF supports three general measurement types: i) absolute measurements, ii) landmark measurements, iii) local & relative measurements.

Disclaimer: The framework is still under development and will be updated, extended, and more generalized in the future.

Modules and Packages

This repository contains the following modules:

1. Holistic Fusion: The core library for the sensor fusion. This library depends mainly on Eigen and GTSAM and can be used with any communication layer (including ROS1 and ROS2).
2. Holistic Fusion ROS: This package provides an example class for HolisticFusion in ROS. It is dependent on HolisticFusion and ROS.
3. Holistic Fusion ROS2: The ROS2 integration layer for HolisticFusion, including ROS2 publishers, subscribers, services, parameter handling, and TF-based static transforms.
4. ROS1 Examples: Examples on how to use HolisticFusion and HolisticFusionRos.
   • ANYmal Estimator - Quadrupedal Robot: This is the implementation of the ANYmal quadrupedal robot estimator as presented in [1], including IMU, GNSS, leg odometry, and absolute LiDAR measurements.
   • HEAP - Excavator: This is the implementation of the HEAP excavator as presented in [1], including IMU, two GNSS antennas, and absolute LiDAR measurements.
   • Grand Tour GT Generation - Leica Total Station Position & GNSS: The GT generation estimator aligning two non-drifting trajectories: i) the Leica total station R3 position, and ii) the Novotel offline optimized SE(3) trajectory.
   • Super Mega Bot - Robot for Teaching Purposes: A wheeled robot integrating IMU, absolute LiDAR poses, and wheel encoders.
   • Pure IMU Integration: A simple example of performing pure IMU integration for dead-reckoning performance testing.
   • IMU Pose3 Fuser: A simple example of fusing an IMU and an SE(3) pose measurement.
   • Holistic Fusion ROS Examples: A meta-package bundling all the aforementioned ROS examples in one package for convenience.
5. ROS2 Examples: Examples built on HolisticFusionRos2.
   • Super Mega Bot Estimator: A ROS2 estimator demonstrating IMU fusion with configurable absolute LiDAR odometry, VIO, wheel-odometry between factors, and wheel linear-velocity factors. The package includes launch files for live operation, simulation, and rosbag replay.

Instructions

Please refer to our Read the Docs for detailed instructions regarding installation and usage.

See the HolisticFusion configuration contract for parameter applicability and GTSAM 4.3 migration details.

Code Documentation

Please refer to our Doxygen for documentation of the code.

Datasets

The datasets used for the ANYmal and HEAP (excavator) examples are publicly available on Google Drive.

Paper

If you find this code useful or use it in your work, please consider citing:

[1] Holistic Fusion

Holistic Fusion: Task- and Setup-Agnostic Robot Localization and State Estimation with Factor Graphs

IEEE Transactions on Robotics (T-RO), 2026

Project page · Paper · arXiv · Video · Datasets

@article{nubert2026holistic,
  title     = {Holistic Fusion: Task- and Setup-Agnostic Robot Localization and State Estimation with Factor Graphs},
  author    = {Nubert, Julian and Tuna, Turcan and Frey, Jonas and Cadena, Cesar and Kuchenbecker, Katherine J. and Khattak, Shehryar and Hutter, Marco},
  journal   = {IEEE Transactions on Robotics},
  year      = {2026},
  publisher = {IEEE},
  doi       = {10.1109/TRO.2026.3714645}
}

[2] Graph-Based Multi-Sensor Fusion

Graph-Based Multi-Sensor Fusion for Consistent Localization of Autonomous Construction Robots

IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, 2022

Project page · Paper · Video

@inproceedings{nubert2022graph,
  title={Graph-based Multi-sensor Fusion for Consistent Localization of Autonomous Construction Robots},
  author={Nubert, Julian and Khattak, Shehryar and Hutter, Marco},
  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
  year={2022},
  organization={IEEE}
}

Acknowledgments

The authors thank their colleagues at ETH Zurich and NASA JPL for their help in conducting the robot experiments and evaluations and using HF on their robots. Special thanks go to Takahiro Miki and the ANYmal Hike team at the Robotic Systems Lab (RSL), ETH Zurich, Nikita Rudin, and David Hoeller for the ANYmal Parkour experiments, Patrick Spieler for running the deployments on the JPL RACER vehicle, the entire excavation team at RSL and Gravis Robotics, Thomas Mantel and the teaching assistants of the ETH Robotic Summer School for their help on the SuperMegaBot, and Mayank Mittal for his help in generating renderings.