MotionBricks: Scalable Real-Time Motions with Modular Latent Generative Model and Smart Primitives

MotionBricks is a modular latent generative model that animates characters and robots in real time. It fills in the motion between four recent poses and up to four target poses. Three learned modules share the work, as How MotionBricks works explains. Smart Primitives turn target poses into building blocks for navigation and object interaction, so behaviors like vaulting and sitting snap together like bricks, with no retraining.

This repository provides pretrained MotionBricks models, based on the SIGGRAPH 2026 paper, and everything you need to put them to work:

• Pretrained models for the SOMA body model and the Unitree G1 robot. They ship as ONNX files, with the pretrained checkpoints on Hugging Face.
• A Python runtime, built on Warp, Newton, and TensorRT, that chains generations into continuous motion.
• Two demos: an Inbetweener that generates motion between poses you place, and a Scene Preview where a character walks, runs, vaults, ducks, and sits its way through a scene.
• Two editors to author your own Smart Primitives and scenes.

The demos show one way to drive the model. Your application decides where the targets come from. New to MotionBricks? The guided tutorial walks through all four apps as it builds one interaction scene.

News

• MotionBricks has its own repository and a pretrained model for SOMA humans. The first release was a Unitree G1 preview in GR00T Whole-Body Control.

Setup

The runtime uses TensorRT, which requires an NVIDIA GPU. The demos were tested on Windows and Linux (x86-64) with an NVIDIA GeForce RTX 4090.

Install Git LFS and uv. Then clone this repository and run these commands from its root:

git lfs pull
uv sync

The first launch of each model and precision builds a TensorRT plan. This takes a few minutes. Plans are cached in .trt-cache/, so later launches start quickly. Each app section below gives its command.

How MotionBricks works

Each generation fills in the motion between two sets of poses at 30 Hz:

• The context is the four most recent poses of the character (green above).
• The target is one to four poses that the motion should reach (blue above). One pose sets where the character ends up and stops. More poses also set how it moves at that point.

The model decides how long the transition takes. One generation produces 6 to 16 tokens of four poses, or about 0.6 to 1.9 s of motion. Generation length explains the frame counts.

Three learned modules share the work:

• The root module predicts the duration and the path of the root.
• The pose module is a masked token transformer. It predicts the motion tokens along that path.
• The decoder turns the tokens into continuous motion that reaches the targets. It uses a VQ-VAE with several codebook heads.

Longer motion comes from chaining generations. The runtime plays the generated motion, records what it played, and uses the last four recorded poses as the context of the next generation. Your application decides the targets. That choice is what turns one inbetweening model into keyframe animation, continuous locomotion, or object interactions. The paper covers the method, the training data, and the evaluation.

One runtime for SOMA and G1

Both models generate the same way. Only the pose layout differs. The apps and the runtime work the same for both rigs, so one code path drives a human skeleton and a robot. Engines describes the two layouts.

Repository layout

The motionbricks package is the library. The demos and editors in app are built on it.

Folder	Contents
motionbricks	The library: engines, motion players, recorder, blend spaces, pivots, and the ONNX models
app	The demos, the two editors, and the character runtime they share
assets	Sample motion clips, Smart Primitives, and the gym scenes
docs	Guides and model cards, with runnable examples in docs/examples
tools/onnx	The standalone ONNX exporter

Inbetweener

Start here to see the core idea. Choose poses from clips, place them in the view, and generate motion through them in order.

SOMA	Unitree G1

Run it with this command:

uv run python -m app.inbetweener_demo

Inbetweener lists its options and things to try. It also walks through the code.

Scene Preview

Drive a character through a scene with continuous locomotion and interactions. It can walk and run forward, strafe while facing the camera, or strafe while crouching. Six style modes, such as Skipping and Zombie, change how it walks. Scene Preview distills the Unreal Engine demo behind the paper's videos into a small Python runtime. It puts the ideas front and center, so you can carry them into any integration.

SOMA	Unitree G1

Run it with the SOMA or the Unitree G1 gym:

uv run python -m app.scene_preview --scene assets/scenes/soma_gym.scene.json
uv run python -m app.scene_preview --scene assets/scenes/g1_gym.scene.json

Scene Preview lists its options and controls. It also explains the runtime behind it: the character, the state machine, locomotion, and interactions.

Editors

A Smart Primitive describes an action as target poses placed around an object. Author one in the Smart Primitive Editor. Place it in a scene for Scene Preview with the Scene Editor.

Smart Primitive Editor	Scene Editor

Run them with these commands:

uv run python -m app.smart_primitive_editor
uv run python -m app.scene_editor

Start with the guided tutorial to learn these tools. It builds an interaction step by step with both editors. Smart Primitive Editor and Scene Editor describe each editor and its documents.

These editors show one way to build dynamic sequences and interactions. Your application can follow the same approach or a different one. For example, these editors sample a handful of target poses from clips. A motion generation model, such as Kimodo or ARDY, could generate those poses instead.

Checkpoints

MotionBricks checkpoints are available for the SOMA body model and the Unitree G1 robot. Their weights use the SafeTensors format.

Note: the apps use the ONNX models bundled in motionbricks/assets/onnx, so you do not need to download a checkpoint to run them. Download one to export your own ONNX models or to build another runtime. This repository includes the ONNX exporter.

Model	Skeleton	Training Data	FPS	Hugging Face	License
MotionBricks-SOMA-RP-v1	SOMA	Bones RigPlay 1 (RP1)	30	Link	NVIDIA Open Model Agreement
MotionBricks-G1-RP-v1	Unitree G1	Bones RigPlay 1 (RP1), retargeted	30	Link	NVIDIA Open Model Agreement

The SOMA model card and the Unitree G1 model card describe each model, its inputs and outputs, and its limitations.

Export ONNX models

The ONNX models in this repository were exported from these checkpoints with the standalone exporter in tools/onnx. To export them yourself, install the Hugging Face CLI, then download the checkpoint folders and run the command for your model from the repository root:

hf download nvidia/MotionBricks --local-dir checkpoints/MotionBricks

# SOMA
uv run --locked --project tools/onnx motionbricks-onnx --model soma --checkpoints checkpoints/MotionBricks/soma

# Unitree G1
uv run --locked --project tools/onnx motionbricks-onnx --model unitree_g1 --checkpoints checkpoints/MotionBricks/g1

To download one model, add --include "soma/*" or --include "g1/*". Keep the downloaded folder structure intact.

The exporter has its own environment, and uv installs it the first time you run the command. FP32 is the default. Add --precision fp16 for mixed FP16. The exporter guide lists every option, and ONNX inference explains the tensor interfaces and batching.

Documentation

New to MotionBricks? Start with the guided tutorial. It explains the core concepts, then builds an interaction scene with all four apps.

Apps

• Inbetweener: Shows how to run the Inbetweener, then walks through its code, from sampling and placing poses to chained playback.
• Scene Preview: Lists the options and controls, then describes the example runtime, from the character and the state machine to locomotion and interactions.
• Smart Primitive Editor: Describes what a Smart Primitive holds and how to author one: the node hierarchy, target poses, interaction boxes, pivots, and sequences.
• Scene Editor: Shows how to build a scene, and describes the .scene.json document that Scene Preview opens.

Integration

• Runtime API: Start here to integrate MotionBricks in your application. Describes the engines, pose layouts, clips, and chaining generations. It includes two runnable scripts that generate motion without a viewer.
• ONNX inference: Describes the tensor shapes, batching, and other inference libraries, for use without the bundled engines.

Models

• ONNX exporter: Describes how to export the bundled ONNX models from the checkpoints on Hugging Face.
• SOMA model card: Describes the SOMA model, its inputs and outputs, its training data, and its limitations.
• Unitree G1 model card: Describes the Unitree G1 model, its inputs and outputs, its training data, and its limitations.

Related humanoid work at NVIDIA

MotionBricks is part of a larger effort to enable humanoid motion for animation, robotics, and physical AI. Check out these related works:

• Kimodo: human motion generation model specializing in controllable offline motion authoring.
• ARDY: builds on Kimodo to enable controllable motion generation in real time for interactive applications.
• SOMA Body Model: a unified parametric human body model.
• BONES-SEED Dataset: a large motion capture dataset of humans and humanoids in SOMA and G1 format.
• ProtoMotions: simulation and learning framework for training physically simulated humans and humanoids.
• SOMA Retargeter: retargets SOMA motion to humanoid robots such as the Unitree G1.
• GEM: human motion reconstruction from video.
• GEAR SONIC: humanoid behavior foundation model for physical robots.

Citation

If you use this code in your research, please cite:

@article{wang2026motionbricks,
  title   = {MotionBricks: Scalable Real-Time Motions with Modular Latent Generative Model and Smart Primitives},
  author  = {Wang, Tingwu and Dionne, Olivier and De Ruyter, Michael and Minor, David and Rempe, Davis and Zhao, Kaifeng and Petrovich, Mathis and Yuan, Ye and Li, Chenran and Luo, Zhengyi and Robison, Brian and Blackwell, Xavier and Antoniazzi, Bernardo and Peng, Xue Bin and Zhu, Yuke and Yuen, Simon},
  journal = {ACM Transactions on Graphics (TOG)},
  year    = {2026},
  volume  = {45},
  number  = {4},
  doi     = {10.1145/3811334}
}

License

Code, documentation, models, and sample motions have different licenses:

• Code: Apache 2.0, including code examples and snippets in the documentation.
• Documentation and media: NVIDIA-authored documentation text and media (images, screenshots, and videos, including the files in docs/media/) use Creative Commons Attribution 4.0 International (CC-BY-4.0).
• Models: NVIDIA Open Model Agreement. This covers the SOMA and G1 checkpoints on Hugging Face and the ONNX models. It permits commercial use and redistribution of modified models, subject to the agreement's terms. Licensed by NVIDIA Corporation under the NVIDIA Open Model Agreement.
• Sample motions: NVIDIA Sample Data Evaluation License. The covered human BVH and retargeted G1 CSV clips are for evaluating and testing NVIDIA technologies. The sample-data notice lists the covered files and credits Bones. These sample terms are separate from the model terms.

Licenses for dependencies and bundled robot assets are in licenses. Exporter dependency licenses are in tools/onnx/licenses.

This project downloads and installs additional components under their own license terms. Review those terms before use.

The TensorRT OSS notice applies to TensorRT OSS components, not the separately installed tensorrt-cu12, tensorrt-cu12-bindings, and tensorrt-cu12-libs runtime packages. Those packages are governed by the NVIDIA TensorRT Software License Agreement distributed with the packages.

Acknowledgments

This project builds upon excellent open source projects:

• Newton for the viewer and scene
• NVIDIA Warp for GPU kernels
• NVIDIA TensorRT for inference
• Dear ImGui through imgui_bundle for the user interface
• OpenUSD for the SOMA character
• Unitree Robotics for the Unitree G1 robot model and meshes

Contact

This project is currently not accepting contributions.

For questions or issues, please open an issue on this repository or reach out directly to the authors.