Skip to content

About

Embodied Intelligence Guide: cross-vendor field guide to the embedded AI stack. Hardware, runtimes and a learning roadmap for NVIDIA Jetson, Raspberry Pi + Hailo, RK3588, OpenVINO, TensorRT, ONNX Runtime, LiteRT and ROS 2, with beginner projects and 2024–2026 deprecations tracked.

Topics

Resources

Code of conduct

Contributing

Stars

5 stars

Watchers

1 watching

Forks

Latest commit

 

History

16 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Embodied Intelligence Guide Awesome

Glowing AI chip on a circuit board: Embodied Intelligence Guide to edge AI, embedded AI, and Physical AI

Edge AI & Physical AI, from sensor → inference → actuation: NVIDIA Jetson, Raspberry Pi + Hailo, RK3588, OpenVINO, TensorRT, ONNX Runtime, LiteRT, and ROS 2.

A vendor-neutral field guide to Edge AI, Embedded AI, and Physical AI. It covers 10+ hardware platforms, 6+ inference runtimes, a Phase 0→7 learning roadmap, reproducible beginner projects, a curated resource collection, and the 2024–2026 renames and deprecations, so you never follow a dead-end tutorial.

License: MIT Content: CC BY 4.0 PRs Welcome
Link Check Awesome Lint Last Commit Stars


Embodied Intelligence Guide is a practical, cross-vendor guide to the embedded AI stack: running AI models on edge devices and robots. It helps you choose an edge AI board (Jetson, Raspberry Pi 5 + Hailo, RK3588, Intel, AMD, or a discrete NPU), pick the right inference runtime (TensorRT, OpenVINO, ONNX Runtime, LiteRT, RKNN), build a real-time camera → inference → action pipeline, and grow from first demo to ROS 2 robotics and Physical AI. Hardware specs and prices were last verified in May 2026.

Note

The edge AI landscape changed a lot in 2024–2026. TensorFlow Lite is now LiteRT; Google Coral / Edge TPU is effectively discontinued; the Raspberry Pi AI Kit was replaced by the AI HAT+ / AI HAT+ 2; Jetson AGX Thor shipped; and the newest ROS 2 LTS is Lyrical Luth. Read Renames & Deprecations → before you buy hardware or pick a runtime.

Contents


What This Hub Solves

The questions that block people getting started in edge AI, and where this hub answers them.

Edge AI and Physical AI knowledge is scattered across vendor docs, blog posts, YouTube, and one-off sample repos, and each assumes you've already picked their stack. Much of it is also out of date (renamed SDKs, discontinued hardware). This hub fixes three things:

1. Hardware and runtime selection paralysis

  • Problem: "TOPS" marketing numbers don't map to real FPS, memory limits aren't obvious, and it's unclear which runtime works on which chip.
  • Solution: one-screen comparison tables plus per-vendor pages with caveats. → Hardware Landscape · Runtimes & SDKs

2. The "model on a bench" → "real pipeline" gap

  • Problem: tutorials stop at a single inference call. Few show the full camera → preprocess → inference → postprocess → action pipeline, or how to build a robot perception graph.
  • Solution: pipeline patterns and reproducible projects you can run on hardware you can buy today. → Edge Pipelines · Beginner Quick-Wins

3. Following stale, dead-end guides

  • Problem: many tutorials online still tell you to buy a Coral or pip install tflite-runtime. Both are effectively dead.
  • Solution: a living "what changed" page and a structured learning order. → Renames & Deprecations · Knowledge Roadmap

What Is Edge AI, Embedded AI, and Physical AI?

The 60-second mental model before you pick hardware. Full version in Concepts & Definitions.

  • Cloud AI: inference in a datacenter. Maximum compute, but higher latency, and data leaves the device.
  • Edge AI: inference on or next to the device producing the data. Low latency, privacy, bandwidth savings, and it works offline.
  • Embedded AI / TinyML: inference on microcontrollers and DSPs with kilobytes of memory and milliwatts of power.
  • Physical AI: systems that perceive, reason about, and act in the physical world (robots, drones, autonomous machines). These are increasingly driven by vision-language-action (VLA) and world foundation models running at the edge.

Why on-device inference matters

  • Real-time control: closed-loop robotics needs deterministic, bounded latency (high-end platforms target multi-sensor fusion in well under ~10 ms).
  • Privacy: faces, license plates, and patient or factory data never leave the device.
  • Bandwidth and cost: send insights, not raw video, to the cloud.
  • Availability: it keeps working when the network doesn't.

Start Here

New to edge AI? Follow these five steps in order.

Step Do this Go to
1. Understand the terms Cloud vs Edge vs Embedded vs Physical AI (10-minute read) Concepts & Definitions →
2. Pick a board Use the decision tree to choose your first edge AI board Getting Started →
3. Get a quick win Run real-time camera object detection on hardware you can buy today Beginner Projects →
4. Follow the roadmap Beginner → advanced, phase by phase Knowledge Roadmap →
5. Go deep Runtimes, pipelines, robotics, deployment, and the market Repository Structure ↓

Three recommended tracks (full detail in Getting Started):

  • 🟢 Maker / lowest cost: Raspberry Pi 5 + AI HAT+ (Hailo). Quick-win →
  • 🟡 Prototyping / robotics: NVIDIA Jetson Orin Nano Super. Quick-win →
  • 🔴 Advanced / humanoid and multi-sensor: Jetson AGX Thor + Isaac ROS. Robotics →

Learning Roadmap

From zero to Physical AI in eight phases. The phases are ordered but modular, so skip what you already know.

flowchart LR
    A["Phase 0\nLinux + Python/C++\nfoundations"] --> B["Phase 1\nEmbedded basics\n& TinyML"]
    B --> C["Phase 2\nInference runtimes\nONNX RT · LiteRT · TensorRT · OpenVINO"]
    C --> D["Phase 3\nMedia pipelines\nGStreamer · DeepStream"]
    D --> E["Phase 4\nCUDA & HW\nacceleration"]
    E --> F["Phase 5\nROS 2 &\nrobotics inference"]
    F --> G["Phase 6\nEdge deployment\n& optimization"]
    G --> H["Phase 7\nReal-world\nPhysical AI pipelines"]
Loading

Detailed, link-by-link version: knowledge-roadmap.md →


Edge AI Hardware at a Glance

A starter comparison of edge AI boards and accelerators. Full per-vendor pages, caveats, and buying advice are in hardware-landscape/. TOPS are vendor maxima at the stated precision. Real throughput differs, so always check independent FPS/token benchmarks.

Platform AI perf (vendor) Power On-board RAM GenAI-capable? Approx. price Best for
RPi 5 + AI HAT+ (Hailo-8L/8) 13 / 26 TOPS (INT8) ~5 W (NPU) — (uses Pi RAM) ❌ $70–110 + Pi Lowest-cost CV, learning
RPi 5 + AI HAT+ 2 (Hailo-10H) 40 TOPS (INT4) ~2.5 W (NPU) 8 GB on HAT ✅ (small LLM/VLM) ~$130 + Pi GenAI-at-edge on a budget
Jetson Orin Nano Super 67 TOPS (INT8) 7–25 W 8 GB shared ✅ (small) $249 Prototyping, robotics entry
Jetson AGX Orin up to 275 TOPS (INT8) 15–60 W 32/64 GB shared ✅ $$$ Multi-camera, mobile robots
Jetson AGX Thor (T5000) up to 2,070 FP4 TFLOPS 40–130 W 128 GB ✅✅ (VLA / LLM) $3,499 (dev kit) Humanoids, multi-sensor fusion
RK3588 boards 6 TOPS (INT8) NPU ~5–10 W board-dependent ⚠️ (small via rkllm) $80–200 Cheap Linux SBC + NPU
Intel Core Ultra + OpenVINO NPU + iGPU (varies) laptop/edge system RAM ✅ varies x86 edge servers, GenAI + CV
AMD Ryzen AI (XDNA 2) up to 50–60 TOPS (NPU) laptop/embedded system RAM ✅ varies AI PCs, industrial/automotive
Hailo-10H (discrete M.2) 40 TOPS INT4 / 20 INT8 ~2.5 W 4/8 GB on-module ✅ module Add GenAI to an x86/ARM host
Axelera Metis (M.2/PCIe) up to 214 / 856 TOPS M.2/card on-card ⚠️ card High-FPS multi-stream CV
Google Coral / Edge TPU 4 TOPS (INT8) ~2 W — ❌ ⚠️ EOL Legacy only, see deprecations

⚠️ Coral / Edge TPU is effectively abandoned (kernel driver removed from staging, no stock). Hailo is the practical successor. Details: renames-and-deprecations.md.


Which Inference Runtime Should I Use?

Full matrix and per-runtime pages are in runtimes-and-sdks/.

If your target is… Reach for Notes
NVIDIA GPU / Jetson TensorRT + (video) DeepStream DeepStream is built on GStreamer
Intel CPU / iGPU / NPU / Arc OpenVINO (+ OpenVINO GenAI) Model Optimizer & Open Model Zoo are deprecated → use OVC + optimum-intel
Qualcomm Snapdragon (Hexagon NPU) Qualcomm AI Hub → LiteRT / ONNX RT / QNN 150+ pre-optimized models
Microcontrollers (Arduino, ESP32, Cortex-M) LiteRT for Microcontrollers (tflite-micro) kilobytes–megabytes of RAM
Phones / mixed on-device LiteRT (was TensorFlow Lite) .tflite format unchanged
"Run my ONNX model anywhere" ONNX Runtime (pick an Execution Provider) CUDA, TensorRT, OpenVINO, QNN, DirectML, CoreML, WebGPU…
Rockchip RK3588 NPU RKNN-Toolkit2 / rkllm convert from ONNX/PyTorch

Renames and Deprecations

What changed in 2024–2026, so you don't follow a stale guide. Full page: renames-and-deprecations.md.

If a guide says… Today it's…
"TensorFlow Lite" / tflite-runtime LiteRT / ai-edge-litert (.tflite unchanged)
"buy a Coral USB Accelerator" abandoned, use Hailo instead
"Raspberry Pi AI Kit" AI HAT+ or AI HAT+ 2
"Jetson Orin Nano = 40 TOPS" 67 TOPS ("Super", JetPack 6.2)
"OpenVINO Model Optimizer / Open Model Zoo" OVC + Hugging Face / optimum-intel
"ONNX Runtime ArmNN EP" removed (1.26), use CPU EP (KleidiAI) or QNN
"ROS 2 Foxy/Galactic" EOL, use Lyrical Luth or Jazzy (LTS)

Repository Structure

What's in the box, and where to go next.

awesome-embedded-ai-stack/
├── getting-started/            # Day 0 → Day 7: choose a board, set it up
├── concepts-and-definitions/   # Cloud vs Edge vs Embedded; Physical AI; latency
├── hardware-landscape/         # Jetson, Pi+Hailo, RK3588, Intel/AMD, discrete NPUs
├── runtimes-and-sdks/          # ONNX Runtime, LiteRT, TensorRT/DeepStream, OpenVINO
├── edge-pipelines/             # camera → preproc → inference → postproc patterns
├── robotics-and-ros2/          # ROS 2 (Lyrical/Jazzy), Isaac ROS, the "3-computer" story
├── beginner-projects/          # reproducible quick-wins with a buy-list per project
├── deployment-and-optimization/# export, quantization, profiling, cloud→edge
├── industry-landscape/         # market size, vendor map, applications by vertical
├── awesome-resources/          # curated cross-cutting resources (docs, repos, videos, papers)
├── diagrams/                   # architecture & roadmap diagrams
├── knowledge-roadmap.md        # beginner → advanced learning path
├── renames-and-deprecations.md # what changed in 2024–2026 (read this!)
└── CONTRIBUTING.md             # how to add resources / projects (with quality bar)

Explore by section


Companion Guides

This hub is part of a three-repo set. Each companion goes deeper on one area and links back here for the shared Renames & Deprecations page.

  • Edge AI Field Guide: an 8-phase roadmap for TinyML and on-device AI, with runnable ONNX, TensorRT, Jetson, and Raspberry Pi projects.
  • Physical AI Field Guide: edge AI chips and NPUs, precision-labeled hardware comparisons, VLA and world models, and robotics applications.

FAQ

What is the best edge AI board for beginners in 2026?

For the lowest cost, a Raspberry Pi 5 with the AI HAT+ (Hailo-8L/8, 13–26 TOPS) is the cheapest credible way to run real-time computer vision. If you want CUDA, robotics with ROS 2, or small on-device LLMs, the NVIDIA Jetson Orin Nano Super ($249, 67 TOPS) is the better first board. See the decision tree.

Is Google Coral (Edge TPU) discontinued?

Effectively, yes. The Edge TPU PCIe driver was removed from the Linux kernel staging tree, upstream tflite-runtime hasn't carried Coral patches for years, and stock is essentially zero. Google's newer "Coral NPU" is RISC-V IP for chipmakers, not a replacement product. Use Hailo accelerators as the practical successor. See Renames & Deprecations.

What replaced TensorFlow Lite?

LiteRT. Google renamed TensorFlow Lite to LiteRT in September 2024. The .tflite model format is unchanged, but the Python package moved from tflite-runtime to ai-edge-litert. The microcontroller version is LiteRT for Microcontrollers (tflite-micro). See the LiteRT page.

What replaced the Raspberry Pi AI Kit?

The AI HAT+ (Hailo-8L at 13 TOPS or Hailo-8 at 26 TOPS) replaced the AI Kit for computer vision. The AI HAT+ 2 (Hailo-10H, 40 TOPS INT4, 8 GB on-board RAM) adds small LLMs and VLMs. See Raspberry Pi + Hailo.

What is the difference between edge AI and Physical AI?

Edge AI means running inference on or near the device that produces the data, such as a camera or sensor. Physical AI builds on edge AI: systems that perceive, reason, and act in the physical world, such as robots, drones, and autonomous machines. Every Physical AI system needs edge inference, but not every edge AI system acts physically. See Concepts & Definitions.

Which inference runtime should I use for edge AI?

Match the runtime to your silicon. Use TensorRT on NVIDIA Jetson, OpenVINO on Intel, RKNN-Toolkit2 on RK3588, HailoRT on Hailo, and LiteRT on phones and microcontrollers. If you need one model to run across many targets, export to ONNX and use ONNX Runtime with the right Execution Provider. See the runtime matrix.

Are TOPS numbers comparable between edge AI chips?

Not directly. TOPS is a theoretical peak at a specific precision (INT8, INT4, or FP4). Real FPS depends on memory bandwidth, on-chip SRAM, the model, and the toolchain, so two chips with the same TOPS can perform very differently. Treat TOPS as a rough class indicator and compare independent benchmarks. See Hardware Landscape.

Which ROS 2 version should I use for a new robot?

Choose a current LTS: Lyrical Luth (released May 2026) or Jazzy Jalisco. Avoid end-of-life distributions such as Foxy and Galactic, which many older tutorials still assume. See Robotics & ROS 2.


Contributing

This hub stays useful only if the community keeps it current. Good first contributions: add a new quick-win project, fix a broken link, add a board page, or correct a stale spec. The quality bar and step-by-step process are in CONTRIBUTING.md, and CI runs awesome-lint and a link checker on your PR.

Quality standards: authoritative or high-signal sources, currently maintained, specific links (no trailing slash), and a one-line description of why each resource is worth a click.

Look for good first issue and help wanted.

Support and Community

  • 🐛 Issues: report a broken link, a stale spec, or a bug via GitHub Issues (templates provided).
  • 💡 PRs: propose a resource or project; see CONTRIBUTING.md.
  • 🔁 Spot something outdated? Flagging a rename or deprecation is one of the most valuable contributions you can make.

Acknowledgements

Structure inspired by the broader Awesome movement and community knowledge hubs across the Jetson, OpenVINO, Hailo, ROS, and RK3588 ecosystems. All product names and trademarks belong to their respective owners; this is an independent, vendor-neutral resource.

Support This Project

If this hub saves you time, help others find it:

  • ⭐ Star the repo so more builders discover it.
  • 🔗 Share it with someone picking their first edge AI board.
  • 🤝 Contribute a project, a fix, or a freshly verified spec.

License

Code/snippets: MIT. Curated prose, tables, and learning materials: CC BY 4.0. See LICENSE.


Building toward Physical AI, one verified link at a time. Vendor-neutral • Kept current • Community-curated

About

Embodied Intelligence Guide: cross-vendor field guide to the embedded AI stack. Hardware, runtimes and a learning roadmap for NVIDIA Jetson, Raspberry Pi + Hailo, RK3588, OpenVINO, TensorRT, ONNX Runtime, LiteRT and ROS 2, with beginner projects and 2024–2026 deprecations tracked.

Topics

Resources

Code of conduct

Contributing

Stars

5 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors