Edge AI & Physical AI, from sensor → inference → actuation: NVIDIA Jetson, Raspberry Pi + Hailo, RK3588, OpenVINO, TensorRT, ONNX Runtime, LiteRT, and ROS 2.
A vendor-neutral field guide to Edge AI, Embedded AI, and Physical AI. It covers 10+ hardware platforms, 6+ inference runtimes, a Phase 0→7 learning roadmap, reproducible beginner projects, a curated resource collection, and the 2024–2026 renames and deprecations, so you never follow a dead-end tutorial.
Embodied Intelligence Guide is a practical, cross-vendor guide to the embedded AI stack: running AI models on edge devices and robots. It helps you choose an edge AI board (Jetson, Raspberry Pi 5 + Hailo, RK3588, Intel, AMD, or a discrete NPU), pick the right inference runtime (TensorRT, OpenVINO, ONNX Runtime, LiteRT, RKNN), build a real-time camera → inference → action pipeline, and grow from first demo to ROS 2 robotics and Physical AI. Hardware specs and prices were last verified in May 2026.
Note
The edge AI landscape changed a lot in 2024–2026. TensorFlow Lite is now LiteRT; Google Coral / Edge TPU is effectively discontinued; the Raspberry Pi AI Kit was replaced by the AI HAT+ / AI HAT+ 2; Jetson AGX Thor shipped; and the newest ROS 2 LTS is Lyrical Luth. Read Renames & Deprecations → before you buy hardware or pick a runtime.
- 🎯 What This Hub Solves
- 🧩 What Is Edge AI, Embedded AI, and Physical AI?
- 🚀 Start Here
- 🧭 Learning Roadmap
- 🔌 Edge AI Hardware at a Glance
- ⚙️ Which Inference Runtime Should I Use?
- 🔁 Renames and Deprecations
- 📚 Repository Structure
- 🔗 Companion Guides
- ❓ FAQ
- 🤝 Contributing
- 📞 Support and Community
- 🙏 Acknowledgements
- ⭐ Support This Project
- 📄 License
The questions that block people getting started in edge AI, and where this hub answers them.
Edge AI and Physical AI knowledge is scattered across vendor docs, blog posts, YouTube, and one-off sample repos, and each assumes you've already picked their stack. Much of it is also out of date (renamed SDKs, discontinued hardware). This hub fixes three things:
- Problem: "TOPS" marketing numbers don't map to real FPS, memory limits aren't obvious, and it's unclear which runtime works on which chip.
- Solution: one-screen comparison tables plus per-vendor pages with caveats. → Hardware Landscape · Runtimes & SDKs
- Problem: tutorials stop at a single inference call. Few show the full camera → preprocess → inference → postprocess → action pipeline, or how to build a robot perception graph.
- Solution: pipeline patterns and reproducible projects you can run on hardware you can buy today. → Edge Pipelines · Beginner Quick-Wins
- Problem: many tutorials online still tell you to buy a Coral or
pip install tflite-runtime. Both are effectively dead. - Solution: a living "what changed" page and a structured learning order. → Renames & Deprecations · Knowledge Roadmap
The 60-second mental model before you pick hardware. Full version in Concepts & Definitions.
- Cloud AI: inference in a datacenter. Maximum compute, but higher latency, and data leaves the device.
- Edge AI: inference on or next to the device producing the data. Low latency, privacy, bandwidth savings, and it works offline.
- Embedded AI / TinyML: inference on microcontrollers and DSPs with kilobytes of memory and milliwatts of power.
- Physical AI: systems that perceive, reason about, and act in the physical world (robots, drones, autonomous machines). These are increasingly driven by vision-language-action (VLA) and world foundation models running at the edge.
- Real-time control: closed-loop robotics needs deterministic, bounded latency (high-end platforms target multi-sensor fusion in well under ~10 ms).
- Privacy: faces, license plates, and patient or factory data never leave the device.
- Bandwidth and cost: send insights, not raw video, to the cloud.
- Availability: it keeps working when the network doesn't.
New to edge AI? Follow these five steps in order.
| Step | Do this | Go to |
|---|---|---|
| 1. Understand the terms | Cloud vs Edge vs Embedded vs Physical AI (10-minute read) | Concepts & Definitions → |
| 2. Pick a board | Use the decision tree to choose your first edge AI board | Getting Started → |
| 3. Get a quick win | Run real-time camera object detection on hardware you can buy today | Beginner Projects → |
| 4. Follow the roadmap | Beginner → advanced, phase by phase | Knowledge Roadmap → |
| 5. Go deep | Runtimes, pipelines, robotics, deployment, and the market | Repository Structure ↓ |
Three recommended tracks (full detail in Getting Started):
- 🟢 Maker / lowest cost: Raspberry Pi 5 + AI HAT+ (Hailo). Quick-win →
- 🟡 Prototyping / robotics: NVIDIA Jetson Orin Nano Super. Quick-win →
- 🔴 Advanced / humanoid and multi-sensor: Jetson AGX Thor + Isaac ROS. Robotics →
From zero to Physical AI in eight phases. The phases are ordered but modular, so skip what you already know.
flowchart LR
A["Phase 0\nLinux + Python/C++\nfoundations"] --> B["Phase 1\nEmbedded basics\n& TinyML"]
B --> C["Phase 2\nInference runtimes\nONNX RT · LiteRT · TensorRT · OpenVINO"]
C --> D["Phase 3\nMedia pipelines\nGStreamer · DeepStream"]
D --> E["Phase 4\nCUDA & HW\nacceleration"]
E --> F["Phase 5\nROS 2 &\nrobotics inference"]
F --> G["Phase 6\nEdge deployment\n& optimization"]
G --> H["Phase 7\nReal-world\nPhysical AI pipelines"]
Detailed, link-by-link version: knowledge-roadmap.md →
A starter comparison of edge AI boards and accelerators. Full per-vendor pages, caveats, and buying advice are in hardware-landscape/. TOPS are vendor maxima at the stated precision. Real throughput differs, so always check independent FPS/token benchmarks.
| Platform | AI perf (vendor) | Power | On-board RAM | GenAI-capable? | Approx. price | Best for |
|---|---|---|---|---|---|---|
| RPi 5 + AI HAT+ (Hailo-8L/8) | 13 / 26 TOPS (INT8) | ~5 W (NPU) | — (uses Pi RAM) | ❌ | $70–110 + Pi | Lowest-cost CV, learning |
| RPi 5 + AI HAT+ 2 (Hailo-10H) | 40 TOPS (INT4) | ~2.5 W (NPU) | 8 GB on HAT | ✅ (small LLM/VLM) | ~$130 + Pi | GenAI-at-edge on a budget |
| Jetson Orin Nano Super | 67 TOPS (INT8) | 7–25 W | 8 GB shared | ✅ (small) | $249 | Prototyping, robotics entry |
| Jetson AGX Orin | up to 275 TOPS (INT8) | 15–60 W | 32/64 GB shared | ✅ | $$$ | Multi-camera, mobile robots |
| Jetson AGX Thor (T5000) | up to 2,070 FP4 TFLOPS | 40–130 W | 128 GB | ✅✅ (VLA / LLM) | $3,499 (dev kit) | Humanoids, multi-sensor fusion |
| RK3588 boards | 6 TOPS (INT8) NPU | ~5–10 W | board-dependent | $80–200 | Cheap Linux SBC + NPU | |
| Intel Core Ultra + OpenVINO | NPU + iGPU (varies) | laptop/edge | system RAM | ✅ | varies | x86 edge servers, GenAI + CV |
| AMD Ryzen AI (XDNA 2) | up to 50–60 TOPS (NPU) | laptop/embedded | system RAM | ✅ | varies | AI PCs, industrial/automotive |
| Hailo-10H (discrete M.2) | 40 TOPS INT4 / 20 INT8 | ~2.5 W | 4/8 GB on-module | ✅ | module | Add GenAI to an x86/ARM host |
| Axelera Metis (M.2/PCIe) | up to 214 / 856 TOPS | M.2/card | on-card | card | High-FPS multi-stream CV | |
| Google Coral / Edge TPU | 4 TOPS (INT8) | ~2 W | — | ❌ | Legacy only, see deprecations |
⚠️ Coral / Edge TPU is effectively abandoned (kernel driver removed from staging, no stock). Hailo is the practical successor. Details: renames-and-deprecations.md.
Full matrix and per-runtime pages are in runtimes-and-sdks/.
| If your target is… | Reach for | Notes |
|---|---|---|
| NVIDIA GPU / Jetson | TensorRT + (video) DeepStream | DeepStream is built on GStreamer |
| Intel CPU / iGPU / NPU / Arc | OpenVINO (+ OpenVINO GenAI) | Model Optimizer & Open Model Zoo are deprecated → use OVC + optimum-intel |
| Qualcomm Snapdragon (Hexagon NPU) | Qualcomm AI Hub → LiteRT / ONNX RT / QNN | 150+ pre-optimized models |
| Microcontrollers (Arduino, ESP32, Cortex-M) | LiteRT for Microcontrollers (tflite-micro) |
kilobytes–megabytes of RAM |
| Phones / mixed on-device | LiteRT (was TensorFlow Lite) | .tflite format unchanged |
| "Run my ONNX model anywhere" | ONNX Runtime (pick an Execution Provider) | CUDA, TensorRT, OpenVINO, QNN, DirectML, CoreML, WebGPU… |
| Rockchip RK3588 NPU | RKNN-Toolkit2 / rkllm | convert from ONNX/PyTorch |
What changed in 2024–2026, so you don't follow a stale guide. Full page: renames-and-deprecations.md.
| If a guide says… | Today it's… |
|---|---|
"TensorFlow Lite" / tflite-runtime |
LiteRT / ai-edge-litert (.tflite unchanged) |
| "buy a Coral USB Accelerator" | abandoned, use Hailo instead |
| "Raspberry Pi AI Kit" | AI HAT+ or AI HAT+ 2 |
| "Jetson Orin Nano = 40 TOPS" | 67 TOPS ("Super", JetPack 6.2) |
| "OpenVINO Model Optimizer / Open Model Zoo" | OVC + Hugging Face / optimum-intel |
| "ONNX Runtime ArmNN EP" | removed (1.26), use CPU EP (KleidiAI) or QNN |
| "ROS 2 Foxy/Galactic" | EOL, use Lyrical Luth or Jazzy (LTS) |
What's in the box, and where to go next.
awesome-embedded-ai-stack/
├── getting-started/ # Day 0 → Day 7: choose a board, set it up
├── concepts-and-definitions/ # Cloud vs Edge vs Embedded; Physical AI; latency
├── hardware-landscape/ # Jetson, Pi+Hailo, RK3588, Intel/AMD, discrete NPUs
├── runtimes-and-sdks/ # ONNX Runtime, LiteRT, TensorRT/DeepStream, OpenVINO
├── edge-pipelines/ # camera → preproc → inference → postproc patterns
├── robotics-and-ros2/ # ROS 2 (Lyrical/Jazzy), Isaac ROS, the "3-computer" story
├── beginner-projects/ # reproducible quick-wins with a buy-list per project
├── deployment-and-optimization/# export, quantization, profiling, cloud→edge
├── industry-landscape/ # market size, vendor map, applications by vertical
├── awesome-resources/ # curated cross-cutting resources (docs, repos, videos, papers)
├── diagrams/ # architecture & roadmap diagrams
├── knowledge-roadmap.md # beginner → advanced learning path
├── renames-and-deprecations.md # what changed in 2024–2026 (read this!)
└── CONTRIBUTING.md # how to add resources / projects (with quality bar)
- 🏁 Getting Started: pick your first edge AI board with a decision tree.
- 🧩 Concepts & Definitions: edge AI, embedded AI, TinyML, and Physical AI explained.
- 🔌 Hardware Landscape: Jetson, Raspberry Pi + Hailo, RK3588, Intel & AMD, discrete NPUs.
- ⚙️ Runtimes & SDKs: ONNX Runtime, LiteRT, TensorRT & DeepStream, OpenVINO.
- 🎥 Edge Pipelines: GStreamer & DeepStream video analytics and ROS 2 perception.
- 🤖 Robotics & ROS 2: Isaac ROS, NITROS, Holoscan, and the Physical AI workflow.
- 🛠️ Beginner Projects: weekend quick-wins on real hardware.
- 🚢 Deployment & Optimization: export, quantize, profile, ship.
- 📈 Industry Landscape: market data, vendors, and where the field is heading.
- ⭐ Awesome Resources: the curated resource collection.
This hub is part of a three-repo set. Each companion goes deeper on one area and links back here for the shared Renames & Deprecations page.
- Edge AI Field Guide: an 8-phase roadmap for TinyML and on-device AI, with runnable ONNX, TensorRT, Jetson, and Raspberry Pi projects.
- Physical AI Field Guide: edge AI chips and NPUs, precision-labeled hardware comparisons, VLA and world models, and robotics applications.
For the lowest cost, a Raspberry Pi 5 with the AI HAT+ (Hailo-8L/8, 13–26 TOPS) is the cheapest credible way to run real-time computer vision. If you want CUDA, robotics with ROS 2, or small on-device LLMs, the NVIDIA Jetson Orin Nano Super ($249, 67 TOPS) is the better first board. See the decision tree.
Effectively, yes. The Edge TPU PCIe driver was removed from the Linux kernel staging tree, upstream tflite-runtime hasn't carried Coral patches for years, and stock is essentially zero. Google's newer "Coral NPU" is RISC-V IP for chipmakers, not a replacement product. Use Hailo accelerators as the practical successor. See Renames & Deprecations.
LiteRT. Google renamed TensorFlow Lite to LiteRT in September 2024. The .tflite model format is unchanged, but the Python package moved from tflite-runtime to ai-edge-litert. The microcontroller version is LiteRT for Microcontrollers (tflite-micro). See the LiteRT page.
The AI HAT+ (Hailo-8L at 13 TOPS or Hailo-8 at 26 TOPS) replaced the AI Kit for computer vision. The AI HAT+ 2 (Hailo-10H, 40 TOPS INT4, 8 GB on-board RAM) adds small LLMs and VLMs. See Raspberry Pi + Hailo.
Edge AI means running inference on or near the device that produces the data, such as a camera or sensor. Physical AI builds on edge AI: systems that perceive, reason, and act in the physical world, such as robots, drones, and autonomous machines. Every Physical AI system needs edge inference, but not every edge AI system acts physically. See Concepts & Definitions.
Match the runtime to your silicon. Use TensorRT on NVIDIA Jetson, OpenVINO on Intel, RKNN-Toolkit2 on RK3588, HailoRT on Hailo, and LiteRT on phones and microcontrollers. If you need one model to run across many targets, export to ONNX and use ONNX Runtime with the right Execution Provider. See the runtime matrix.
Not directly. TOPS is a theoretical peak at a specific precision (INT8, INT4, or FP4). Real FPS depends on memory bandwidth, on-chip SRAM, the model, and the toolchain, so two chips with the same TOPS can perform very differently. Treat TOPS as a rough class indicator and compare independent benchmarks. See Hardware Landscape.
Choose a current LTS: Lyrical Luth (released May 2026) or Jazzy Jalisco. Avoid end-of-life distributions such as Foxy and Galactic, which many older tutorials still assume. See Robotics & ROS 2.
This hub stays useful only if the community keeps it current. Good first contributions: add a new quick-win project, fix a broken link, add a board page, or correct a stale spec. The quality bar and step-by-step process are in CONTRIBUTING.md, and CI runs awesome-lint and a link checker on your PR.
Quality standards: authoritative or high-signal sources, currently maintained, specific links (no trailing slash), and a one-line description of why each resource is worth a click.
Look for good first issue and help wanted.
- 🐛 Issues: report a broken link, a stale spec, or a bug via GitHub Issues (templates provided).
- 💡 PRs: propose a resource or project; see CONTRIBUTING.md.
- 🔁 Spot something outdated? Flagging a rename or deprecation is one of the most valuable contributions you can make.
Structure inspired by the broader Awesome movement and community knowledge hubs across the Jetson, OpenVINO, Hailo, ROS, and RK3588 ecosystems. All product names and trademarks belong to their respective owners; this is an independent, vendor-neutral resource.
If this hub saves you time, help others find it:
- ⭐ Star the repo so more builders discover it.
- 🔗 Share it with someone picking their first edge AI board.
- 🤝 Contribute a project, a fix, or a freshly verified spec.
Code/snippets: MIT. Curated prose, tables, and learning materials: CC BY 4.0. See LICENSE.
Building toward Physical AI, one verified link at a time. Vendor-neutral • Kept current • Community-curated