Skip to content

[codex] Add Go2 visual event notifier demo#2286

Draft
Gshfrez wants to merge 2 commits into
dimensionalOS:mainfrom
Gshfrez:codex/go2-companion-alert
Draft

[codex] Add Go2 visual event notifier demo#2286
Gshfrez wants to merge 2 commits into
dimensionalOS:mainfrom
Gshfrez:codex/go2-companion-alert

Conversation

@Gshfrez
Copy link
Copy Markdown

@Gshfrez Gshfrez commented May 28, 2026

Summary

  • Add BrowserNotificationSkill, a reusable DimOS skill that serves an HTTPS phone/browser alert page and exposes notify_user(...) over MCP.
  • Add a lightweight OpenCV visual-event detector for stable red/yellow/green signals.
  • Add a direct Go2/Webcam demo script that watches camera frames and sends phone alerts through the local browser page and optional ntfy Android notifications.
  • Add docs, a demo signal page, and focused tests.

Why

This demonstrates a reusable DimOS application pattern:

robot perception -> stable visual event -> external app action

The recorded demo can use a traffic-light-style companion scenario, but the contribution generalizes to facility status lights, package alerts, warning signs, delivery events, and patrol notifications. The demo does not require an LLM API key, robot speech, navigation, or watch SDK integration.

Demo Path

  1. Connect the laptop to the Go2 network and discover the robot IP.
  2. Run:
    uv run python examples/go2_visual_event_notifier/go2_traffic_light_companion.py \
      --source go2 \
      --robot-ip "$ROBOT_IP" \
      --browser-alerts \
      --listen-host 0.0.0.0 \
      --browser-port 8450 \
      --ntfy-topic "$NTFY_TOPIC" \
      --stable-frames 3 \
      --cooldown-s 2 \
      --fps 5
  3. Open https://<laptop-ip>:8450/notify on the Samsung phone and tap Enable alerts.
  4. Open examples/go2_visual_event_notifier/demo_signals.html and show green/yellow/red to the Go2 camera.
  5. The phone receives Go, Caution, and Stop alerts when the state is stable.

Changes

  • TrafficLightColorDetector classifies RGB/BGR frames with HSV color masks and optional normalized ROI cropping.
  • StableStateDebouncer emits events only after repeated stable frames and cooldown.
  • go2_traffic_light_companion.py supports --source go2 through UnitreeWebRTCConnection and --source webcam for rehearsal.
  • demo_signals.html provides manual and auto-cycling red/yellow/green signals for recording.
  • docs/examples/go2_visual_event_notifier.md documents setup, fallbacks, and the 90-second demo script.

Validation

  • uv run python -m py_compile dimos/perception/visual_events/traffic_light.py examples/go2_visual_event_notifier/go2_traffic_light_companion.py
  • uv run pytest dimos/agents/skills/test_browser_notification_skill.py dimos/perception/visual_events/test_traffic_light_color_detector.py
  • uv run ruff check dimos/agents/skills/browser_notification_skill.py dimos/perception/visual_events examples/go2_visual_event_notifier/go2_traffic_light_companion.py
  • uv run python examples/go2_visual_event_notifier/go2_traffic_light_companion.py --help
  • Browser integration smoke: started the demo publisher, verified https://127.0.0.1:8450/notify returned 200 text/html, and queued a high-priority alert.

Safety Note

This is a prototype visual-event trigger and not a production traffic-safety system. The robot remains stationary; the demo only uses camera frames.

@Gshfrez Gshfrez changed the title [codex] Add Go2 companion alert notifications [codex] Add Go2 visual event notifier demo May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant