feat: Emulator streaming - #3211
Conversation
|
here is the broker that goes into the pcsx2 container if you want to take a look at the api inside |
|
I've finished setting up the integration for dolphin as well, I'm going to split it out into a different PR for each integration There is thankfully minimal changes but I turn some hardcoding into variables Once you get done looking through this I'll adapt the others and put them in as I have them completed |
9c8da66 to
6a0e5ff
Compare
|
@gantoine I changed my mind, and just added all the emulators I've gotten working along with their respective changes. I recruited Claude to help make my PR description a little better as well. Should outline everything I've touched and how it all works. |
b44b7dc to
6818507
Compare
Add /api/streaming endpoints for session claim/release, ROM launch, save states, volume control, and save-and-exit against per-platform emulator broker containers. Sessions are stored in Redis with atomic SET NX claims, owner-bound, and admin force-releasable. Config comes from config.yml (streaming.enabled + containers list) with a STREAMING_SAVE_TIMEOUT env override for slow-saving brokers. Includes 19 endpoint tests covering auth scopes, claim races, ownership, broker failure paths, and force-release.
Add a Pinia streaming store (config fetch, session lifecycle, broker controls), a v1 player view at rom/:rom/stream with save state and volume controls, and a cast button on PlayBtn for platforms with a configured streaming container. The route registers a v2 named view via v2For so the v2 fallback screen shows until a v2 player lands.
6818507 to
d94c5b6
Compare
Move the streaming player UI from v1 to v2 per the v1-frozen rule. The shared streaming store and backend stay canonical; only the frontend surface moves. - New v2 player view (Player/Stream.vue) porting the launch/playing/ error state machine, session claim, iframe UI auto-hide, volume, save/load state, fullscreen, save-and-exit and stop/release. - Fold streaming into the single Play action via useGameActions: routes to /rom/:id/stream when a container is configured, else EJS, else Ruffle. Streaming wins over in-browser emulation. - Hydrate streaming config in AppLayout (v2 users never mount v1 Main.vue, where v1 put the fetchConfig call). - Make the rom/:rom/stream route v2-only (v1 default falls back to Home). - Add stream-* i18n keys to all 18 locales. - Remove the v1 Stream player, PlayBtn cast button and Main.vue fetchConfig wiring. Generated-By: PostHog Code Task-Id: e091168f-69fd-4d4a-9f12-d473dd3f17c9
Address review findings on the streaming framework. Backend (backend/endpoints/streaming.py): - Session keys now carry a TTL (6h) so an abandoned session (broker dead / backend crashed) eventually frees the container instead of wedging it until an admin force-releases; control calls refresh the TTL so active sessions never expire. - _container_for_platform validates each entry has a scheme-bearing host, skipping malformed ones (mirroring /config) so claim/control return a clean 404 instead of a 500 KeyError on container["host"]. - _parse_host_url / _derive_broker_host reject schemeless hosts that would produce the broken "//None:8000/..." broker URL and colliding session keys; _broker_url raises a clear 502 when no usable host. - save-and-exit with wait=false drains the session (short-lived key) instead of deleting it outright, so a concurrent new claim can't /launch on top of a not-yet-dead emulator and lose the in-flight save. - New tests cover schemeless/missing-host skips, the claim TTL, and the wait=false drain window. Frontend: - stores/streaming.ts: releaseSession/saveAndExit drop the local session record only after the backend confirms, so a failed release no longer leaves the user wedged behind their own still-held session. - v2/views/Player/Stream.vue: disambiguate the 404 from claimSession (ROM not found vs container not configured) so the message matches the real failure; new stream-error-rom-not-found i18n key in all 18 locales. Generated-By: PostHog Code Task-Id: e091168f-69fd-4d4a-9f12-d473dd3f17c9
Replace English placeholder values for the stream-* keys in play.json with translations for all 16 non-English locales. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move raw HTTP calls out of the streaming store into a dedicated services/api/streaming.ts module, matching the services/api/* convention. Thin wrappers with no store state (setVolume, setMute, saveState, loadState) are dropped from the store and called directly via streamingApi in Stream.vue; the store keeps only state-managing actions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the repeated urllib request construction (URL, secret, payload, headers, urlopen, parsing) shared by the six broker functions into _broker_request and its best-effort _broker_request_safe variant. The best-effort wrappers collapse to a few lines each and the nosec B310 annotation now lives in one place. Slim the tests with a _streaming() context manager plus _auth/_claim_ok helpers, removing the repeated cm.get_config patch boilerplate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-platform slot capabilities were duplicated across the frontend store's hardcoded platformCapabilities() map and the backend's pydantic Field bounds, independent of the broker's own ceiling. Consolidate the RomM-side knowledge into one _PLATFORM_CAPABILITIES table in the streaming endpoint: - /config now ships each container's capabilities, and the frontend selector reads them instead of keeping its own copy. - The SaveState/LoadState request bounds derive from the table, and the routes validate the exact per-platform ceiling before calling the broker (clean 422 instead of a broker 502). The broker remains the ultimate enforcer; RomM no longer keeps two independent hardcoded copies of its slot semantics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@LoneAngelFayt alright i'm happy with the state of this PR, want to give it a spin and see if everything still works? |
|
Gladly! |
|
@gantoine looks like its all working. One thing to note though. Since the v2 migration, it looks like controller inputs are being snagged by the UI as well as the stream. I fix this in a commit I have later, along with a lot of other cool features I've been working on. Fix: re-add the setPlaying wiring in Stream.vue (set it while launching/playing, clear it on teardown): in case you wanted to just drop in the fix for this, or we can let it slide till the next PR i have around the corner |
|
@LoneAngelFayt two things:
|
|
@gantoine i got you. I can push everything to this branch. I also cleaned up the way the stream page looks, as well as adding some save/state sync since that was opened up since this originally dropped. Need me to prune anything out or are you ok with a larger review all at once? |
useGamepad gates its pad-to-UI translation on the playing store flag, but the v2 Stream player never set it, so button presses kept navigating the app (and a Back/Start press could tear the session down mid-game). Flag the session while launching or playing and clear it on unmount.
The g-prefix sequences and the / search shortcut listened during play, so pressing common game keys (g then h/p/c, or /) navigated away and killed the session. Gate the handler on the playing store flag. EmulatorJS set the flag on play but never cleared it on unmount, leaving pad and hotkey navigation dead after exiting a game. Ruffle never set it at all, so Flash sessions had no protection. Both players now flag the session on play and release it on unmount.
|
got them, it was the global hotkeys that were slipping through (G and H most likely culprits) found ruffle didn't even set playing tag at all so that was just waiting to fail sometime. now they all set playing flag when a game starts and release it on exit, tested end to end. Should be good to go. |
|
A team member linked me to this PR, cool stuff. If you need anything upstream we can add things like env vars to pass in rom paths to pass to the emulator that is completely possible just let me know the spec. I do something similar with sealskin e.g. https://github.com/linuxserver/sealskin-apps/blob/master/autostart/eden-wayland. Just need new env vars in the autostart files to ingest paths to stuff to launch and possible raw CLI for specific launch options. Just a heads up the RPCS3 image needs some attention I will try to prioritize it as it is pinned back now, last time it was at head a bunch of stuff was busted because it expected more host level access and docker was not cutting it. I do not have much more of a summary than that at this time. (the gamepad interposer was a major blocker, but lots of /sys probing stuff) Also wanted to mention https://github.com/linuxserver/docker-shadps4, it runs a lot of stuff surprisingly well I even got a bunch of indies running on my little intel n97 box. Another mention is https://github.com/linuxserver/docker-webstation, I personally use this with ESDE fully setup on it, but it might be easier to focus on one container that has every single emu in it like this does and this has the flexibility to add odd ones if needed. The gpu setup on wayland/nvidia can get hairy let me know if you need any help with that. with 595+ drivers dummy plugs and kernel params are not needed anymore I can get it running with just : On some hosts need to run Another thing that comes to mind is more automated emulator setup pointing at files (bios, dash, firmware) Sorry for the rant, but I literally made all this stuff to play games with my friends after hitting a wall with emscripten on my old emulatorjs fork based around making a coinops web version https://github.com/linuxserver/emulatorjs. So this type of integration is one that is particularly of interest to me. Couch play is completely possible with controller passing and is why the sealskin collaboration feature exists in the first place. https://github.com/selkies-project/sealskin/blob/master/server/app/collaboration.py. This is all open feel free to rip anything you need for your solution no accreditation is needed. If you have any questions feel free to reach out on our lsio discord I will setup a collab and show you what I am talking about. |
@thelamer This is great! I'll reach out! |
|
Just looking over the integration I think you might be able to save a lot of headache by having the broker be a single container with a docker socket mounted into it to launch containers and act as a proxy to them. That way you don't have to worry about container states and every container is ephemeral for a play session. I refer back to https://github.com/selkies-project/sealskin/blob/master/server/app/collaboration.py and https://github.com/selkies-project/sealskin/blob/master/server/app/providers/docker_provider.py. Basically the idea would be rip out the core components of sealskin that are needed to make a broker. (caddy integration, container launching, and collaboration) like a mini sealskin that passes in env vars and mounts to individual containers on demand and can run multiple container sessions until the user "powers them off". The broker could have its own web interface for configuring how containers are launched (gpu to use and defaults). Then I can modify the emulator images to accept a series of env vars to pass into the underlying app or to make it more universal in init have it detect a specific toml/yaml file and use that for launching the underlying emulator so you could have the flexibility of modifying it to your needs without needs for container changes. Just spit balling here, more expand the broker to be its own universal thing instead of emulator specific and most of the hard work is already done what is left is boring data structure stuff. |
Description
This PR adds a streaming framework to RomM that enables launching games directly from the RomM UI into a native emulator running in a separate container, with full session management, save state control, and volume control, all without leaving the browser.
The emulator runs in a linuxserver Docker container with a Selkies WebRTC stream. A small HTTP broker sidecar inside that container handles game launch, extraction, and I/O. RomM communicates with the broker via a new
/api/streamingbackend endpoint and displays the stream in an embedded player.This initial PR covers four emulators. Additional integrations (PS3/rpcs3, etc.) will follow in separate PRs once the framework is established.
Supported emulators
ps2ngc,wii,wiiuxboxswitchArchitecture
Configuration (
config.yml)Each emulator is its own container entry. Multiple platforms can share a container (e.g.
ngc/wii/wiiuall pointing at the same Dolphin instance) or use separate ones. Thehostis the URL the browser loads the stream from;broker_hostis the server-side URL RomM uses to send commands.Two optional env vars:
BROKER_SECRETauthenticates broker API calls (set on both the RomM container and the emulator container), andSTREAMING_SAVE_TIMEOUTraises the save-and-exit wait for brokers that save slowly (default 45s).Changes
Backend
backend/endpoints/streaming.py- new router: session management, broker proxy, config endpoint. Sessions are claimed by ROM id (the filesystem path is derived server-side from the database, never taken from the client), stored in Redis via atomicSET NX(multi-worker safe), and bound to the claiming user; only the owner or an admin can control or release one.backend/tests/endpoints/test_streaming.py- 19 endpoint tests: auth scopes, claim races, ownership, broker failure paths, force-release.backend/config/config_manager.py-STREAMING_ENABLED+STREAMING_CONTAINERSconfig fieldsbackend/main.py- registers streaming routerFrontend
frontend/src/stores/streaming.ts- Pinia store: config, session lifecycle, per-platform capabilitiesfrontend/src/views/Player/Stream/Player.vue- streaming player view: Selkies iframe, loading overlay, save state controls, volume/mutefrontend/src/components/common/Game/PlayBtn.vue- "Play on [label]" button appears when a streaming container is configured for the platformfrontend/src/layouts/Main.vue- streaming config fetched on app startfrontend/src/plugins/router.ts-rom/:rom/streamrouteConfig
examples/config.example.yml- streaming config example with commentsenv.template-BROKER_SECRETandSTREAMING_SAVE_TIMEOUTvariablesThe player view follows the existing v1 player pattern (same shape as the EmulatorJS/Ruffle views). The route also registers a v2 named view via
v2For, so v2 users get the standard "not ready yet" fallback; a native v2 player is planned as a follow-up once this framework lands.Broker containers
Each emulator has a companion Docker mod repo with the broker sidecar (stdlib-only Python, no extra dependencies):
Each repo includes a
docker-compose.ymlshowing the full setup.UX flow
AI assistance disclosure
This PR was developed with substantial AI assistance (Claude Code). I designed the architecture, made the design decisions, and tested everything against real emulator containers on my own hardware; the majority of the code was written by the AI under my direction and review, and the branch went through multiple AI-assisted review passes before submission.
Checklist
Please check all that apply.
Screenshots (if applicable)