-
Notifications
You must be signed in to change notification settings - Fork 15
Add http backend #202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add http backend #202
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
bfe79a7
http backend
almarklein f681a6e
Implement Python side
almarklein 73373ca
Merge branch 'main' into http
almarklein 88b010f
Progress
almarklein aef94fd
clean js and use binary frames instead of base64
almarklein e6aea2f
fix for uviloop
almarklein 0bb5929
Allow mounting in a larger app
almarklein c4f16d7
Add examples
almarklein 6e74944
Demonstrate customizing the page
almarklein 06d54fb
make the code multi-client aware
almarklein b42f4ad
shown in status whether active or passive
almarklein 81f94c9
Merge branch 'main' into http
almarklein e45e2a2
Small tweaks, renames, and fixes for closing connections
almarklein 451716e
Some cleanup
almarklein 9c742ee
Merge branch 'main' into http
almarklein 3545f77
Improvements, for multiple canvases, and stats
almarklein 9507205
docs
almarklein ff5183e
implement title, sizing etc
almarklein e624c2e
small tweak
almarklein 1e6528d
Add test
almarklein ed802cf
remove temp test code from example
almarklein d983c28
Apply suggestions from code review
almarklein 257d805
Merge branch 'main' into http
almarklein d4239fc
docs
almarklein File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| """ | ||
| Cube in the browser | ||
| ------------------- | ||
|
|
||
| Run a wgpu example with the http backend. Note that the http backend can be used | ||
| with most examples by simply using ``from rendercanvas.http import RenderCanvas, | ||
| loop``. This example also shows how the web-page can be customized. | ||
|
|
||
| Also see fastapi_app.py for how to integrate a rendercanvas into a larger web | ||
| application. | ||
| """ | ||
|
|
||
| # run_example = false | ||
|
|
||
| from rendercanvas.http import RenderCanvas, loop, resources | ||
| from rendercanvas.utils.cube import setup_drawing_sync | ||
| from rendercanvas.core.encoders import encode_png | ||
| import numpy as np | ||
|
|
||
|
|
||
| canvas = RenderCanvas( | ||
| title="The wgpu cube example on $backend", update_mode="continuous" | ||
| ) | ||
| draw_frame = setup_drawing_sync(canvas) | ||
| canvas.request_draw(draw_frame) | ||
|
|
||
|
|
||
| # Define custom HTML. This is optional. | ||
| html = """<!DOCTYPE html> | ||
| <html> | ||
| <head> | ||
| <title>RenderCanvas over http</title> | ||
| <script type='module' src='renderview.js'></script> | ||
| <script type='module' src='renderview-client.js'></script> | ||
| <link rel="stylesheet" href="renderview.css"> | ||
| <link rel="icon" href="logo.png"> | ||
| </head> | ||
| <body style='margin:0'> | ||
|
|
||
| <div id='canvas' class='' style='display:block; width:100vw; height:min(100vh,80vw); --line-thickness:0'> | ||
| Loading ... | ||
| </div> | ||
|
|
||
| <div id='status' style='position:fixed; top:10px; right:10px; background:#ccc; color:#000; padding:0.5em; font-family: monospace; border-radius:5px; '></div> | ||
| </body> | ||
| </html> | ||
| """ | ||
|
|
||
| # The resources is simply a dict that maps filenames to (content-type, body) tuples. | ||
| resources["index.html"] = "text/html", html | ||
|
|
||
|
|
||
| # You can also add new resources, like images or even extra web pages. | ||
| im = np.random.uniform(0, 255, (16, 16, 3)).astype(np.uint8) | ||
| resources["logo.png"] = "image/png", encode_png(im) | ||
|
|
||
|
|
||
| # The loop.run() of this backend uses uvicorn to start a webserver. | ||
| # The args are optional and default to "localhost" and port 60649 | ||
| loop.run("localhost", 8080) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| """ | ||
| FastAPI | ||
| ------- | ||
|
|
||
| Rendercanvas can do remote rendering as part of a web application. | ||
| It implements its own little ASGI application, that can be mounted | ||
| as part of a larger web application. This example demonstrates this | ||
| with the FastAPI web framework. | ||
|
|
||
| You can now run this like any AGI app, e.g. with uvicorn: | ||
|
|
||
| uvicorn fastapi_app:app | ||
|
|
||
| """ | ||
|
|
||
| from fastapi import FastAPI | ||
| from fastapi.responses import HTMLResponse | ||
| from rendercanvas.http import RenderCanvas, asgi | ||
| from rendercanvas.utils.cube import setup_drawing_sync | ||
|
|
||
|
|
||
| # FastAPI code | ||
|
|
||
| app = FastAPI() | ||
|
|
||
|
|
||
| @app.get("/", response_class=HTMLResponse) | ||
| async def home(): | ||
| return """ | ||
| <!DOCTYPE html> | ||
| <html> | ||
| <head> | ||
| <title>Test</title> | ||
| </head> | ||
| <body> | ||
| <h1>Hello world</h1> | ||
| <p>Head over to the <a href='rc/'>rendercanvas client<a></p> | ||
| </body> | ||
| </html> | ||
| """ | ||
|
|
||
|
|
||
| # Prepare a canvas to render something | ||
|
|
||
| canvas = RenderCanvas( | ||
| title="The wgpu cube example on $backend", update_mode="continuous" | ||
| ) | ||
| draw_frame = setup_drawing_sync(canvas) | ||
| canvas.request_draw(draw_frame) | ||
|
|
||
|
|
||
| # Mount rendercanvas in the app | ||
| app.mount("/rc", asgi) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,176 @@ | ||
| /************************************************************************************************* | ||
| renderview-client.js | ||
|
|
||
| Code to use renderview in a remote browser (rendercanvas http backend). | ||
| There are basically two approaches to take. Either use renderview-afm.js and re-use the render logic, | ||
| but implement an AFM host. Or directly attach a RenderView to a websocket. I went for the latter. Even | ||
| though that means duplicating some code, it looks like this leads to simpler code. | ||
|
|
||
| *************************************************************************************************/ | ||
|
|
||
| /* global BaseRenderView WebSocket */ | ||
|
|
||
| const wrapperElement = document.getElementById('canvas') | ||
| const statusElement = document.getElementById('status') | ||
| let view = null | ||
| let websocket = null | ||
| let isActive = null | ||
|
|
||
| updateStatus() | ||
| openWebsocketConnection() | ||
| window.openWebsocketConnection = openWebsocketConnection | ||
|
|
||
| class ClientRenderView extends BaseRenderView { | ||
| constructor (wrapperElement) { | ||
| wrapperElement.classList.add('renderview-wrapper') | ||
|
|
||
| // Create view element | ||
| const viewElement = document.createElement('img') | ||
| viewElement.decoding = 'sync' | ||
| viewElement.loading = 'eager' | ||
| viewElement.style.touchAction = 'none' // prevent default pan/zoom behavior | ||
| viewElement.ondragstart = () => false // prevent browser's built-in image drag | ||
|
|
||
| // Instantiate | ||
| super(viewElement, wrapperElement) | ||
| this.setThrottle(20) // 20ms -> max 50 move/wheel events per second | ||
|
|
||
| this.frames = [] | ||
| this.imgUpdatePending = false | ||
| this.lastSrc = null | ||
| } | ||
|
|
||
| onEvent (event) { | ||
| if (websocket !== null) { | ||
| websocket.send(JSON.stringify(event)) | ||
| } | ||
| } | ||
|
|
||
| requestAnimationFrame () { | ||
| // Request an animation frame. | ||
| // Before the anywidget refactor, we did this via a tiny delay, which supposedly made things more smooth, | ||
| // but it also increases the delay for a frame to hit the screen, and limits the max fps, so let's not do that. | ||
| if (!this.imgUpdatePending) { | ||
| this.imgUpdatePending = true | ||
| window.requestAnimationFrame(this.animate.bind(this)) | ||
| } | ||
| } | ||
|
|
||
| animate () { | ||
| this.imgUpdatePending = false | ||
| if (this.frames.length === 0) { return }; | ||
|
|
||
| // Pick the oldest frame from the stack, and get its source | ||
| const frame = this.frames.shift() | ||
| let newSrc | ||
| if (frame.buffers && frame.buffers.length > 0) { | ||
| const blob = new Blob([frame.buffers[0]], { type: frame.mimetype }) | ||
| newSrc = URL.createObjectURL(blob) | ||
| } else { | ||
| newSrc = frame.data_b64 | ||
| } | ||
|
|
||
| // Revoke last objectURL | ||
| URL.revokeObjectURL(this.lastSrc) | ||
| this.lastSrc = newSrc | ||
|
|
||
| // Update the image sources | ||
| view.viewElement.src = newSrc | ||
| view.viewElement.onload = this.requestAnimationFrame.bind(this) | ||
|
|
||
| // Let the server know we processed the image (even if it's not shown yet) | ||
| this.sendResponse(frame) | ||
| } | ||
|
|
||
| sendResponse (frame) { | ||
| // Let Python know what we have at the frame. | ||
| const event = { type: '_framefeedback', index: frame.index, timestamp: frame.timestamp, localtime: Date.now() / 1000 } | ||
| this.onEvent(event) | ||
| } | ||
| } | ||
|
|
||
| function updateStatus () { | ||
| if (statusElement === null) { return } | ||
|
|
||
| let activeText = '' | ||
| if (isActive !== null) { | ||
| activeText = isActive ? ' (active)' : '(passive)' | ||
| } | ||
|
|
||
| if (websocket === null) { | ||
| statusElement.innerHTML = "<span style='color:#900'>?</span> Disconnected <button onclick='openWebsocketConnection()'>reconnect</button>" | ||
| } else { | ||
| statusElement.innerHTML = `<span style='color:#090'>+</span> Connected ${activeText}` | ||
| } | ||
| } | ||
|
|
||
| function openWebsocketConnection () { | ||
| const ws = new WebSocket('ws://' + window.location.host + window.location.pathname) | ||
|
|
||
| ws.onopen = (e) => { | ||
| console.log('websocket opened') | ||
| websocket = ws | ||
| window.websocket = ws // allow manual closing to mimic lost connection | ||
| if (view === null) { | ||
| view = new ClientRenderView(wrapperElement) | ||
| console.log('created ClientRenderView') | ||
| } | ||
| updateStatus() | ||
| } | ||
| ws.onerror = (e) => { | ||
| console.log(`websocket error: ${e}`) | ||
| websocket = null | ||
| updateStatus() | ||
| } | ||
|
|
||
| let pendingMsg | ||
| ws.onmessage = (e) => { | ||
| let msg = null | ||
|
|
||
| // First some handling to support a message with buffers | ||
| if (typeof e.data === 'string' || e.data instanceof String) { | ||
| msg = JSON.parse(e.data) | ||
| if (msg.nbuffers && msg.nbuffers > 0) { | ||
| pendingMsg = msg | ||
| pendingMsg.buffers = [] | ||
| msg = null | ||
| } else { | ||
| pendingMsg = null // discard unfinished pending message (if any) | ||
| } | ||
| } else { // Blob | ||
| if (pendingMsg !== null) { | ||
| pendingMsg.buffers.push(e.data) | ||
| if (pendingMsg.buffers.length >= pendingMsg.nbuffers) { | ||
| msg = pendingMsg | ||
| pendingMsg = null | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (msg === null) { return } | ||
|
|
||
| // Process message | ||
| // console.log(msg) | ||
| if (msg.type === 'framebufferdata') { | ||
| view.frames.push(msg) | ||
| view.requestAnimationFrame() | ||
| } else if (msg.type === 'active') { | ||
| isActive = msg.value | ||
| updateStatus() | ||
| } else if (msg.type === 'cursor') { | ||
| view.setCursor(msg.value) | ||
| } else if (msg.type === 'title') { | ||
| view.setTitle(msg.value) | ||
| } else if (msg.type === 'css_width') { | ||
| view.setCssWidth(msg.value) | ||
| } else if (msg.type === 'css_height') { | ||
| view.setCssHeight(msg.value) | ||
| } | ||
| } | ||
|
|
||
| ws.onclose = (e) => { | ||
| console.log(`websocket closed: ${e.reason} (${e.code})`) | ||
| websocket = null | ||
| updateStatus() | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.