Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 51 additions & 3 deletions docs/backends.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,12 @@ The table below gives an overview of the names in the different ``rendercanvas``
| ``RenderCanvas`` (alias)
| ``loop`` (a ``StubLoop``)
- | For offscreen rendering.
* - ``http``
- | ``HttpRenderCanvas``
| ``RenderCanvas`` (alias)
| ``loop`` (an ``AsyncioLoop``)
- | Run an http server that streams the images
| to a view in the browser.
* - ``anywidget``
- | ``AnywidgetRenderCanvas``
| ``RenderCanvas`` (alias)
Expand All @@ -50,13 +56,14 @@ The table below gives an overview of the names in the different ``rendercanvas``
- | ``JupyterRenderCanvas``
| ``RenderCanvas`` (alias)
| ``loop`` (an ``AsyncioLoop``)
- | Integrate in notebooks via ``jupyter_rfb`` (deprecated).
- | Integrate in notebooks via ``jupyter_rfb``
| (deprecated).
* - ``pyodide``
- | ``PyodideRenderCanvas`` (toplevel)
| ``RenderCanvas`` (alias)
| ``loop`` (an ``AsyncioLoop``)
- | Backend when Python is running in the browser,
| via Pyodide or PyScript.
- | Backend when Python is running in the
| browser via Pyodide or PyScript.

There are also three loop-backends. These are mainly intended for use with the glfw backend:

Expand Down Expand Up @@ -272,6 +279,47 @@ object, but in some cases it's convenient to do so with a canvas-like API.
array = canvas.draw() # numpy array with shape (400, 500, 4)


Support for http / web (experimental)
-------------------------------------

The http backend can be used to open a canvas in a browser, streaming the images over a websocket.

.. autoclass:: rendercanvas.http.HttpRenderCanvas
:members:

It can be enabled simply by selecting the http backend:

.. code-block:: py

from rendercanvas.http import RenderCanvas

... your normal code as usual

The only dependency is ``uvicorn``, a low-level ASGI server.
It is also possible to customize the server's resources, see the ``cube_http.py`` example for details:

.. code-block:: py

from rendercanvas.http import RenderCanvas, loop, resources

resources["index.html"] = "text/html", your custom html
resources["logo.png"] = "image/png", encode_png(numpy_array)

Finally, you can embed it in a larger ASGI application. This requires the framework to support mounting an ASGI application:

.. code-block:: py

from fastapi import FastAPI
from rendercanvas.http import RenderCanvas, asgi

app = FastAPI()

...

# Attach the rendercanvas part at a sub path
app.mount("/rendercanvas_path", asgi)


Support for notebooks
---------------------

Expand Down
60 changes: 60 additions & 0 deletions examples/cube_http.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
"""
Cube in the browser
-------------------

Run a wgpu example with the http backend. Note that the http backend can be used
with most examples by simply using ``from rendercanvas.http import RenderCanvas,
loop``. This example also shows how the web-page can be customized.

Also see fastapi_app.py for how to integrate a rendercanvas into a larger web
application.
"""

# run_example = false

from rendercanvas.http import RenderCanvas, loop, resources
from rendercanvas.utils.cube import setup_drawing_sync
from rendercanvas.core.encoders import encode_png
import numpy as np


canvas = RenderCanvas(
title="The wgpu cube example on $backend", update_mode="continuous"
)
draw_frame = setup_drawing_sync(canvas)
canvas.request_draw(draw_frame)


# Define custom HTML. This is optional.
html = """<!DOCTYPE html>
<html>
<head>
<title>RenderCanvas over http</title>
<script type='module' src='renderview.js'></script>
<script type='module' src='renderview-client.js'></script>
<link rel="stylesheet" href="renderview.css">
<link rel="icon" href="logo.png">
</head>
<body style='margin:0'>

<div id='canvas' class='' style='display:block; width:100vw; height:min(100vh,80vw); --line-thickness:0'>
Loading ...
</div>

<div id='status' style='position:fixed; top:10px; right:10px; background:#ccc; color:#000; padding:0.5em; font-family: monospace; border-radius:5px; '></div>
</body>
</html>
"""

# The resources is simply a dict that maps filenames to (content-type, body) tuples.
resources["index.html"] = "text/html", html


# You can also add new resources, like images or even extra web pages.
im = np.random.uniform(0, 255, (16, 16, 3)).astype(np.uint8)
resources["logo.png"] = "image/png", encode_png(im)


# The loop.run() of this backend uses uvicorn to start a webserver.
# The args are optional and default to "localhost" and port 60649
loop.run("localhost", 8080)
53 changes: 53 additions & 0 deletions examples/fastapi_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
"""
FastAPI
-------

Rendercanvas can do remote rendering as part of a web application.
It implements its own little ASGI application, that can be mounted
as part of a larger web application. This example demonstrates this
with the FastAPI web framework.

You can now run this like any AGI app, e.g. with uvicorn:

uvicorn fastapi_app:app

"""

from fastapi import FastAPI
from fastapi.responses import HTMLResponse
from rendercanvas.http import RenderCanvas, asgi
from rendercanvas.utils.cube import setup_drawing_sync


# FastAPI code

app = FastAPI()


@app.get("/", response_class=HTMLResponse)
async def home():
return """
<!DOCTYPE html>
<html>
<head>
<title>Test</title>
</head>
<body>
<h1>Hello world</h1>
<p>Head over to the <a href='rc/'>rendercanvas client<a></p>
</body>
</html>
"""


# Prepare a canvas to render something

canvas = RenderCanvas(
title="The wgpu cube example on $backend", update_mode="continuous"
)
draw_frame = setup_drawing_sync(canvas)
canvas.request_draw(draw_frame)


# Mount rendercanvas in the app
app.mount("/rc", asgi)
176 changes: 176 additions & 0 deletions rendercanvas/core/renderview-client.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
/*************************************************************************************************
renderview-client.js

Code to use renderview in a remote browser (rendercanvas http backend).
There are basically two approaches to take. Either use renderview-afm.js and re-use the render logic,
but implement an AFM host. Or directly attach a RenderView to a websocket. I went for the latter. Even
though that means duplicating some code, it looks like this leads to simpler code.

*************************************************************************************************/

/* global BaseRenderView WebSocket */

const wrapperElement = document.getElementById('canvas')
const statusElement = document.getElementById('status')
let view = null
let websocket = null
let isActive = null

updateStatus()
openWebsocketConnection()
window.openWebsocketConnection = openWebsocketConnection

class ClientRenderView extends BaseRenderView {
constructor (wrapperElement) {
wrapperElement.classList.add('renderview-wrapper')

// Create view element
const viewElement = document.createElement('img')
viewElement.decoding = 'sync'
viewElement.loading = 'eager'
viewElement.style.touchAction = 'none' // prevent default pan/zoom behavior
viewElement.ondragstart = () => false // prevent browser's built-in image drag

// Instantiate
super(viewElement, wrapperElement)
this.setThrottle(20) // 20ms -> max 50 move/wheel events per second

this.frames = []
this.imgUpdatePending = false
this.lastSrc = null
}

onEvent (event) {
if (websocket !== null) {
websocket.send(JSON.stringify(event))
}
}

requestAnimationFrame () {
// Request an animation frame.
// Before the anywidget refactor, we did this via a tiny delay, which supposedly made things more smooth,
// but it also increases the delay for a frame to hit the screen, and limits the max fps, so let's not do that.
if (!this.imgUpdatePending) {
this.imgUpdatePending = true
window.requestAnimationFrame(this.animate.bind(this))
}
}

animate () {
this.imgUpdatePending = false
if (this.frames.length === 0) { return };

// Pick the oldest frame from the stack, and get its source
const frame = this.frames.shift()
let newSrc
if (frame.buffers && frame.buffers.length > 0) {
const blob = new Blob([frame.buffers[0]], { type: frame.mimetype })
newSrc = URL.createObjectURL(blob)
} else {
newSrc = frame.data_b64
}

// Revoke last objectURL
URL.revokeObjectURL(this.lastSrc)
this.lastSrc = newSrc

// Update the image sources
view.viewElement.src = newSrc
view.viewElement.onload = this.requestAnimationFrame.bind(this)

// Let the server know we processed the image (even if it's not shown yet)
this.sendResponse(frame)
}

sendResponse (frame) {
// Let Python know what we have at the frame.
const event = { type: '_framefeedback', index: frame.index, timestamp: frame.timestamp, localtime: Date.now() / 1000 }
this.onEvent(event)
}
}

function updateStatus () {
if (statusElement === null) { return }

let activeText = ''
if (isActive !== null) {
activeText = isActive ? ' (active)' : '(passive)'
}

if (websocket === null) {
statusElement.innerHTML = "<span style='color:#900'>?</span> Disconnected <button onclick='openWebsocketConnection()'>reconnect</button>"
} else {
statusElement.innerHTML = `<span style='color:#090'>+</span> Connected ${activeText}`
}
}

function openWebsocketConnection () {
const ws = new WebSocket('ws://' + window.location.host + window.location.pathname)

ws.onopen = (e) => {
console.log('websocket opened')
websocket = ws
window.websocket = ws // allow manual closing to mimic lost connection
if (view === null) {
view = new ClientRenderView(wrapperElement)
console.log('created ClientRenderView')
}
updateStatus()
}
ws.onerror = (e) => {
console.log(`websocket error: ${e}`)
websocket = null
updateStatus()
}

let pendingMsg
ws.onmessage = (e) => {
let msg = null

// First some handling to support a message with buffers
if (typeof e.data === 'string' || e.data instanceof String) {
msg = JSON.parse(e.data)
if (msg.nbuffers && msg.nbuffers > 0) {
pendingMsg = msg
pendingMsg.buffers = []
msg = null
} else {
pendingMsg = null // discard unfinished pending message (if any)
}
} else { // Blob
if (pendingMsg !== null) {
pendingMsg.buffers.push(e.data)
if (pendingMsg.buffers.length >= pendingMsg.nbuffers) {
msg = pendingMsg
pendingMsg = null
}
}
}

if (msg === null) { return }

// Process message
// console.log(msg)
if (msg.type === 'framebufferdata') {
view.frames.push(msg)
view.requestAnimationFrame()
} else if (msg.type === 'active') {
isActive = msg.value
updateStatus()
} else if (msg.type === 'cursor') {
view.setCursor(msg.value)
} else if (msg.type === 'title') {
view.setTitle(msg.value)
} else if (msg.type === 'css_width') {
view.setCssWidth(msg.value)
} else if (msg.type === 'css_height') {
view.setCssHeight(msg.value)
}
}

ws.onclose = (e) => {
console.log(`websocket closed: ${e.reason} (${e.code})`)
websocket = null
updateStatus()
}
}
4 changes: 2 additions & 2 deletions rendercanvas/core/renderview.js
Original file line number Diff line number Diff line change
Expand Up @@ -596,7 +596,7 @@ class BaseRenderView {
viewElement.addEventListener('pointerdown', (ev) => {
// When pointer is down, set focus to the focus-element.
if (!LOOKS_LIKE_MOBILE) {
this._focusElement.focus({ preventScroll: true, focusVisble: false })
this._focusElement.focus({ preventScroll: true, focusVisible: false })
Comment thread
almarklein marked this conversation as resolved.
}
// capture the pointing device.
// Because we capture the event, there will be no other events when buttons are pressed down,
Expand Down Expand Up @@ -917,7 +917,7 @@ class BaseRenderView {
}
// Prevent the text box from growing
if (!ev.isComposing) {
this.focus_el.value = ''
this._focusElement.value = ''
Comment thread
almarklein marked this conversation as resolved.
}

const event = {
Expand Down
Loading
Loading