Skip to content

feat(windows): add DirectComposition (visual) hosting for WebView2 - #1762

Open
jerryoh512 wants to merge 1 commit into
tauri-apps:devfrom
jerryoh512:feat/webview2-composition-hosting
Open

feat(windows): add DirectComposition (visual) hosting for WebView2#1762
jerryoh512 wants to merge 1 commit into
tauri-apps:devfrom
jerryoh512:feat/webview2-composition-hosting

Conversation

@jerryoh512

@jerryoh512 jerryoh512 commented Jul 7, 2026

Copy link
Copy Markdown

Motivation

WebView2 supports two hosting models: the windowed controller wry uses today, and a composition controller (CreateCoreWebView2CompositionController) that renders into a caller-supplied DirectComposition visual instead of an HWND. Composition hosting is what makes layered DirectComposition apps possible: native content (e.g. a wgpu/D3D swapchain on a sibling visual) drawn beneath a transparent webview in the same window, with the webview used as the UI chrome layer. The windowed model cannot express this — a child HWND always occludes everything under it.

This PR adds composition hosting as an opt-in mode. It is used in production by a Windows PDF application (Colophon) that renders documents on a native wgpu surface beneath a transparent Tauri/wry webview UI.

Related: #391 (offscreen rendering) asks for a different but adjacent capability; the composition controller introduced here is also the natural substrate for future texture-stream/offscreen work.

API

Two entry points, both #[cfg(windows)]:

  • WebViewBuilderExtWindows::with_composition_visual_target(visual: IUnknown) — supplies an IDCompositionVisual (any version, passed as IUnknown so no new windows-crate interface features leak into wry's public API). When set, the webview is created through CreateCoreWebView2CompositionController with that visual as the root visual target, instead of creating a WRY_WEBVIEW child window.

  • register_composition_visual_target(hwnd: isize, visual: IUnknown) (crate-root re-export) — the out-of-band counterpart for embedding layers that construct the WebViewBuilder internally and never expose it to the application, most importantly tauri-runtime-wry. The host registers the visual for its window before asking the embedder to create the webview; webview creation on that HWND consumes the entry (once) when the builder carries no target of its own. The registry is per-HWND (concurrent window creations cannot cross-wire), consume-once, and UI-thread-only by construction (thread-local; registration and creation both happen on the UI thread, which is also the COM STA the visual lives in). A builder-supplied target always takes precedence. This is the piece that lets a Tauri app opt a window into composition hosting today with no tauri-runtime-wry changes; if upstreaming a first-class knob to tauri-runtime-wry later, the builder method is the primitive it would call.

Behavior in composition mode

  • No container HWND. InnerWebView.hwnd is the host window itself and is_composition is set. Guarded on that flag: set_bounds only updates controller bounds (never SetWindowPos — that would move the host window), set_visible only toggles controller.SetIsVisible (never ShowWindow on the host), reparent returns Error::UnsupportedWindowHandle (there is no child window to move; the host subclass and visual tree are bound to the original window), and Drop detaches the composition subclass instead of the windowed parent subclass.
  • Controller creation mirrors create_controller: the ICoreWebView2Environment10 options path carries incognito + profile name + default background color; otherwise plain ICoreWebView2Environment3 creation (composition hosting does not exist below env3) with the background color applied post-creation. with_transparent(true) flows through unchanged as DefaultBackgroundColor alpha = 0.
  • All existing glue is shared: initialization scripts, the IPC bridge, custom protocols (including the {scheme}.localhost workaround), settings, theme, navigation/download/new-window/permission handlers — init_webview is common to both modes; the only branch is which subclass gets attached.

Host input forwarding

A composition-hosted webview receives no input from Windows — the host window owns the input queue. The new host subclass (subclass id WM_USER+0x67, distinct from the existing +0x64/+0x65/+0x66 so both families can coexist on one HWND) provides:

  • Mouse: every WM_MOUSE* message (move/leave/wheel/hwheel/all buttons including double-clicks and X buttons) is translated to SendMouseInput (COREWEBVIEW2_MOUSE_EVENT_KIND equals the WM_* code by definition). Wheel coordinates are screen-based and translated with ScreenToClient. Mouse capture is held across button drags; TrackMouseEvent(TME_LEAVE) generates the leave events hover cleanup needs.
  • Keyboard/IME: deliberately not forwarded message-by-message. On mouse button-down (and WM_SETFOCUS) the subclass calls MoveFocus(PROGRAMMATIC); WebView2's hidden input HWND then takes real Win32 focus and receives keys and IME natively.
  • Touch/pen: WM_POINTERDOWN/UPDATE/UP for PT_TOUCH/PT_PEN forwarded via SendPointerInput with an ICoreWebView2PointerInfo filled from GetPointerTouchInfo/GetPointerPenInfo (pixel locations and contact rect translated to client coordinates).
  • Cursor: CursorChanged events cache an HCURSOR, applied on WM_SETCURSOR for HTCLIENT hits (non-client hits fall through so frameless-resize arrows keep working). The handle prefers SystemCursorIdLoadCursorW (the OS shared cursor, crisp at any monitor DPI) over the Cursor bitmap property, which the runtime rasterises at base DPI and Windows then stretches blurry on scaled displays; custom CSS cursors (no system id) still use the bitmap handle.
  • Size/move/focus: WM_SIZESetBounds(client rect); WM_MOVE/WM_MOVINGNotifyParentWindowPositionChanged; WM_SETFOCUS/WM_ENTERSIZEMOVEMoveFocus (the same behavior the windowed parent subclass provides).
  • Pass-through: forwarded messages still reach the host's own window proc (DefSubclassProc), so tao's event stream and window machinery observe them unchanged. Both sides seeing input is inherent to this hosting model.

Dependency change: Win32_UI_Input_Pointer added to the windows crate features (for the WM_POINTER APIs). Nothing else.

What is NOT covered / known limitations

  • Touch/pen is best-effort. The himetric location/contact fields of ICoreWebView2PointerInfo are left zeroed (WebView2 hit-tests on the pixel fields). The mouse path is the one exercised in production.
  • OLE drag-drop (with_drag_drop_handler) is untested in composition mode. It would register on the host HWND; a correct composition-mode implementation would likely use ICoreWebView2CompositionController3::DragEnter instead. The producing app disables it.
  • JS window.close() destroys the host window in composition mode: the existing WindowCloseRequested handler destroys hwnd, which is the container in windowed mode but the host here. Arguably the right default (the webview is the window's content), but flagged for review.
  • Non-Windows code paths and the default (non-composition) Windows path are untouched; a webview built without a composition target takes exactly the existing code.

Verification

  • cargo check --target x86_64-pc-windows-msvc and cargo fmt --check pass; native-host cargo check unaffected.
  • Runtime verification comes from production use in the Colophon PDF app (this code, as a wry 0.55.1 fork with the same diff): a transparent composition-hosted webview over a wgpu-rendered document visual, exercised through a manual UAT gate covering mouse move/click/double-click/drag/capture, wheel and horizontal wheel scroll, hover states and mouse-leave cleanup, text selection, keyboard and IME composition input, cursor shapes on mixed-DPI monitors, window resize/move/minimize/restore, focus handoff, and multi-window creation via the per-HWND registry (through unmodified tauri-runtime-wry). Not run on Windows by CI in this PR; happy to add an example if maintainers want one (a minimal composition.rs example needs a DComp device + visual setup, ~100 lines).

Add an opt-in composition-hosted mode for WebView2: the webview is
created through CreateCoreWebView2CompositionController targeting a
caller-supplied IDCompositionVisual instead of a child HWND, so native
DirectComposition content can be layered beneath a (transparent)
webview in the same window.

API:

- `WebViewBuilderExtWindows::with_composition_visual_target(IUnknown)`
  supplies the target visual (an IDCompositionVisual of any version,
  passed as IUnknown so no new `windows` interface features leak into
  the public API).
- `register_composition_visual_target(hwnd, IUnknown)` is the
  out-of-band counterpart for embedders that construct the builder
  internally (e.g. tauri-runtime-wry): a per-HWND, consume-once,
  UI-thread-only registry checked when the builder carries no target
  of its own.

In composition mode no WRY_WEBVIEW container window is created; the
host window is subclassed (a distinct subclass id, coexisting with the
existing parent subclass ids) to forward mouse input via SendMouseInput
(with capture and TME_LEAVE tracking), touch/pen via SendPointerInput,
apply the webview cursor on WM_SETCURSOR (preferring SystemCursorId so
cursors stay crisp on scaled displays), keep controller bounds synced
on WM_SIZE, and hand focus to the webview on click so keyboard/IME
input arrives natively. set_bounds/set_visible become controller-only
in this mode and reparent returns Error::UnsupportedWindowHandle.

All other initialization (scripts, IPC, custom protocols, settings,
handlers) is shared with the windowed path. A default (non-composition)
webview takes exactly the existing code path.

Adds the Win32_UI_Input_Pointer feature to the `windows` dependency for
the WM_POINTER APIs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jerryoh512
jerryoh512 requested a review from a team as a code owner July 7, 2026 21:50
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Package Changes Through 753e111

There are 1 changes which include wry with minor

Planned Package Versions

The following package releases are the planned based on the context of changes in this pull request.

package current next
wry 0.55.1 0.56.0

Add another change file through the GitHub UI by following this link.


Read about change files or the docs at github.com/jbolda/covector

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants