feat(windows): add DirectComposition (visual) hosting for WebView2 - #1762
Open
jerryoh512 wants to merge 1 commit into
Open
feat(windows): add DirectComposition (visual) hosting for WebView2#1762jerryoh512 wants to merge 1 commit into
jerryoh512 wants to merge 1 commit into
Conversation
Add an opt-in composition-hosted mode for WebView2: the webview is created through CreateCoreWebView2CompositionController targeting a caller-supplied IDCompositionVisual instead of a child HWND, so native DirectComposition content can be layered beneath a (transparent) webview in the same window. API: - `WebViewBuilderExtWindows::with_composition_visual_target(IUnknown)` supplies the target visual (an IDCompositionVisual of any version, passed as IUnknown so no new `windows` interface features leak into the public API). - `register_composition_visual_target(hwnd, IUnknown)` is the out-of-band counterpart for embedders that construct the builder internally (e.g. tauri-runtime-wry): a per-HWND, consume-once, UI-thread-only registry checked when the builder carries no target of its own. In composition mode no WRY_WEBVIEW container window is created; the host window is subclassed (a distinct subclass id, coexisting with the existing parent subclass ids) to forward mouse input via SendMouseInput (with capture and TME_LEAVE tracking), touch/pen via SendPointerInput, apply the webview cursor on WM_SETCURSOR (preferring SystemCursorId so cursors stay crisp on scaled displays), keep controller bounds synced on WM_SIZE, and hand focus to the webview on click so keyboard/IME input arrives natively. set_bounds/set_visible become controller-only in this mode and reparent returns Error::UnsupportedWindowHandle. All other initialization (scripts, IPC, custom protocols, settings, handlers) is shared with the windowed path. A default (non-composition) webview takes exactly the existing code path. Adds the Win32_UI_Input_Pointer feature to the `windows` dependency for the WM_POINTER APIs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Package Changes Through 753e111There are 1 changes which include wry with minor Planned Package VersionsThe following package releases are the planned based on the context of changes in this pull request.
Add another change file through the GitHub UI by following this link. Read about change files or the docs at github.com/jbolda/covector |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
WebView2 supports two hosting models: the windowed controller wry uses today, and a composition controller (
CreateCoreWebView2CompositionController) that renders into a caller-supplied DirectComposition visual instead of an HWND. Composition hosting is what makes layered DirectComposition apps possible: native content (e.g. a wgpu/D3D swapchain on a sibling visual) drawn beneath a transparent webview in the same window, with the webview used as the UI chrome layer. The windowed model cannot express this — a child HWND always occludes everything under it.This PR adds composition hosting as an opt-in mode. It is used in production by a Windows PDF application (Colophon) that renders documents on a native wgpu surface beneath a transparent Tauri/wry webview UI.
Related: #391 (offscreen rendering) asks for a different but adjacent capability; the composition controller introduced here is also the natural substrate for future texture-stream/offscreen work.
API
Two entry points, both
#[cfg(windows)]:WebViewBuilderExtWindows::with_composition_visual_target(visual: IUnknown)— supplies anIDCompositionVisual(any version, passed asIUnknownso no newwindows-crate interface features leak into wry's public API). When set, the webview is created throughCreateCoreWebView2CompositionControllerwith that visual as the root visual target, instead of creating aWRY_WEBVIEWchild window.register_composition_visual_target(hwnd: isize, visual: IUnknown)(crate-root re-export) — the out-of-band counterpart for embedding layers that construct theWebViewBuilderinternally and never expose it to the application, most importantlytauri-runtime-wry. The host registers the visual for its window before asking the embedder to create the webview; webview creation on that HWND consumes the entry (once) when the builder carries no target of its own. The registry is per-HWND (concurrent window creations cannot cross-wire), consume-once, and UI-thread-only by construction (thread-local; registration and creation both happen on the UI thread, which is also the COM STA the visual lives in). A builder-supplied target always takes precedence. This is the piece that lets a Tauri app opt a window into composition hosting today with no tauri-runtime-wry changes; if upstreaming a first-class knob to tauri-runtime-wry later, the builder method is the primitive it would call.Behavior in composition mode
InnerWebView.hwndis the host window itself andis_compositionis set. Guarded on that flag:set_boundsonly updates controller bounds (neverSetWindowPos— that would move the host window),set_visibleonly togglescontroller.SetIsVisible(neverShowWindowon the host),reparentreturnsError::UnsupportedWindowHandle(there is no child window to move; the host subclass and visual tree are bound to the original window), andDropdetaches the composition subclass instead of the windowed parent subclass.create_controller: theICoreWebView2Environment10options path carries incognito + profile name + default background color; otherwise plainICoreWebView2Environment3creation (composition hosting does not exist below env3) with the background color applied post-creation.with_transparent(true)flows through unchanged asDefaultBackgroundColoralpha = 0.{scheme}.localhostworkaround), settings, theme, navigation/download/new-window/permission handlers —init_webviewis common to both modes; the only branch is which subclass gets attached.Host input forwarding
A composition-hosted webview receives no input from Windows — the host window owns the input queue. The new host subclass (subclass id
WM_USER+0x67, distinct from the existing+0x64/+0x65/+0x66so both families can coexist on one HWND) provides:WM_MOUSE*message (move/leave/wheel/hwheel/all buttons including double-clicks and X buttons) is translated toSendMouseInput(COREWEBVIEW2_MOUSE_EVENT_KINDequals theWM_*code by definition). Wheel coordinates are screen-based and translated withScreenToClient. Mouse capture is held across button drags;TrackMouseEvent(TME_LEAVE)generates the leave events hover cleanup needs.WM_SETFOCUS) the subclass callsMoveFocus(PROGRAMMATIC); WebView2's hidden input HWND then takes real Win32 focus and receives keys and IME natively.WM_POINTERDOWN/UPDATE/UPforPT_TOUCH/PT_PENforwarded viaSendPointerInputwith anICoreWebView2PointerInfofilled fromGetPointerTouchInfo/GetPointerPenInfo(pixel locations and contact rect translated to client coordinates).CursorChangedevents cache anHCURSOR, applied onWM_SETCURSORforHTCLIENThits (non-client hits fall through so frameless-resize arrows keep working). The handle prefersSystemCursorId→LoadCursorW(the OS shared cursor, crisp at any monitor DPI) over theCursorbitmap property, which the runtime rasterises at base DPI and Windows then stretches blurry on scaled displays; custom CSS cursors (no system id) still use the bitmap handle.WM_SIZE→SetBounds(client rect);WM_MOVE/WM_MOVING→NotifyParentWindowPositionChanged;WM_SETFOCUS/WM_ENTERSIZEMOVE→MoveFocus(the same behavior the windowed parent subclass provides).DefSubclassProc), so tao's event stream and window machinery observe them unchanged. Both sides seeing input is inherent to this hosting model.Dependency change:
Win32_UI_Input_Pointeradded to thewindowscrate features (for theWM_POINTERAPIs). Nothing else.What is NOT covered / known limitations
ICoreWebView2PointerInfoare left zeroed (WebView2 hit-tests on the pixel fields). The mouse path is the one exercised in production.with_drag_drop_handler) is untested in composition mode. It would register on the host HWND; a correct composition-mode implementation would likely useICoreWebView2CompositionController3::DragEnterinstead. The producing app disables it.window.close()destroys the host window in composition mode: the existingWindowCloseRequestedhandler destroyshwnd, which is the container in windowed mode but the host here. Arguably the right default (the webview is the window's content), but flagged for review.Verification
cargo check --target x86_64-pc-windows-msvcandcargo fmt --checkpass; native-hostcargo checkunaffected.composition.rsexample needs a DComp device + visual setup, ~100 lines).