How to create and initialize a Direct3D Device

[1170300710]

Sorry my english is not good.
My python version is 3.7.3, system is windows10.
I am try to record the screen.
I have already get the item by using "pick_single_item_async", and then I want to create a frame pool by "Direct3D11CaptureFramePool.create_free_threaded", but I don't know how to get the first argument whose type is IDirect3DDevice.

I find a way to get a D3D device by MediaCapture, and I can create a frame pool:

        media_capture = MediaCapture()

        async def coroutine():
            async_action = media_capture.initialize_async()
            if async_action:
                await async_action
        asyncio.run(coroutine())

        if not media_capture.media_capture_settings:
            raise OSError("Unable to initialize a Direct3D Device.")
        self.frame_pool = Direct3D11CaptureFramePool.create_free_threaded(
            media_capture.media_capture_settings.direct3_d11_device,
            DirectXPixelFormat.B8_G8_R8_A8_UINT_NORMALIZED,
            1,
            self.item.size
        )

but I got an error in line "await async_action":

OSError: [WinError -1072845856] No capture devices are available.

what should I do to create a frame pool.
Thanks a lot!

[dlech]

I am try to record the screen.

There are a few example of how to get a GraphicsCaptureItem.

• Unable to complete pick_single_item_async #6 (comment)
• Request to add binding for CreateForWindow or GetWindowIdFromWindow #5 (comment)

[1170300710]

Thanks a lot! That is helpful!
But maybe I didn't express clear.
I have already get a GraphicsCaptureItem. and I want to create a Direct3D11CaptureFramePool, and I don't know how to get the fiirst argument of function Direct3D11CaptureFramePool.create_free_threaded

[dlech]

MediaCapture is for capturing from a webcam, so I don't think it is relevant if you are wanting to capture the screen.

I found https://github.com/robmikh/screenshot-rs that shows how to use this in Rust, but is uses Win32 APIs that aren't available in Python.

If there is some sort of Python GUI library that uses Direct3D already, it may be possible to add some interop to connect it to this library. But, I couldn't find any other than a very old, unmaintained DirectPython library.

I also found https://github.com/SerpentAI/D3DShot which may already do what you want. It uses ctypes to call all of the Win32 APIs.

[Avasam]

MediaCapture can be used as a temporary workaround. But as soon as the user lacks any CaptureDevice, it will fail.

I stumbled on DirectPython, but yeah, way outdated.

D3DShot does seem to obtain an ID3D11Device (see exactly how they implement it here: https://github.com/SerpentAI/D3DShot/blob/4472120a21a1cb397d4f30d7a36af993e7544bc9/d3dshot/dll/d3d.py#L251 ). Since I'm already using it anyway for Desktop Duplication, I have tried to reuse the D3DDevice (example code below), or to reimplement it myself without success.

import ctypes

import win32gui
from d3dshot.dll.d3d import initialize_d3d_device
from winsdk.windows.graphics.capture import Direct3D11CaptureFramePool
from winsdk.windows.graphics.capture.interop import create_for_window
from winsdk.windows.graphics.directx import DirectXPixelFormat


def print_hwnd(hwnd: int, _):
    window_text = win32gui.GetWindowText(hwnd)
    if (
        window_text
        and window_text not in ("Default IME", "MSCTFIME UI")
        and "Window" not in window_text
    ):
        print(hwnd, window_text)


win32gui.EnumWindows(print_hwnd, '')


while True:
    try:
        hwnd = int(input("^ Enter an HWND from above ^ :"))
    except:
        print("not an int")
        continue

    try:
        print("Selected window:", win32gui.GetWindowText(hwnd))
    except:
        print("not a valid HWND")
        continue

    graphics_capture_item = create_for_window(hwnd)

    d3d_device_pointer = initialize_d3d_device(None)[0]
    d3d_device = ctypes.byref(d3d_device_pointer)
    test = Direct3D11CaptureFramePool.create_free_threaded(
        d3d_device, # or directly  d3dshot.create().displays[0].d3d_device
        DirectXPixelFormat.B8_G8_R8_A8_UINT_NORMALIZED,
        1,
        graphics_capture_item.size)

reuslts in

Traceback (most recent call last):
  File "c:\Users\Avasam\Desktop\example_3d3.py", line 41, in <module>
    test = Direct3D11CaptureFramePool.create_free_threaded(
TypeError: convert_to returned null

Microsoft documentation and examples use a CanvasDevice, which comes from Microsoft.Graphics.Canvas. I haven't seen any python bindings for this either.
https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/screen-capture#create-a-capture-frame-pool-and-capture-session

[Avasam]

For the record, D3DShot is dead and has been archived.

[lnfectedru]

Did anyone manage to find a solution?

[Avasam]

@lnfectedru I had someone recommend the device from winsdk.windows.ai.machinelearning Toufool/AutoSplit#175 . But according to microsoft docs, it still has a minimal version requirement of 1809 (build 10.0.17763).

So I went with an hybrid approach just in case a user was using an older build:
https://github.com/Toufool/Auto-Split/blob/2.0.0/src/utils.py#L83

import asyncio
from winsdk.windows.ai.machinelearning import LearningModelDevice, LearningModelDeviceKind
from winsdk.windows.media.capture import MediaCapture

def get_direct3d_device():
    try:
      direct_3d_device = LearningModelDevice(LearningModelDeviceKind.DIRECT_X_HIGH_PERFORMANCE).direct3_d11_device
    except: # TODO: Unknown potential error, I don't have an older Win10 machine to test.
      direct_3d_device = None 
    if not direct_3d_device:
        # Note: Must create in the same thread (can't use a global) otherwise when ran from not the main thread it will raise:
        # OSError: The application called an interface that was marshalled for a different thread
        media_capture = MediaCapture()

        async def coroutine():
            await (media_capture.initialize_async() or asyncio.sleep(0))
        asyncio.run(coroutine())
        direct_3d_device = media_capture.media_capture_settings and \
            media_capture.media_capture_settings.direct3_d11_device
    if not direct_3d_device:
        raise OSError("Unable to initialize a Direct3D Device.")
    return direct_3d_device
    
def try_get_direct3d_device():
    try:
      return get_direct3d_device()
    except OSError:
      return None

[lnfectedru]

@Avasam Looks like it works for me. Thank you for your response!

[sacrificerXY]

In my case, using MediaCapture works fine. But the d3d device from LearningModelDevice seems to behave differently.

When I use it for the capture pool, the frames I get from calling pool.try_get_next_frame() doesn't seem to get released. So if I set the pool buffer size to 3, I only ever get 3 captured frames total.

I tried calling frame.close(), using the frame as a context manager, and explicitly deleting the variable and setting it to None. Didn't work.

[tfluan0606]

Is there a way to release framepool buffer now?
I can't use MediaCapture as d3d device. I use LearningModelDevice but meet same issue.