Concept: Switch MP2 to D3D11/D2D rendering (1 Viewer)

[morpheus_xx]

I'd like to pick up a question related to accessing the backbuffer of the rendered scene: Due to changing the render device from DX9 to DX11, also the access have changed.

Before you could access:

Surface backBuffer = SkinContext.Device.GetRenderTarget(0);

With DX11+D2D the access is:

Bitmap1 backBuffer = SkinContext.Device11.RenderTarget2D;

This bitmap can be used i.e. by a BitmapBrush to "paint" a resized version on a new bitmap.

Important: once the switch to DX11 is completed, we will probably remove the public access to the DX9 device, as it is only required for VideoPlayer interop.

 

[Lightning303]

Regarding AtmoLight changes for DX11.

Please keep in mind that i have no idea what im doing . DirectX stuff is a black box for me.

Both my aproaches (BitmapBrush and renderTarget.DrawBitmap) i just tested seem to fail at creating a RenderTarget. So, how exactly do i do that?
The constructor suggests either using a pointer or a Factory, a Surface and some properties. I tried a null pointer, but that doesnt work. Tryed adding the Factories that SkinContext exposes, but they seem to be wrong aswell.
Please, somebody take my hand and show me the path .

 

[morpheus_xx]

Let's combine few parts for an example:

var bitmapWidth = 100;
var bitmapHeight = 100;
var targetRect = new RectangleF(0f, 0f, bitmapWidth, bitmapHeight);
Bitmap1 captureFrame = new Bitmap1(...); // target bitmap. Format must match backbuffer one, size you specify
using (new TemporaryRenderTarget2D(captureFrame)) // tells the 2D context to draw on captureFrame
{
  BitmapBrushProperties1 props = new BitmapBrushProperties1
  {
    ExtendModeX = ExtendMode.Clamp,
    ExtendModeY = ExtendMode.Clamp,
    InterpolationMode = GraphicsDevice11.Instance.ImageInterpolationMode
  };

  var backbuffer = SkinContext.Device11.RenderTarget2D;
  // Note: resource creation is expensive, store all device-resources in fields and make sure to dispose them when no longer needed!
  using (var bmpBrush = new BitmapBrush1(GraphicsDevice11.Instance.Context2D1, backbuffer , props))
  using (var rectGeom = new RectangleGeometry(GraphicsDevice11.Instance.Context2D1.Factory, targetRect))
  { 
    // Need to adjust the BitmapBrush to match input/output dimension and position. Should also be done only once
    Matrix3x2 brushTransform = Matrix3x2.Identity; 
    brushTransform *= Matrix3x2.Scaling(targetRect.Width / backbuffer.PixelSize.Width, targetRect.Height / backbuffer.PixelSize.Height);
    brushTransform *= Matrix3x2.Translation(targetRect.X, targetRect.Y);
    bmpBrush.Transform = brushTransform;
 
    // Fills the rectangle with the Bitmap Brush
    GraphicsDevice11.Instance.Context2D1.FillGeometry(rectGeom, bmpBrush);
  }
}

Source: https://github.com/MediaPortal/Medi...nEngine/Rendering/ImageContext2D.cs#L188-L203

Note: there are multiple ways to do this and I have not "benchmarked" them to know which is the most efficient one...

Edit: corrected "_bitmap" to "backbuffer"

 

[Lightning303]

Thanks for the example.

Sadly i still have some problems. It seems that i cant create the bitmapbrush1. I get these errors while creating the brush:

[2015-01-22 17:00:16,539] [12769  ] [DX Render] [ERROR] - AtmoLight: Exception: HRESULT: [0x88990021], Module: [SharpDX.Direct2D1], ApiCode: [D2DERR_BITMAP_CANNOT_DRAW/BitmapCannotDraw], Message: Unknown
[2015-01-22 17:00:16,550] [12780  ] [DX Render] [WARN ] - GraphicsDevice: DirectX Exception
SharpDX.SharpDXException: HRESULT: [0x88990024], Module: [SharpDX.Direct2D1], ApiCode: [D2DERR_INVALID_TARGET/InvalidTarget], Message: Unknown
   bei SharpDX.Result.CheckError()
   bei SharpDX.Direct2D1.RenderTarget.EndDraw(Int64& tag1, Int64& tag2)
   bei MediaPortal.UI.SkinEngine.DirectX.RenderPipelines.AbstractRenderPipeline.EndRender()
   bei MediaPortal.UI.SkinEngine.DirectX11.GraphicsDevice11.Render(Boolean doWaitForNextFame)

The internet says:

D2DERR_INVALID_TARGET
0x88990024
Cannot set the image as a target because it is either an effect or is a bitmap that does not have the D2D1_BITMAP_OPTIONS_TARGET flag set.

I first created the captureFrame like this:

Bitmap1 captureFrame = new Bitmap1(GraphicsDevice11.Instance.Context2D1, new Size2(bitmapWidth, bitmapHeight), new BitmapProperties1(new PixelFormat(Format.R8G8B8A8_UNorm, AlphaMode.Ignore)));

but even with a target option it does not work.

Bitmap1 captureFrame = new Bitmap1(GraphicsDevice11.Instance.Context2D1, new Size2(bitmapWidth, bitmapHeight), new BitmapProperties1(new PixelFormat(Format.R8G8B8A8_UNorm, AlphaMode.Ignore), 1, 1, BitmapOptions.Target));

Thanks again for your help!

 

[morpheus_xx]

Please look at: https://github.com/MediaPortal/Medi...ment/AssetCore/RenderTarget2DAssetCore.cs#L39.

Try this pixel format and this https://github.com/MediaPortal/Medi.../AssetCore/RenderTarget2DAssetCore.cs#L82-L86 to create the bitmap.

While collecting the info, it would be even easier for you to use the inbuild routines of ContentManager:

(see EvrCallback as example)

  _bitmapAsset2D = ContentManager.Instance.GetRenderTarget2D(_instanceKey);
  ((RenderTarget2DAsset)_bitmapAsset2D).AllocateRenderTarget(cx, cy);

_instanceKey is a unique string (like AtmoWinCapture) and cx, cy defines width/height. Good thing here is, that you don't need to care about releasing this asset manually, it's done by MP2.

 

[Lightning303]

Please look at: https://github.com/MediaPortal/Medi...ment/AssetCore/RenderTarget2DAssetCore.cs#L39.

Ok, will try. Was under the impression i had to use the same as the backbuffer, so i checked here: https://github.com/MediaPortal/Medi...SkinEngine/DirectX11/GraphicsDevice11.cs#L205
But gonna try with those settings.

While collecting the info, it would be even easier for you to use the inbuild routines of ContentManager:
(see EvrCallback as example)
Code (C#):
_bitmapAsset2D = ContentManager.Instance.GetRenderTarget2D(_instanceKey);
((RenderTarget2DAsset)_bitmapAsset2D).AllocateRenderTarget(cx, cy);
_instanceKey is a unique string (like AtmoWinCapture) and cx, cy defines width/height. Good thing here is, that you don't need to care about releasing this asset manually, it's done by MP2.

Gonna try to finish the other way first, and then have look at this

Thanks!


Edit:
Creating the brush with this target bitmap:

        Bitmap1 captureFrame = new Bitmap1(GraphicsDevice11.Instance.Context2D1, new Size2(bitmapWidth, bitmapHeight), new BitmapProperties1(new PixelFormat(Format.B8G8R8A8_UNorm, AlphaMode.Premultiplied), 96, 96, BitmapOptions.Target));

gives this error:

[2015-01-22 18:05:02,490] [9003   ] [DX Render] [ERROR] - AtmoLight: Exception: HRESULT: [0x88990021], Module: [SharpDX.Direct2D1], ApiCode: [D2DERR_BITMAP_CANNOT_DRAW/BitmapCannotDraw], Message: Unknown

No warnings from MP2 anymore though.


Edit 2:
Second approach from EVRCallback doest work aswell, or im just to dumb.

        RenderTarget2DAsset renderTarget = ContentManager.Instance.GetRenderTarget2D("AtmoLight");
        renderTarget.AllocateRenderTarget(bitmapWidth, bitmapHeight);
        renderTarget.Bitmap.CopyFromBitmap(SkinContext.Device11.RenderTarget2D);

throws

[2015-01-22 21:39:57,380] [10762  ] [DX Render] [ERROR] - AtmoLight: Exception: HRESULT: [0x80070057], Module: [General], ApiCode: [E_INVALIDARG/Invalid Arguments], Message: Falscher Parameter.

for copyfrombitmap.

 

[morpheus_xx]

After few weeks I continued on the DX11 switch. Finally I have the basic image rendering converted to use a custom D2D effect.

The main issue is, that I could not find a way to control sampler states for D2D render process. While there are methods for DX11 devices, all test didn't show an effect: rendering always used "Clamp" mode (repeating the outer pixel in non-texture area).

This looked very strange ofc. I worked around that issue by manual adding "border" logic to the shader: https://github.com/morpheusxx/Media...4946#diff-9bd808be0ee9a99868de94431a06863cR58.

If anyone is able to tell me how I can set a real SamplerState which works in combination with Direct2D, I'm very happy...

Next step is to switch (back) the Video rendering to the ImageContext to allow custom shader effects (like sharpen). Now I'm stuck in the effects registration. The problem here is, that effects can be only registered by a Type, but we use different combination of shader effects. I have moodified SharpDX to allow this (https://github.com/sharpdx/SharpDX/issues/535), but unfortunately I can't compile the SharpDX solution. Any help is very appreciated.

Is any of our @Developers able to help?

 

[morpheus_xx]

Today I got SharpDX compiling This means I now can create a custom SharpDX build that supports flexible effect registration. Things are progressing again...

 

[morpheus_xx]

VideoBrush now is using CustomEffects based approach, which restores the known scaling/post processing features from DX9 version.

The mentioned issue of SamplerState control remains: all Sampler seem to default to Clamp mode which looks very bad for non-full-size images. You see a 720x576 video on a 1920x1080 display in "original" size. Those strange "stripes" to all sides are the Clamp mode, where the wanted "Border" mode would simply use black.

 

[morpheus_xx]

The 4th testbuild for DX11/D2D support is available in the first post for download!

Please take a look at the changelog there, few known issues are remaining. But the list gets shorter This MP2-Client is compatible with Weekly builds from 02/2015, so you can try it in parallel...