[Poll] Expose present surface object as in Framegrabber as event? (1 Viewer)

[gemx]

Please just forget my previous post.
I was just too tired yesterday

BTW: with StretchRect, according to the docs, the resize SHOULD be done by the GPU already. That's why it takes much less time.
I compared it with a 1080i video and here are durations for the process:

StretchRect (whole image): ~ 500 ms
StretchRect (64x48) : ~ 15 ms

So the stretch rect with a resizing took significantly less time, which i think comes from the fact that the GPU does the resize and then transfers the data which is always the same wether HD or SD.
BTW: the stretcrect time for SD and resizing is nearly the same as for HD.
This also lets me think that the GPU does the resizing here.

I think i have a goot solution to solve the problems the other noted.
Will post an updated patch later today

 

[tourettes]

StretchRect (64x48) : ~ 15 ms

That's awful lot of time when you compare that frames need to be processed on 20 ms time frame with 50hz picture

BTW: the stretcrect time for SD and resizing is nearly the same as for HD.

Maybe it is stretching the already stretch frame? (SD for HD resolution)

 

[arion_p]

BTW: the stretcrect time for SD and resizing is nearly the same as for HD.

Maybe it is stretching the already stretch frame? (SD for HD resolution)

The present surface is always the same resolution as the desktop (if TV is Full HD then 1920x1080), so SD or HD doesn't matter. It is the video texture that varies in size depending on SD/HD.

Still 15ms is too much, unless the timing is too coarse.

 

[rtv]

AW: Re: [Poll] Expose present surface object as in Framegrabber as event?

Still 15ms is too much, unless the timing is too coarse.

Yep, with 1080p60 (which might even get more common with all that 3D hype) we've only 16,6 ms for each pass...

 

[gemx]

BTW: the stretcrect time for SD and resizing is nearly the same as for HD.

Maybe it is stretching the already stretch frame? (SD for HD resolution)

The present surface is always the same resolution as the desktop (if TV is Full HD then 1920x1080), so SD or HD doesn't matter. It is the video texture that varies in size depending on SD/HD.

Still 15ms is too much, unless the timing is too coarse.

Nope, that's not true.
If i use the the patched core, then i only get the video in this surface and not the entire desktop (GUI of MP).
So the surface dimensions are always the same as the video BUT NOT the desktop.

Also, i did some extensive reading on StretchRect and it is 100% an operation that takes place ONLY on the GPU.
Also did some testing and measuring the stretchrect operation always lasted 0 ms (with only a resolution of ms) regardless if it's HD or SD.
What takes it's time are the "SurfaceLoader.SaveToStream... and the Bitmap bmp=new Bitmap(stream)" calls.
That is exactly the point were the buffer is copied from GPU to systemmemory and therefore the bottleneck.

 

[tourettes]

What takes it's time are the "SurfaceLoader.SaveToStream... and the Bitmap bmp=new Bitmap(stream)" calls.
That is exactly the point were the buffer is copied from GPU to systemmemory and therefore the bottleneck.

GPU RAM to CPU RAM is always a bottleneck (by design). It's a HW limitation as GPUs aren't generally required to transfer huge amounts of data back to the main RAM.

 

[gemx]

Ok, i think i found the best possible solution to solve this.

See the attached patch (first post also updated)

It basically does:

1) extends VideoSurfaceToRGBSurface in DXUtil.dll to also pass the height and width since resizing is done here via StretchRect
2) Adds an overload to Framebuffer.GetCurrentImage() to be able to set the size of the bitmap. The overload is for backwards compatibility. So other plugins or routines who use this function don't have to be changed

This enables any plugin to use the already available FrameBuffer class and to also specify the dimensions of the target bitmap.

Just to make some things clear:

1) The frame you get with the FrameGrabber IS NOT a screenshot but only contains the video being played in the resolution this video has, regardless if it's shown in fullscreen or in a small window it has always the dimensions of the source video. Therefore HD material takes more time to copy than SD because the resolution is much higher. To get the whole desktop you have to use GUIGraphicsContext.DX9Device.GetBackBuffer() which always gets the whole screen and is very slow.

2) StretchRect is done completely in the GPU mem, therefore it hardly doesn't need CPU time < 1 ms (hardly measurable)

3) The only thing that takes CPU time is copying from GPU to CPU via SurfaceLoader.SaveToStream()

#3 is therefore the only problem which can be minimized by using this patch.
I did some tests with the following results (720p video):

FrameGrabber.GetCurrentImage(): ~ 110 ms per grab
FrameGrabber.GetCurrentImage(300x200): ~ 20 ms per grab

That is a huge different.

Please also note that setting the captureWidth and height in the GetCurrentImage(...) function doesn't interfere with other requests at the same time with different values since the function uses a lock

So what do you think about this patch?

 

[Mr Hipp]

If you only want to read the texture, then it's enough to lock it.

Check GUIImage, LockMemoryImageTexture. But I think lockflags should be none.

 

[gemx]

Thanks this also saves several ms of CPU time

Here is the updated patch

BTW: I don't agree with I-Loop in the aspect as the analog clock enhancement is clearly a new feature whereas this patch helps to save CPU time under certain circumstances.
I must admit that this is also not really a bugfix - it just improves timing - so it's something in between a feature and a bugfix.

At least the LockRectangle call instead of SurfaceLoader.SaveToStream already saves several ms.

Also due to the clever locking mechanism inside Framegrabber between OnNewFrame and GetCurrentImage() this won't stall the rendering process not matter how long it takes.

 

[Mr Hipp]

I see that you use LockFlags discard in your patch. It there a reason for that?

I think you should use LockFlags.ReadOnly if you don't update the buffer (at least for clearity)

LockFlags .ReadOnly The application does not write to the buffer. This flag enables resources stored in non-native formats to save the recompression step when unlocking.

Also since you return a bitmap that has a pointer to the internal data of the texture, the texture might get updated (i don't know if the internal pointer can change, then you would really be in trouble, becuase then you have a pointer pointing to something that does not exists any longer).