[Poll] Expose present surface object as in Framegrabber as event? (1 Viewer)

Add the patch to SVN?


  • Total voters
    7

gemx

Retired Team Member
  • Premium Supporter
  • October 31, 2006
    1,972
    539
    Home Country
    Germany Germany
    Please just forget my previous post.
    I was just too tired yesterday :)

    BTW: with StretchRect, according to the docs, the resize SHOULD be done by the GPU already. That's why it takes much less time.
    I compared it with a 1080i video and here are durations for the process:

    StretchRect (whole image): ~ 500 ms
    StretchRect (64x48) : ~ 15 ms

    So the stretch rect with a resizing took significantly less time, which i think comes from the fact that the GPU does the resize and then transfers the data which is always the same wether HD or SD.
    BTW: the stretcrect time for SD and resizing is nearly the same as for HD.
    This also lets me think that the GPU does the resizing here.

    I think i have a goot solution to solve the problems the other noted.
    Will post an updated patch later today
     

    tourettes

    Retired Team Member
  • Premium Supporter
  • January 7, 2005
    17,301
    4,800
    StretchRect (64x48) : ~ 15 ms

    That's awful lot of time when you compare that frames need to be processed on 20 ms time frame with 50hz picture :D

    BTW: the stretcrect time for SD and resizing is nearly the same as for HD.

    Maybe it is stretching the already stretch frame? (SD for HD resolution)
     

    arion_p

    Retired Team Member
  • Premium Supporter
  • February 7, 2007
    3,373
    1,626
    Athens
    Home Country
    Greece Greece
    BTW: the stretcrect time for SD and resizing is nearly the same as for HD.
    Maybe it is stretching the already stretch frame? (SD for HD resolution)

    The present surface is always the same resolution as the desktop (if TV is Full HD then 1920x1080), so SD or HD doesn't matter. It is the video texture that varies in size depending on SD/HD.

    Still 15ms is too much, unless the timing is too coarse.
     

    rtv

    Retired Team Member
  • Premium Supporter
  • April 7, 2005
    3,622
    301
    Osnabruck
    Home Country
    Germany Germany
    AW: Re: [Poll] Expose present surface object as in Framegrabber as event?

    Still 15ms is too much, unless the timing is too coarse.

    Yep, with 1080p60 (which might even get more common with all that 3D hype) we've only 16,6 ms for each pass...
     

    gemx

    Retired Team Member
  • Premium Supporter
  • October 31, 2006
    1,972
    539
    Home Country
    Germany Germany
    BTW: the stretcrect time for SD and resizing is nearly the same as for HD.
    Maybe it is stretching the already stretch frame? (SD for HD resolution)

    The present surface is always the same resolution as the desktop (if TV is Full HD then 1920x1080), so SD or HD doesn't matter. It is the video texture that varies in size depending on SD/HD.

    Still 15ms is too much, unless the timing is too coarse.

    Nope, that's not true.
    If i use the the patched core, then i only get the video in this surface and not the entire desktop (GUI of MP).
    So the surface dimensions are always the same as the video BUT NOT the desktop.

    Also, i did some extensive reading on StretchRect and it is 100% an operation that takes place ONLY on the GPU.
    Also did some testing and measuring the stretchrect operation always lasted 0 ms (with only a resolution of ms) regardless if it's HD or SD.
    What takes it's time are the "SurfaceLoader.SaveToStream... and the Bitmap bmp=new Bitmap(stream)" calls.
    That is exactly the point were the buffer is copied from GPU to systemmemory and therefore the bottleneck.
     

    tourettes

    Retired Team Member
  • Premium Supporter
  • January 7, 2005
    17,301
    4,800
    What takes it's time are the "SurfaceLoader.SaveToStream... and the Bitmap bmp=new Bitmap(stream)" calls.
    That is exactly the point were the buffer is copied from GPU to systemmemory and therefore the bottleneck.

    GPU RAM to CPU RAM is always a bottleneck (by design). It's a HW limitation as GPUs aren't generally required to transfer huge amounts of data back to the main RAM.
     

    gemx

    Retired Team Member
  • Premium Supporter
  • October 31, 2006
    1,972
    539
    Home Country
    Germany Germany
    Ok, i think i found the best possible solution to solve this.

    See the attached patch (first post also updated)

    It basically does:

    1) extends VideoSurfaceToRGBSurface in DXUtil.dll to also pass the height and width since resizing is done here via StretchRect
    2) Adds an overload to Framebuffer.GetCurrentImage() to be able to set the size of the bitmap. The overload is for backwards compatibility. So other plugins or routines who use this function don't have to be changed

    This enables any plugin to use the already available FrameBuffer class and to also specify the dimensions of the target bitmap.

    Just to make some things clear:

    1) The frame you get with the FrameGrabber IS NOT a screenshot but only contains the video being played in the resolution this video has, regardless if it's shown in fullscreen or in a small window it has always the dimensions of the source video. Therefore HD material takes more time to copy than SD because the resolution is much higher. To get the whole desktop you have to use GUIGraphicsContext.DX9Device.GetBackBuffer() which always gets the whole screen and is very slow.

    2) StretchRect is done completely in the GPU mem, therefore it hardly doesn't need CPU time < 1 ms (hardly measurable)

    3) The only thing that takes CPU time is copying from GPU to CPU via SurfaceLoader.SaveToStream()

    #3 is therefore the only problem which can be minimized by using this patch.
    I did some tests with the following results (720p video):

    FrameGrabber.GetCurrentImage(): ~ 110 ms per grab
    FrameGrabber.GetCurrentImage(300x200): ~ 20 ms per grab

    That is a huge different.

    Please also note that setting the captureWidth and height in the GetCurrentImage(...) function doesn't interfere with other requests at the same time with different values since the function uses a lock

    So what do you think about this patch?
     

    Attachments

    • FrameGrabber with resize option.diff
      5.4 KB

    Mr Hipp

    Retired Team Member
  • Premium Supporter
  • April 2, 2006
    1,261
    188
    56
    Malmö
    Home Country
    Sweden Sweden
    If you only want to read the texture, then it's enough to lock it.

    Check GUIImage, LockMemoryImageTexture. But I think lockflags should be none.
     

    gemx

    Retired Team Member
  • Premium Supporter
  • October 31, 2006
    1,972
    539
    Home Country
    Germany Germany
    Thanks this also saves several ms of CPU time :)

    Here is the updated patch

    BTW: I don't agree with I-Loop in the aspect as the analog clock enhancement is clearly a new feature whereas this patch helps to save CPU time under certain circumstances.
    I must admit that this is also not really a bugfix - it just improves timing - so it's something in between a feature and a bugfix.

    At least the LockRectangle call instead of SurfaceLoader.SaveToStream already saves several ms.

    Also due to the clever locking mechanism inside Framegrabber between OnNewFrame and GetCurrentImage() this won't stall the rendering process not matter how long it takes.
     

    Attachments

    • FrameGrabber with resize option take2.diff
      6 KB

    Mr Hipp

    Retired Team Member
  • Premium Supporter
  • April 2, 2006
    1,261
    188
    56
    Malmö
    Home Country
    Sweden Sweden
    I see that you use LockFlags discard in your patch. It there a reason for that?

    I think you should use LockFlags.ReadOnly if you don't update the buffer (at least for clearity)

    LockFlags .ReadOnly The application does not write to the buffer. This flag enables resources stored in non-native formats to save the recompression step when unlocking.

    Also since you return a bitmap that has a pointer to the internal data of the texture, the texture might get updated (i don't know if the internal pointer can change, then you would really be in trouble, becuase then you have a pointer pointing to something that does not exists any longer).
     

    Users who are viewing this thread

    Top Bottom