[Approved] Framegrabber improvement - Capture from UI when Video is minimized or running in Mini-display (1 Viewer)

popy

MP Donator
  • Premium Supporter
  • July 3, 2011
    617
    141
    Hey guys.

    I have made some patches (to core & atmolight plugin) so also when a video is running minimized the UI frame data could be captured. So its a better user experience for atmolight users. What you see on tv will be expanded by atmolight in any situation ;)
    One patch is for mediaportal 1.11pre (master branch) & the other for atmolight plugin itself.
    On my HTPC the patch is working like it should -> when the video is minimized, now the gui frame data is used ;)

    Core Patch:
    I have extended the event handler "NewFrameHandler" of the framegrabber with an new FrameSource Parameter.
    The event handler "NewFrameHandler" gets called when:
    • no video is playing -> event will be raised with GUI data and FrameSource.GUI as flag
    • video is playing in fullscreen -> event will be raised with video data and FrameSource.Video as flag
    • video is playing minimized -> two events will be raised, one with video data and FrameSource.Video as flag and the second one with GUI data and FrameSource.GUI as flag
    • !! VideoModeSwitcher ALWAYS just gets the video frame data. It does'nt depends on video is minimized or fullscreen !!
    So a plugin (currently just AtmoLight) can decide which types of framedata it wants to use (video or GUI data).
    I have also added null checks (like they where before) so if there is no event subscriber the performance should'nt be affected.

    Atmolight Patch:
    The patch subscribes to the extended eventhandler "NewFrame" of the MP core and drops some framesource types on some situations depending on a new user setting "Use GUI data when video is minimized in MediaPortal Live Mode":
    • New Setting "true" & video is running minimized -> drop video frames and use GUI frames
    • New Setting "false" -> Drop GUI frames when video is playing -> behaviour like in 1.11pre (= legacy option)
    So the user can decide if he also want to use GUI framedata when video is playing minimized.

    Compatibility:
    Its nessacary to use the new patched core and patched Atmolight plugin.
    So when this patch finds its way into a new MP version ALL users must use new Atmolight version.
    I have gone this way because its a CLEANER way of implementation and from performance view.
    So there is NO legacy support for older MP version from Atmolights view and the other way round.

    Installation/Test of patched binarys:

    Prerequisites:

    • You must have 1.11pre Mediaportal with Atmolight 1.15.1.0 up and running
    • If you are using my previous patch, just reinstall AtmoLight plugin within extension manager and revert the ProcessPlugins.dll to the original from 1.11pre (also attached in zip "ProcessPlugins.dll_v1.11pre (Original).zip")
    Installation:

    !! Dont forget to backup all files from your current Installation !!

    Extract "Atmolight_DEV_gui_capture_v5_patched_bin.zip" and copy the files in the following directorys:
    • AtmoLight.MediaPortal1.dll & AtmoLight.Core.dll -> C:\Program Files (x86)\Team MediaPortal\MediaPortal\plugins\process
    • *.xml -> C:\ProgramData\Team MediaPortal\MediaPortal\language\Atmolight
    Extract "Mediaportal_1.11pre_gui_capture_v3_patched_bin.zip" and copy the files in the following directorys:
    • Core.dll & Mediaportal.exe -> C:\Program Files (x86)\Team MediaPortal\MediaPortal

    Atmolight Patch:
    The patch subscribes JUST to the exctended eventhandler "NewFrame" of the MP core and drops some framesource types on some situations depending on a new user setting "Use GUI and video framedata in MPLiveMode (MP 1.12 and up)":
    • New Setting "true" & video is running minimized -> drop video frames
    • New Setting "false" -> Drop GUI frames
    So the user can decide if he want to use GUI frame data at all.
    Atmolight Patch:
    The patch subscribes JUST to the exctended eventhandler "NewFrame" of the MP core and drops some framesource types on some situations:
    • drop gui frames when video is minimized and users has set the menu effect "MPLiveView"
    • Drop GUI frames when menu effect is NOT "MPLiveView", so just video frames will pass when video effect is also set to "MediaPortalLiveMode"
    v3 Atmolight download: http://www60.zippyshare.com/v/6eWHQJr3/file.html
    Core Patch:
    I have added a new event handler named "NewFrameHandlerWithGUI" to the framegrabber.

    The new event handler "NewFrameHandlerWithGUI" gets called when:
    • no video is playing -> GUI data will be used
    • fullscreen video is playing -> video data will be used
    • video is playing minimized -> GUI data will be used
    • !! VideoModeSwitcher ALWAYS get the video frame data! It doesnt depends on video is minimized or fullscreen.
    "old" event handler "NewFrameHandler" gets called:
    • video is playing fullscreen or minimized -> video data will be used (like it was before)
    So a plugin (AtmoLight) can decide if it/User wants to use JUST video frame data (NewFrameHandler event) or it/User wants to use gui & video frame data (NewFrameHandlerWithGUI event). I have added a user setting to the AtmoLight, so the user can decide (see attached patched plugin).
    I have also added null checks (like they where before) so if there is no event subscriber the performance should'nt be affected.

    Atmolight Patch:
    As stated above, i have added a user setting "Use GUI framedata (MP 1.12 and up)" where the user can decide if he wants to JUST use video frame data or also GUI data when video is minimized or not playing. My implementation also checks for compatibility. When the "NewFrameHandlerWithGUI" event exists in core.dll and the user has set the setting "Use GUI framedata (MP 1.12 and up)" the new feature is used ;)

    Compatibility:
    I have tested the following combinations:
    • patched MP 1.11pre -> patched Atmolight plugin -> works :p
    • patched MP 1.11pre -> original Atmolight plugin -> works
    • original MP 1.11pre -> patched Atmolight plugin -> works
    With this implementation there should'nt be compatibility issues.

    Installation/Test of patched binarys:

    Prerequisites:

    • You must have 1.11pre Mediaportal with Atmolight 1.15.1.0 up and running
    • If you are using v1 of my patch, just reinstall AtmoLight plugin within extension manager and revert the ProcessPlugins.dll to the original from 1.11pre (also attached in zip "ProcessPlugins.dll_v1.11pre (Original).zip")
    Installation:

    !! Dont forget to backup all files from your current Installation !!

    Extract "Atmolight_1.15_gui_capture_v2_patched_bin.zip" and copy the files in the following directorys:
    • AtmoLight.MediaPortal1.dll -> C:\Program Files (x86)\Team MediaPortal\MediaPortal\plugins\process
    • *.lng -> C:\ProgramData\Team MediaPortal\MediaPortal\language\Atmolight
    Extract "Mediaportal_1.11pre_gui_capture_v2_patched_bin.zip" and copy the files in the following directorys:
    • Core.dll & Mediaportal.exe -> C:\Program Files (x86)\Team MediaPortal\MediaPortal
    v2 files download: http://www1.zippyshare.com/v/HRIFuPDr/file.html
    VideoModeSwitcher changes and effect for users which doesnt have atmolight plugin installed:[/B]
    My patch also include changes in VideoModeSwitcher so it doesnt use frame data when video is playing and is minimized -> because without the patch it has switched aspect ratio depending on the GUI frame data.
    Summary: Users which dont have atmolight installed, have just one throwback -> when video is minimized the VideoModeSwitcher dont get current video data and dont switch the aspect ratio until the video is maximized again.

    Atmolight Patch:
    Why Atmolight patch is needed? -> it isnt really needed BUT i have added the IsFullScreenVideo bool flag to the NewFrameHandler, so when atmolight plugin or any other (new) plugin grabs frame data and want to know if frame data is a fullscreenvideo or is gui frame data. So the plugin can decide itself if it uses the data or drop it.

    Changelog:
    • if video is playing not in fullscreen the current GUI frame data is used for the NewFrameHandler event
    • added is "IsFullScreenVideo" bool to Event NewFrameHandler (thats the reason why atmolight plugin needs to be patched)
    • VideoModeSwitcher is not using the GUI frame data
    attached in the zip you can find the patched core.dll, mediaportal.exe, ProcessPlugins.dll & Atmolight plugin which uses the new OnNewFrame handler event. The two unified patches (mediaportal & atmolight plugin) are also attached.

    Attachments:
    https://forum.team-mediaportal.com/attachments/atmolight-mediaportal1-plugin-patch.161719/
    https://forum.team-mediaportal.com/attachments/mediaportal_atmolight_gui-patch.161721/
    https://forum.team-mediaportal.com/attachments/atmolight_patch_1-11pre-zip.161720/

    @Lightning303 & All: Can you please review the patches and let me know your thinkings.
    Would be nice if the patches are finds its way into the next releases :p

    Have fun
    pOpY
     

    Attachments

    • Mediaportal_1.11pre_gui_capture_v3.patch
      7 KB
    • Mediaportal_1.11pre_gui_capture_v3_patched_bin.zip
      1.6 MB
    • ProcessPlugins.dll_v1.11pre (Original).zip
      102.6 KB
    • Atmolight_DEV_gui_capture_v5.patch
      15.3 KB
    • Atmolight_DEV_gui_capture_v5_patched_bin.zip
      124.3 KB
    Last edited:

    popy

    MP Donator
  • Premium Supporter
  • July 3, 2011
    617
    141
    if video is playing not in fullscreen the current GUI frame data is used for the NewFrameHandler event
    i think this should be an option inside Atmolight .

    ill think this is not easy to implement because the grabbing code is in the core and atmolight is a plugin. So there is no interface of setting core settings out of an plugin.

    pOpY
     

    popy

    MP Donator
  • Premium Supporter
  • July 3, 2011
    617
    141
    Hey guys.

    I have reworked the core patch, now it does not have any effect on videomodeswitcher and also its compatible with older versions of atmologht plugin.
    Also my reworked atmolight plugin patch is compatible with older versions of MP.
    Another addition is a new option in atmolight plugin where the user could select if he wants to use ""GUI frame data".

    Upload comes tomorrow, its late here :p

    pOpY
     

    Lightning303

    MP Donator
  • Premium Supporter
  • September 12, 2009
    798
    577
    Home Country
    Germany Germany
    In regards to AtmoLight, i dont think this is the right solution. I just flew over the patches, so maybe im missing something or misunderstanding. But from what i can see, you mainly renamed the event.
    In AtmoLight you added a new setting, to choose if one wants to use the new event. If this setting is enabled AtmoLight subscribes to the new event, if not, it tries to subscribe to the old one. But this only adds support for pre change and post change versions. Nothing else is added by this.
    If i enable the settings gui capture works, and if i watch a video and then minimize it, AtmoLight reveives the GUI data.
    If i disable this settings, MediaPortal Live Mode will only work during video playback, Live Mode in GUI (no media playback) is not working at all anymore.

    So instead of toggling which source should be used when we minimize a video (which was the idea of all of this), we now can toggle between "Show exactly what is on screen" and "Dont ever show GUI frames (AtmoLight behaviour from before 1.11pre).

    So why would i want to ever disable this with MP1.12 or ever enable this with older versions? It just adds confusion.

    Maybe the best approch is to let MediaPortal raise the OnNewFrame event on every new frame. So if we minimize a video, let an event get raised for every GUI frame and an event for every video frame. And then attach an identifier of where this frame came from (gui or video renderer). Then let ViewModeSwicther and AtmoLight decide which frame to use and which to discard. Yes, this means if you want to use a new MP version you would need to update AtmoLight to a new version aswell, but its better to do this right in my opinion.
    Another option would be similar to your approach, have 2 events, one that is called OnNewFrame and wont be changed to keep lagacy support and add a new one which has the bahaviour i described. We then can add a check in AtmoLight what MP version is running and subscribe to the right event. If its the new version we check which frame we want to use and send that one to the existing method (AtmolightPlugin_OnNewFrame) for processing.

    Hope i make sense :). Keep up the good work.
     

    popy

    MP Donator
  • Premium Supporter
  • July 3, 2011
    617
    141
    thx for your feedback.

    In regards to AtmoLight, i dont think this is the right solution. I just flew over the patches, so maybe im missing something or misunderstanding. But from what i can see, you mainly renamed the event.

    yes, you missed something :p
    now there are 2 events. the old "NewFrameHandler" completly without GUI frame data (just for legacy).
    And then tehre is the new one "NewFrameHandlerWithGUI" with GUI framedata.
    I have updated the OP where i try to describe the two events ;)

    A big problem is that the OnFrameGUI functions in Mediaportal.cs gets called when no video is running. But they DONT gets called when a video is running fullscreen or not, that doesnt matter. So thats the reason why i have added the code:

    Code:
          //Do we have any subcribers for event NewFrameHandlerWithGUI
          if (OnNewFrameWithGUI != null)
          {
              try
              {
                  if (GUIGraphicsContext.IsFullScreenVideo)
                  {
                      //we are in fullscreen mode -> use video surface
                      //raise event to any subcribers for event NewFrameHandlerWithGUI
                      OnNewFrameWithGUI(width, height, arWidth, arHeight, pSurface);
                  }
                  else
                  {
                      //If we are not in fullscreen, call OnFrameGUI without parameter to use the actual GUI backbuffer
                      OnFrameGUI();
                  }
              }
              catch (Exception)
              {
              }
          }

    in the original function " public void OnFrame(Int16 width, Int16 height, Int16 arWidth, Int16 arHeight, uint pSurface)" which gets called everytime when a video frame will be renderd (called from PlaneScene.cs).

    So when a video is running minimized and there is a subscriber for the new event "OnNewFrameWithGUI" the GUI data is used (call OnFrameGUI();). when the video is running in fullscreen the current video framedata is used.

    Ill hope it makes sense :)
    Do you have any better idea how to get GUI data when video is minimized?
    Ill think it has something todo that the GUI is rendered in another thread when a video is running.

    If i enable the settings gui capture works, and if i watch a video and then minimize it, AtmoLight reveives the GUI data.
    If i disable this settings, MediaPortal Live Mode will only work during video playback, Live Mode in GUI (no media playback) is not working at all anymore.

    Thats the expected behaviour.

    So instead of toggling which source should be used when we minimize a video (which was the idea of all of this), we now can toggle between "Show exactly what is on screen" and "Dont ever show GUI frames (AtmoLight behaviour from before 1.11pre).
    So why would i want to ever disable this with MP1.12 or ever enable this with older versions? It just adds confusion.

    Ok, i am with you.
    Removing the option is the better way and dont confuse the user.

    Maybe the best approch is to let MediaPortal raise the OnNewFrame event on every new frame. So if we minimize a video, let an event get raised for every GUI frame and an event for every video frame. And then attach an identifier of where this frame came from (gui or video renderer). Then let ViewModeSwicther and AtmoLight decide which frame to use and which to discard. Yes, this means if you want to use a new MP version you would need to update AtmoLight to a new version aswell, but its better to do this right in my opinion.
    Another option would be similar to your approach, have 2 events, one that is called OnNewFrame and wont be changed to keep lagacy support and add a new one which has the bahaviour i described. We then can add a check in AtmoLight what MP version is running and subscribe to the right event. If its the new version we check which frame we want to use and send that one to the existing method (AtmolightPlugin_OnNewFrame) for processing.

    Ill think the better option, from a users view, is to keep legacy support (keep OnNewFrame support) and add a new event handler which is called with an source parameter so th plugin can decide.

    The VIdeoModeSwitcher DONT uses the OnNewFrame event, it uses the "GetCurrentImage" function from framegrabber.cs.

    Will work now on the new discussed implementation.

    pOpY
     

    Lightning303

    MP Donator
  • Premium Supporter
  • September 12, 2009
    798
    577
    Home Country
    Germany Germany
    now there are 2 events. the old "NewFrameHandler" completly without GUI frame data (just for legacy).
    And then tehre is the new one "NewFrameHandlerWithGUI" with GUI framedata.
    Yeah i got that part ;).

    A big problem is that the OnFrameGUI functions in Mediaportal.cs gets called when no video is running. But they DONT gets called when a video is running fullscreen or not, that doesnt matter. So thats the reason why i have added the code:
    I see, i knew it wasnt calling the gui frames during video playback (minimized) before, because otherwise we would already end up with 2 frames. I assumed you had changed that, instead of checking if fullscreen in enabled and then choose what to grab. My bad :(.

    Thats the expected behaviour.
    Sill, its breaking allready working stuff. Using MP + AtmoLight last versions (1.15.1 and 1.11pre), MP Live Mode works in GUI (no playback), with your changes it doesnt anymore using the lagacy event (e.g. when the new setting is disabled or im not using a newer AtmoLight version).

    Do you have any better idea how to get GUI data when video is minimized?
    The VIdeoModeSwitcher DONT uses the OnNewFrame event, it uses the "GetCurrentImage" function from framegrabber.cs.
    I think its still possible as i described in my last post, a bit different, but still. Question is mainly if there should a second event or one (which breaks lagacy). If only AtmoLight is effected by this change, then i would say stay with 1 event. But its not my place to decide that ;). You are doing the work and MP devs need to decide what they think is cleaner/better. If 2 events are being used, then i think the event that is in place now should not be changed at all. So it works like it does in 1.11pre.
    For sending both frames, the OnFrame method, which you renamed into OnFrameGUI could be changed so that it would not choose which single frame to send with an if, but rather check if it has to send a second one.
    Code:
    if (!GUIGraphicsContext.IsFullScreenVideo)
    {
      OnFrame();
    }
    OnNewFrame(width, height, arWidth, arHeight, pSurface);

    So, if we are not in fullscreen send both, gui and video, if we are in fullscreen, only send video.

    Of course both would have to be extended with an identifier. A nice enum e.g., so AtmoLight or any other plugin could choose which to use.
     

    popy

    MP Donator
  • Premium Supporter
  • July 3, 2011
    617
    141
    I have thinked about it and from a performance point of view its cleaner and faster to use one event and extend it.
    Done so in my patch v3, its uploaded and OP was updated.

    @Lightning303: Can you please take a look at it and test it?

    thx
    pOpY
     
    Last edited:

    Users who are viewing this thread

    Top Bottom