[fixed] [MP2-833] 2.2.2 server crash during import (1 Viewer)

JSchuricht

Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    This was an interesting test. It could be my imagination but the import seems significantly faster. I pulled in 8268 movies without a crash. I also noticed there is barely any CPU usage during the import. Normally when I start an import the CPU is near 100% on all threads, also there are some large spikes in memory usage that didn't occur this time.

    I am wondering if we are hitting a memory limit in the OCVVideoThumbnailer plugin with the server process being 32 bit. I ran across the OutOfMemoryException in the logs an hour before the MP2-Server.exe process crashed. I stopped getting data at the time of the crash but when the SNMP poll found data in the from the OID stream MP2-Server.exe was eating up 2.36GB. Making a bit of a leap but if OCVVideoThumbnailer is making the difference in memory usage during the import, that one plugin could be hitting the 2GB limit.

    @morpheus_xx IIRC there are some things not supported in x64 server which is holding up the ability to publish a x64 version. Is it possible to have a x64 MP2 server and keep the tvserver or whatever is still 32 bit at the same time? Basically a hybrid x64 server with some 32bit parts.

    Crashed import with OCVVideoThumbnailer.
    ScreenShot546.jpg

    Import without OCVVideoThumbnailer.
    ScreenShot539.jpg

    ScreenShot547.jpg
     

    Lehmden

    Retired Team Member
  • Premium Supporter
  • December 17, 2010
    12,207
    3,726
    Lehmden
    Germany Germany
    Country flag
    Hi.
    It could be my imagination but the import seems significantly faster.
    I can't believe it's imagination. Generating ten thousands of thumbnails will eat a lot of time, CPU and memory for sure. Imho the import has to be faster a lot without thumbnail generation. If it wouldn't be faster there something really must go wrong...

    With OCVVideoThumbnailer disabled, that media item imported for the first time.
    This indicates a faulty video for sure. Most videos out there are not 100% clean and perfect. This especially is valid for TV recordings but it happens to nearly any type of video... A lot (most) of those "not that perfect" videos are still playing well but sometimes you can not recode them without failures. And creating thumbnails is a sort of recode. In general MediaInfo is the one that noticed those faulty videos at first. But then it would not import at all so in this case it should be ffmpeg who is not willing to work with this video...

    But here the culprit most likely is the fact that this specific "mkv" is no mkv at all. It's a m2ts BluRay stream with a wrong file extension... Try to rename the file to .m2ts or remux it into a real mkv container (with MKVToolnix or Media-Buddy for example, it's as fast as copying the file to a different space) and after that it most likely will be imported without issues...
     

    morpheus_xx

    Lead Dev MP2
  • Team MediaPortal
  • March 24, 2007
    11,600
    7,060
    Germany Germany
    Country flag
    We could use the former ffmpeg.exe based extractor again. It has the advantage that it spawns extra processes (also 64 bit). And if process leaks memory it has no influence on MP2 server process.

    For testing I can post a 64bit server exe. But a mix with 32 bit does not work, so no tv then.

    Gesendet von meinem SM-G930F mit Tapatalk
     

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    @Lehmden You are correct about the video source. I could convert this one file, what concerns me is I have hundreds of videos done the same way and only one has an issue during MP2 imports.
     

    Lehmden

    Retired Team Member
  • Premium Supporter
  • December 17, 2010
    12,207
    3,726
    Lehmden
    Germany Germany
    Country flag
    Hi.
    I have hundreds of videos done the same way
    It never is a good idea to rename files to a wrong file extension, never ever... You can mass-remux all your wrongly named files into mkv container with only one single mouse click in Media-Buddy and make "real" mkv out of them or you can batch rename all your wrongly named "mkv" to "m2ts" also with very little work in e.g. TotalCommander or other tools that can do batch renaming... Mediaportal 2 can handle m2ts files without issues so no need to name them wrong. Especially as MP2 tries to read Matroska tags out of those fake mkv files...
     

    henso

    Development Group
  • Team MediaPortal
  • February 16, 2012
    2,256
    776
    Denmark Denmark
    Country flag
    We could use the former ffmpeg.exe based extractor again.
    I was thinking the same thing, but it will be slower than with OpenCV if I remember correctly. I was also thinking if we should store the thumbnails in the fanart cache instead of in the database? It might lighten the load on the database during import.
     

    HTPCSourcer

    Retired Team Member
  • Premium Supporter
  • May 16, 2008
    11,431
    2,334
    Germany Germany
    Country flag
    I was thinking the same thing, but it will be slower than with OpenCV
    Is the thumbnail creation an asynchronous process? If not, would it make sense to convert it into one? From what morpheus and you write, I conclude that you rather not believe in a memory leak. Could you imagine to add code for testing that logs heap information to confirm or deny this hypothesis?
     

    Users who are viewing this thread

    Similar threads
    OP Title Forum Replies Date
    M MP2 - V2.3 MP2 & USB Problems Submit: Bug Reports 29
    L MP2 - V2.3 Troubles upgrading from MP2 2.2.3 to 2.3 General 0
    T MP2 - V2.3 MP2 x64 won't run Submit: Bug Reports 5
    J Copying Old VHS tapes Newcomers Forum 8
    W MP2 - V2.3 MP2 2.3 Import von Audioquellen / Datenbankprobleme Allgemeines Support- und Diskussionsforum 26
    G MP2 - V2.3 New install and crashing MP2-2.3 General 6
    S MP2 - V2.3 MP2 No video General 6
    W MP2 - V2.3 MP2 Server stürzt immer nach kurzer Zeit ab Allgemeines Support- und Diskussionsforum 77
    toricred MP2 - V2.3 MP2 2.3 Client won't start General 6
    Grisu002 MP2 - V2.3 MP2 2.3 Client startet nicht - bleibt beim Laden hängen Allgemeines Support- und Diskussionsforum 61
    M [solved] MP2 PROBLEME Standby Allgemeines Support- und Diskussionsforum 2
    Scheibes Was passiert bei MP2? Allgemeines Support- und Diskussionsforum 42
    S Programmierte Aufnahme in MP2 - kein Aufwachen des PC Allgemeines Support- und Diskussionsforum 5
    morpheus_xx [Finished] "What's New" plugin for MP2 upgrades MediaPortal 2 20
    da3dsoul [Request] An MVP Plugin example for custom settings in MP2 Plugin Development 2
    morpheus_xx [Evaluate] Checking MP2 code portability to .Net Core MediaPortal 2 15
    da3dsoul Does MP2 not have any active devs? General 8
    da3dsoul MP2 Plugin - Getting Started (Directory Tree and Player) Plugin Development 27
    R [solved] Kein Player registriert Allgemeines Support- und Diskussionsforum 1
    -Merkur- MP2 Client startet nicht, "libXBMC_addon.dll" fehlt Allgemeines Support- und Diskussionsforum 1
    mp2-ulrich [solved] MP 2.2.3 spielt nur manche Sender Allgemeines Support- und Diskussionsforum 2
    H MP2 database won't keep in synch with recordings General 8
    H [solved] MP2 Wiedergabe von Aufnahmen und Live TV geht nicht Allgemeines Support- und Diskussionsforum 6
    ge2301 [WiP] MP2Web - A platform independent web client for MP2 MediaPortal 2 72
    F Problems with new install of MP2 Newcomers Forum 3

    Similar threads

    Top Bottom