[fixed] [MP2-833] 2.2.2 server crash during import (1 Viewer)

henso

Development Group
  • Team MediaPortal
  • February 16, 2012
    2,256
    776
    Denmark Denmark
    Country flag
    Is the thumbnail creation an asynchronous process?
    You could say that, yes.
    I conclude that you rather not believe in a memory leak
    From the memory graph shown it looks like memory usage is pretty stable except for the spikes occurring at times. Memory leaks should look like increasing memory usage that never decreases again. I could add throttling to avoid too many thumbnails being generated at once which might be the cause of the spikes.
     

    HTPCSourcer

    Retired Team Member
  • Premium Supporter
  • May 16, 2008
    11,431
    2,334
    Germany Germany
    Country flag
    Memory leaks should look like increasing memory usage that never decreases again.
    OK. The theoretical memory usage limit for the 32bit process would be 4 GB, in practice maybe 3.5 GB because of administratve overhead. Or is it even just 2 GB? What is the code supposed to do if the memory requirements grow above such limits, e.g. instead of importing 8000 movies you would look at 20000 with respectively more pictures to load/create, "thumbnail" them and store? If you're correct then throttling would help, yet it would be subjct to the CPU speed, i.e. crash more likely on a fast PC rather than a slow one.
     

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    A more powerful CPU may increase the chance of this issue happening but I have seen the importer crash on lower powered systems. I was pointing out the memory spike since there is a 2GB limit on most 32 bit processes. I do agree with a memory leak being linear on the graph vs spikes. I have chased down a few leaks on manufacturing tools, the memory isn't released and the application just keeps grabbing more till something fails. What I am seeing with MP2 looks like a sudden demand for memory usage and maybe it is hitting a limit and causing a crash. The issue could still be other things.

    I grabbed a few more graphs to show more than just the memory spike.

    ScreenShot556.jpg

    ScreenShot555.jpg

    ScreenShot554.jpg

    ScreenShot553.jpg

    ScreenShot552.jpg

    ScreenShot551.jpg

    ScreenShot550.jpg

    ScreenShot549.jpg

    ScreenShot548.jpg
     

    morpheus_xx

    Lead Dev MP2
  • Team MediaPortal
  • March 24, 2007
    11,600
    7,060
    Germany Germany
    Country flag
    Yes those spikes could mean that there is a corrupt file, which causes on "decoding" an huge memory allocation once.
    I have read about manipulated zip files before that create an unlimited size file on decompression, so called zip bomb. It also allocates all memory until something crashes

    Gesendet von meinem SM-G930F mit Tapatalk
     

    henso

    Development Group
  • Team MediaPortal
  • February 16, 2012
    2,256
    776
    Denmark Denmark
    Country flag
    If you're correct then throttling would help, yet it would be subjct to the CPU speed, i.e. crash more likely on a fast PC rather than a slow one.
    By throttling I mean how many thumbnails can be extracted at the same time. I will also try to test if we could move the thumbnail extraction to the fanart handling instead. This should increase the import speed and decrease database size/load.
     

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    I started the stress test with WIP 168. Should OCVVideoThumbnailer be present in the installation? The import is going fast but with OCVVideoThumbnailer not in the plugin directory it leaves me wondering if it should be there.
     

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    I have a "OpenCvLib" folder in my setup.
    Didn't realize that was the same thing/replacement.

    Anyways, first 34% was really quick but it has slowed down. Up to 41% now with memory <300MB. Should have results in a day but it is looking promising so far.
     

    Users who are viewing this thread

    Similar threads
    OP Title Forum Replies Date
    M MP2 - V2.3 MP2 & USB Problems Submit: Bug Reports 29
    L MP2 - V2.3 Troubles upgrading from MP2 2.2.3 to 2.3 General 0
    T MP2 - V2.3 MP2 x64 won't run Submit: Bug Reports 5
    J Copying Old VHS tapes Newcomers Forum 8
    W MP2 - V2.3 MP2 2.3 Import von Audioquellen / Datenbankprobleme Allgemeines Support- und Diskussionsforum 26
    G MP2 - V2.3 New install and crashing MP2-2.3 General 6
    S MP2 - V2.3 MP2 No video General 6
    W MP2 - V2.3 MP2 Server stürzt immer nach kurzer Zeit ab Allgemeines Support- und Diskussionsforum 77
    toricred MP2 - V2.3 MP2 2.3 Client won't start General 6
    Grisu002 MP2 - V2.3 MP2 2.3 Client startet nicht - bleibt beim Laden hängen Allgemeines Support- und Diskussionsforum 61
    M [solved] MP2 PROBLEME Standby Allgemeines Support- und Diskussionsforum 2
    Scheibes Was passiert bei MP2? Allgemeines Support- und Diskussionsforum 42
    S Programmierte Aufnahme in MP2 - kein Aufwachen des PC Allgemeines Support- und Diskussionsforum 5
    morpheus_xx [Finished] "What's New" plugin for MP2 upgrades MediaPortal 2 20
    da3dsoul [Request] An MVP Plugin example for custom settings in MP2 Plugin Development 2
    morpheus_xx [Evaluate] Checking MP2 code portability to .Net Core MediaPortal 2 15
    da3dsoul Does MP2 not have any active devs? General 8
    da3dsoul MP2 Plugin - Getting Started (Directory Tree and Player) Plugin Development 27
    R [solved] Kein Player registriert Allgemeines Support- und Diskussionsforum 1
    -Merkur- MP2 Client startet nicht, "libXBMC_addon.dll" fehlt Allgemeines Support- und Diskussionsforum 1
    mp2-ulrich [solved] MP 2.2.3 spielt nur manche Sender Allgemeines Support- und Diskussionsforum 2
    H MP2 database won't keep in synch with recordings General 8
    H [solved] MP2 Wiedergabe von Aufnahmen und Live TV geht nicht Allgemeines Support- und Diskussionsforum 6
    ge2301 [WiP] MP2Web - A platform independent web client for MP2 MediaPortal 2 72
    F Problems with new install of MP2 Newcomers Forum 3

    Similar threads

    Top Bottom