[fixed] [MP2-821] Importer glitches in 2.2 (1 Viewer)

JSchuricht

Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    MediaPortal 2 Version: 2.2.1

    Description
    MP2 server service crashing, might be the same issue fixed here.

    Backstory, my MP2 importer hasn't been working for a couple of weeks. It gets to 29% and gives up. Rather than fight it, I setup a new clean Windows 10 1903 VM and started importing just movies for now. Movies are on a single SMB share with ~8,000 movies.

    So far the MP2 server service has crashed twice with a bunch of OutOfMemoryExceptions in the logs.
    Code:
    System.InvalidOperationException: There was an error generating the XML document. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
    Steps to Reproduce:
    Import 8,000 movies.
     

    Attachments

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    I am still playing with this... Importing has always been lacking a bit for me. On the live MP2 server 7770 movies are seen with 8074 currently on the server. Sometimes new movies show up quickly, sometimes weeks later and sometimes not at all. Weirdly, the number of movies seen on MP2 fluctuates down sometimes. I had always thought it was from ISO files but recently converted everything to MKV and still see this issue. It's nearly impossible to track down which movies MP2 isn't seeing unless I am looking for a specific one. Recently I was looking for a specific one and realized Local media sees the folder but not the movie, the movie played fine from the MP2 client with MPCHC using the same network credentials.

    Anyways, with the new MP2 server I am trying to setup I discovered it's doing the same thing with empty folders in Local media. I have tried setting it up twice now. Try one saw 3293 movies on 6/6 and stalled out at 4370 movies on 7/5. Try two saw 3491 movies on the first day, 7/6 and stalled out at 3492 movies on 7/18. What stands out to me in the logs is 404 errors during the import. I think I am getting blocked from lookup's due to importing massive amounts of movies at once. Then I think MP2 isn't able to retry even days later and ends up skipping those movies in future imports. It's just a guess, might be something else. All of the movies appear to have folders in Local media.

    Today, I decided to try again with a new method. My live MP2 server sees the movies as \\server\movies\(a-z)\(movie name)\(movie name.mkv). To mitigate importing too many things at once I setup individual shares for each letter on the server. MP2 server would see the moves as \\server\(a-z)\(movie name)\(movie name.mkv). The plan was to add one share to MP2 at a time which would also let me check the number of movies imported and maybe find individual troublemakers.

    That leads me back to the latest crash with MP2 server on a fresh Win 10 install only running for 15 minutes. The crash happened shortly after starting the import with one share containing 127 movies.

    Event log:
    Code:
    Faulting application name: MP2-Server.exe, version: 2.2.1904.12006, time stamp: 0x5cb0f336
    Faulting module name: ntdll.dll, version: 10.0.18362.1, time stamp: 0x9bbcb4a9
    Exception code: 0xc0000374
    Fault offset: 0x000df8cd
    Faulting process id: 0x1980
    Faulting application start time: 0x01d53d9f009b567e
    Faulting application path: C:\Program Files (x86)\Team MediaPortal\MP2-Server\MP2-Server.exe
    Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
    Report Id: 2ab9ebee-5ed4-480d-82f3-7f99bbda281f
    Faulting package full name:
    Faulting package-relative application ID:
     

    Attachments

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    MP2 did find all 127 movies in the first share # after restarting the service. I'll keep experimenting with the failing imports.
     
    Last edited:

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    Crash #2. I think I discovered another minor glitch. The previous logs didn't show debug. The client shows debug checked. I toggled debug on the client and now the logs show debug. It looks like MP2 client remembers the debug setting after being disconnected from the server but doesn't push debug enabled to the server after being connected to a new server.
     

    Attachments

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    Finished importing the 2nd share A which should have 467 movies. Came up 4 short after the import. There isn't enough data yet but what stands out to me with the 4 missed imports is they are two groups of movies with nearly the same names followed by a 404 in the logs.
    The Amazing Spider-Man (2012)
    The Amazing Spider-Man 2 (2014)
    The Amityville Horror (1979)
    The Amityville Horror (2005)

    All of the movies have unique nfo files generated by TMM.
     

    Attachments

    Last edited:

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    The 3rd share B is a bit more telling. 648 movies but 105 missing from movies. All have folders in media, the ones that show no items inside the folders are the troublemakers.

    The sample size is large enough to rule out a few things.
    Many are still similar names which also makes them sequential in the list but there are enough unrelated movies to rule this out.
    My live MP2 installation isn't missing all of these movies, its import was done with an earlier version of MP2 and upgraded.
    The missing movies (only checked a few of them) are all seen in the debug log during import, most do not have a 404 error near them.

    I think my next step will be another VM with an older version of MP2 to see if a recent change made MP2 ignore movies during import.

    List of missing movies:

    Back to School (1986)
    Back to the Future Part III (1990)
    Backtrack (2015)
    Bad Ass (2010)
    Bad Ass (2012)
    Bad Boys (1995)
    Bad Boys II (2003)
    Bad Moms (2016)
    Bad Santa (2003)
    Bad Taste (1987)
    Bambi (1942)
    Bambi II (2006)
    Bananas (1971)
    BASEketball (1998)
    Batman Return of the Caped Crusaders (2016)
    Bionicle 3 Web of Shadows (2005)
    Bird Box (2018)
    Black Butterfly (2017)
    Black Mass (2015)
    Black Mirror Bandersnatch (2018)
    Black Sheep (1996)
    Black Snake Moan (2006)
    Blackhat (2015)
    Blackout (2013)
    Blades of Glory (2007)
    Blair Witch (2016)
    Blankman (1994)
    Body Double (1984)
    Body Heat (1981)
    Bolt (2008)
    Brazil (1985)
    Breakfast at Tiffany's (1961)
    Breakin' (1984)
    Breakin' 2 Electric Boogaloo (1984)
    Breakin' All the Rules (2004)
    Breaking and Entering (2006)
    Break-Up Nightmare (2016)
    Breathless (2012)
    Breezy (1973)
    Brewster's Millions (1985)
    Bride of Chucky (1998)
    Bridge of Spies (2015)
    Bright (2017)
    Brighton Beach Memoirs (1986)
    Bring Me the Head of the Machine Gun Woman (2012)
    Broken Arrow (1996)
    Broken City (2013)
    Brothers' Nest (2018)
    Bubba Ho-Tep (2002)
    Buffalo Soldiers (2001)
    Bug (2006)
    Bullet Head (2017)
    Bulworth (1998)
    Bumblebee (2018)
    Butterfly Crush (2010)
    Bye Bye Birdie (1963)
    The Babymoon (2017)
    The Babysitter (2017)
    The Bad News Bears (1976)
    The Ballad of Buster Scruggs (2018)
    The Batman vs. Dracula (2005)
    The Baytown Outlaws (2012)
    The Beach (2000)
    The Belly of an Architect (1987)
    The Benchwarmers (2006)
    The Best Little Whorehouse in Texas (1982)
    The Best Man (1999)
    The BFG (2016)
    The Big Bounce (2004)
    The Big Chill (1983)
    The Big Hit (1998)
    The Big Kahuna (1999)
    The Big Lebowski (1998)
    The Big Short (2015)
    The Big Sick (2017)
    The Big Sleep (1946)
    The Big Sleep (1978)
    The Bigfoot Project (2017)
    The Birth of a Nation (2016)
    The Blind Side (2009)
    The Blue Lagoon (1980)
    The Blues Brothers (1980)
    The Bodyguard (1992)
    The Boogey Man (1980)
    The Boondock Saints (1999)
    The Bourne Ultimatum (2007)
    The Boy Who Could Fly (1986)
    The Boy Who Cried Werewolf (2010)
    The Brainwashing of My Dad (2015)
    The Breadwinner (2017)
    The Breakfast Club (1985)
    The Break-Up (2006)
    The Bridge at Remagen (1969)
    The Bridge on the River Kwai (1957)
    The Brothers Grimm (2005)
    The Brothers Warner (2007)
    The 'Burbs (1989)
    The Butterfly Effect 2 (2006)
    The Bye Bye Man (2017)
     

    Attachments

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    Tried MP 2.2pre, failed on the first share # with 112 of 127 movies imported.

    Trying MP 2.1.3 now. First two shares are 100% imported. I am going to keep importing all 8074 movies to verify but it's looking like this is a MP2.2 issue.
     

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    To make this even more fun, here are a few screenshots from my live MP 2.2 install.

    Browse media showing movie folders where MP didn't find the movie. This is common on the new MP 2.2+ installs as well.
    ScreenShot378.jpg

    Duplicated folders which I just discovered. One has the movie and the other doesn't.
    ScreenShot379.jpg

    And series which I hadn't mentioned yet but is interesting none the less. It's also part of the reason I started trying a new MP 2.2 install.
    Two Lost in Space, the first one shows 8 seasons and 119 episodes. The second was correct yesterday, checking it today shows it's almost as bad as the first listing.
    ScreenShot380.jpg

    Inside, it has American Dad, Masters of Sex and Lost in Space
    ScreenShot381.jpg
     
    Last edited:

    JSchuricht

    Test Group
  • Team MediaPortal
  • January 13, 2009
    238
    44
    Arizona
    United States of America United States of America
    Country flag
    So this is a MP 2.2 issue. Around 10% failure in importing movies where the folder is seen in media but the movie is missing. MP 2.1.3 is finding them all in media, still have 4 that are not in movies. That may be a bad nfo, not sure how I am going to track those down. Maybe I can get a dump of movie names from the DB or flag mover and compare it to a dump of movie names from the directory. Would be a nice feature to show a flag on movies in media that are not in movies.
     

    morpheus_xx

    Lead Dev MP2
  • Team MediaPortal
  • March 24, 2007
    11,461
    7,010
    Germany Germany
    Country flag
    @henso could you take a look at this issue please?

    Gesendet von meinem SM-G930F mit Tapatalk
     

    Users Who Are Viewing This Thread (Users: 0, Guests: 1)

    OP Similar threads Forum Replies Date
    Scheibes Was passiert bei MP2? Allgemeines Support- und Diskussionsforum 9
    S Programmierte Aufnahme in MP2 - kein Aufwachen des PC Allgemeines Support- und Diskussionsforum 5
    morpheus_xx [WiP] "What's New" plugin for MP2 upgrades MediaPortal 2 5
    da3dsoul [Request] An MVP Plugin example for custom settings in MP2 Plugin Development 2
    morpheus_xx [Evaluate] Checking MP2 code portability to .Net Core MediaPortal 2 11
    da3dsoul Does MP2 not have any active devs? General 8
    da3dsoul MP2 Plugin - Getting Started (Directory Tree and Player) Plugin Development 27
    R [solved] Kein Player registriert Allgemeines Support- und Diskussionsforum 1
    -Merkur- MP2 Client startet nicht, "libXBMC_addon.dll" fehlt Allgemeines Support- und Diskussionsforum 1
    mp2-ulrich [solved] MP 2.2.3 spielt nur manche Sender Allgemeines Support- und Diskussionsforum 2
    H MP2 database won't keep in synch with recordings General 8
    H [solved] MP2 Wiedergabe von Aufnahmen und Live TV geht nicht Allgemeines Support- und Diskussionsforum 6
    ge2301 [WiP] MP2Web - A platform independent web client for MP2 MediaPortal 2 68
    F Problems with new install of MP2 Newcomers Forum 3
    catalpa Bug MP2-837 Allgemeines Support- und Diskussionsforum 2
    P MP2: US user, OTA - can NOT get SchedulesDirect plugin to work on MP1 or MP2 --> how do I get OTA schedule? General 5
    bigboss97 Does anyone has a SoundGraph imon remote file for MP2 to share? General 0
    ge2301 Android based client for MP2 MediaPortal 2 3
    R MP2 GUI Issues General 7
    P [solved] MP2 new install, 2 x hdhomerun tuners, client on same PC unable to view TV & recordings Newcomers Forum 8
    Lichloathe MP2 importing keyart instead of poster General 0
    Lichloathe Trakt for MP2 question General 7
    C Streaming/transcoding MP2 vs MP1? General 4
    P MP 2 - Request to add an additional TV series Filename protocal Feature Requests 0
    C Streaming to clients over Wifi. MP1 or MP2? Newcomers Forum 0
    Similar threads

























    Top Bottom