[fixed] [MP2-821] Importer glitches in 2.2 (2 Viewers)

JSchuricht

MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    MediaPortal 2 Version: 2.2.1

    Description
    MP2 server service crashing, might be the same issue fixed here.

    Backstory, my MP2 importer hasn't been working for a couple of weeks. It gets to 29% and gives up. Rather than fight it, I setup a new clean Windows 10 1903 VM and started importing just movies for now. Movies are on a single SMB share with ~8,000 movies.

    So far the MP2 server service has crashed twice with a bunch of OutOfMemoryExceptions in the logs.
    Code:
    System.InvalidOperationException: There was an error generating the XML document. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.

    Steps to Reproduce:
    Import 8,000 movies.
     

    Attachments

    • MediaPortal2-Logs-2019-06-03-21.05.09.zip
      8.3 MB
    • MediaPortal2-Logs-2019-06-04-08.51.15.zip
      8 MB

    JSchuricht

    MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    I am still playing with this... Importing has always been lacking a bit for me. On the live MP2 server 7770 movies are seen with 8074 currently on the server. Sometimes new movies show up quickly, sometimes weeks later and sometimes not at all. Weirdly, the number of movies seen on MP2 fluctuates down sometimes. I had always thought it was from ISO files but recently converted everything to MKV and still see this issue. It's nearly impossible to track down which movies MP2 isn't seeing unless I am looking for a specific one. Recently I was looking for a specific one and realized Local media sees the folder but not the movie, the movie played fine from the MP2 client with MPCHC using the same network credentials.

    Anyways, with the new MP2 server I am trying to setup I discovered it's doing the same thing with empty folders in Local media. I have tried setting it up twice now. Try one saw 3293 movies on 6/6 and stalled out at 4370 movies on 7/5. Try two saw 3491 movies on the first day, 7/6 and stalled out at 3492 movies on 7/18. What stands out to me in the logs is 404 errors during the import. I think I am getting blocked from lookup's due to importing massive amounts of movies at once. Then I think MP2 isn't able to retry even days later and ends up skipping those movies in future imports. It's just a guess, might be something else. All of the movies appear to have folders in Local media.

    Today, I decided to try again with a new method. My live MP2 server sees the movies as \\server\movies\(a-z)\(movie name)\(movie name.mkv). To mitigate importing too many things at once I setup individual shares for each letter on the server. MP2 server would see the moves as \\server\(a-z)\(movie name)\(movie name.mkv). The plan was to add one share to MP2 at a time which would also let me check the number of movies imported and maybe find individual troublemakers.

    That leads me back to the latest crash with MP2 server on a fresh Win 10 install only running for 15 minutes. The crash happened shortly after starting the import with one share containing 127 movies.

    Event log:
    Code:
    Faulting application name: MP2-Server.exe, version: 2.2.1904.12006, time stamp: 0x5cb0f336
    Faulting module name: ntdll.dll, version: 10.0.18362.1, time stamp: 0x9bbcb4a9
    Exception code: 0xc0000374
    Fault offset: 0x000df8cd
    Faulting process id: 0x1980
    Faulting application start time: 0x01d53d9f009b567e
    Faulting application path: C:\Program Files (x86)\Team MediaPortal\MP2-Server\MP2-Server.exe
    Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
    Report Id: 2ab9ebee-5ed4-480d-82f3-7f99bbda281f
    Faulting package full name:
    Faulting package-relative application ID:
     

    Attachments

    • MediaPortal2-Logs-2019-07-18-15.16.48.zip
      56.1 KB

    JSchuricht

    MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    MP2 did find all 127 movies in the first share # after restarting the service. I'll keep experimenting with the failing imports.
     
    Last edited:

    JSchuricht

    MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    Crash #2. I think I discovered another minor glitch. The previous logs didn't show debug. The client shows debug checked. I toggled debug on the client and now the logs show debug. It looks like MP2 client remembers the debug setting after being disconnected from the server but doesn't push debug enabled to the server after being connected to a new server.
     

    Attachments

    • MediaPortal2-Logs-2019-07-18-15.52.24.zip
      2.9 MB

    JSchuricht

    MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    Finished importing the 2nd share A which should have 467 movies. Came up 4 short after the import. There isn't enough data yet but what stands out to me with the 4 missed imports is they are two groups of movies with nearly the same names followed by a 404 in the logs.
    The Amazing Spider-Man (2012)
    The Amazing Spider-Man 2 (2014)
    The Amityville Horror (1979)
    The Amityville Horror (2005)

    All of the movies have unique nfo files generated by TMM.
     

    Attachments

    • MediaPortal2-Logs-2019-07-18-16.26.11.zip
      3.3 MB
    Last edited:

    JSchuricht

    MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    The 3rd share B is a bit more telling. 648 movies but 105 missing from movies. All have folders in media, the ones that show no items inside the folders are the troublemakers.

    The sample size is large enough to rule out a few things.
    Many are still similar names which also makes them sequential in the list but there are enough unrelated movies to rule this out.
    My live MP2 installation isn't missing all of these movies, its import was done with an earlier version of MP2 and upgraded.
    The missing movies (only checked a few of them) are all seen in the debug log during import, most do not have a 404 error near them.

    I think my next step will be another VM with an older version of MP2 to see if a recent change made MP2 ignore movies during import.

    List of missing movies:

    Back to School (1986)
    Back to the Future Part III (1990)
    Backtrack (2015)
    Bad Ass (2010)
    Bad Ass (2012)
    Bad Boys (1995)
    Bad Boys II (2003)
    Bad Moms (2016)
    Bad Santa (2003)
    Bad Taste (1987)
    Bambi (1942)
    Bambi II (2006)
    Bananas (1971)
    BASEketball (1998)
    Batman Return of the Caped Crusaders (2016)
    Bionicle 3 Web of Shadows (2005)
    Bird Box (2018)
    Black Butterfly (2017)
    Black Mass (2015)
    Black Mirror Bandersnatch (2018)
    Black Sheep (1996)
    Black Snake Moan (2006)
    Blackhat (2015)
    Blackout (2013)
    Blades of Glory (2007)
    Blair Witch (2016)
    Blankman (1994)
    Body Double (1984)
    Body Heat (1981)
    Bolt (2008)
    Brazil (1985)
    Breakfast at Tiffany's (1961)
    Breakin' (1984)
    Breakin' 2 Electric Boogaloo (1984)
    Breakin' All the Rules (2004)
    Breaking and Entering (2006)
    Break-Up Nightmare (2016)
    Breathless (2012)
    Breezy (1973)
    Brewster's Millions (1985)
    Bride of Chucky (1998)
    Bridge of Spies (2015)
    Bright (2017)
    Brighton Beach Memoirs (1986)
    Bring Me the Head of the Machine Gun Woman (2012)
    Broken Arrow (1996)
    Broken City (2013)
    Brothers' Nest (2018)
    Bubba Ho-Tep (2002)
    Buffalo Soldiers (2001)
    Bug (2006)
    Bullet Head (2017)
    Bulworth (1998)
    Bumblebee (2018)
    Butterfly Crush (2010)
    Bye Bye Birdie (1963)
    The Babymoon (2017)
    The Babysitter (2017)
    The Bad News Bears (1976)
    The Ballad of Buster Scruggs (2018)
    The Batman vs. Dracula (2005)
    The Baytown Outlaws (2012)
    The Beach (2000)
    The Belly of an Architect (1987)
    The Benchwarmers (2006)
    The Best Little Whorehouse in Texas (1982)
    The Best Man (1999)
    The BFG (2016)
    The Big Bounce (2004)
    The Big Chill (1983)
    The Big Hit (1998)
    The Big Kahuna (1999)
    The Big Lebowski (1998)
    The Big Short (2015)
    The Big Sick (2017)
    The Big Sleep (1946)
    The Big Sleep (1978)
    The Bigfoot Project (2017)
    The Birth of a Nation (2016)
    The Blind Side (2009)
    The Blue Lagoon (1980)
    The Blues Brothers (1980)
    The Bodyguard (1992)
    The Boogey Man (1980)
    The Boondock Saints (1999)
    The Bourne Ultimatum (2007)
    The Boy Who Could Fly (1986)
    The Boy Who Cried Werewolf (2010)
    The Brainwashing of My Dad (2015)
    The Breadwinner (2017)
    The Breakfast Club (1985)
    The Break-Up (2006)
    The Bridge at Remagen (1969)
    The Bridge on the River Kwai (1957)
    The Brothers Grimm (2005)
    The Brothers Warner (2007)
    The 'Burbs (1989)
    The Butterfly Effect 2 (2006)
    The Bye Bye Man (2017)
     

    Attachments

    • MediaPortal2-Logs-2019-07-18-19.20.30.zip
      7.2 MB

    JSchuricht

    MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    Tried MP 2.2pre, failed on the first share # with 112 of 127 movies imported.

    Trying MP 2.1.3 now. First two shares are 100% imported. I am going to keep importing all 8074 movies to verify but it's looking like this is a MP2.2 issue.
     

    JSchuricht

    MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    To make this even more fun, here are a few screenshots from my live MP 2.2 install.

    Browse media showing movie folders where MP didn't find the movie. This is common on the new MP 2.2+ installs as well.
    ScreenShot378.jpg

    Duplicated folders which I just discovered. One has the movie and the other doesn't.
    ScreenShot379.jpg

    And series which I hadn't mentioned yet but is interesting none the less. It's also part of the reason I started trying a new MP 2.2 install.
    Two Lost in Space, the first one shows 8 seasons and 119 episodes. The second was correct yesterday, checking it today shows it's almost as bad as the first listing.
    ScreenShot380.jpg

    Inside, it has American Dad, Masters of Sex and Lost in Space
    ScreenShot381.jpg
     
    Last edited:

    JSchuricht

    MP Donator
  • Premium Supporter
  • January 13, 2009
    239
    45
    Arizona
    Home Country
    United States of America United States of America
    So this is a MP 2.2 issue. Around 10% failure in importing movies where the folder is seen in media but the movie is missing. MP 2.1.3 is finding them all in media, still have 4 that are not in movies. That may be a bad nfo, not sure how I am going to track those down. Maybe I can get a dump of movie names from the DB or flag mover and compare it to a dump of movie names from the directory. Would be a nice feature to show a flag on movies in media that are not in movies.
     

    morpheus_xx

    Retired Team Member
  • Team MediaPortal
  • March 24, 2007
    12,073
    7,459
    Home Country
    Germany Germany
    @henso could you take a look at this issue please?

    Gesendet von meinem SM-G930F mit Tapatalk
     

    Users who are viewing this thread

    Top Bottom