Import process hangs, doesn't continue (1 Viewer)

dir

MP Donator
  • Premium Supporter
  • August 30, 2006
    408
    36
    Home Country
    Afghanistan Afghanistan
    I've been trying for 3 days to get a brand new htpc set up with MediaPortal 1.2b, specifically Moving Pictures. I currently have an intermittent and very slow Internet connection so I've been assuming that to be the cause of the problem, but after looking at the MP logs, I'm not so sure. Also, my net connection's been up for 9 hours solidly over night and MP is still hanging/crashing.

    The import process goes through my 1600 movie collection and always uses the Internet to find the best match. Then, once all 1600 movies are matched, it proceeds to start downloading all the info about it. Only after this happens does it add the movie to the collection.

    At some point, usually during this first pass but sometimes later on when it actually starts storing info about each movie (what I call the 2nd pass), it hangs. By "hang", I mean that the status line shows that it's "Retrieving details for: <movie>", and never changes, even several hours later.

    When it hangs, which I had assumed was because it gets a network glitch, there's no way to resume it. When it's hung, or rather - stuck on a movie and never moves on to the next one - I can go through the movie list and do a manual search, and it will retrieve the info for that one movie. So MP isn't actually hung, only the main retreival thread, stuck on whatever movie it was on when the network glitched or it internally crashed. There's no timeout to move on to the next movie, and no way to prod the process into continuing.

    I've just woken up after leaving it running again from scratch, only to find that it was on movie 5 out of 1600 - so again, it got through the entire 1600 movies for the first pass and was only just starting to do any actual work when it hung. The relevent part of the log file shows a strange error that doesn't seem to be network related:

    24-May-2011 04:14:53 Info [ MovieImporter]: Retrieving details for "White Christmas"
    24-May-2011 04:14:53 Debug [ DatabaseManager]: INSERTING: Irving Berlin's White Christmas (1954).avi
    24-May-2011 04:14:54 Debug [ FollwitConnector]: Response received from follw.it: <?xml version="1.0" encoding="UTF-8"?> <methodResponse> <params> <param> <value> <array> <data> <value> <struct> <member> <name>InternalId</name> <value> <string>13</string> </value> </member> <member> <name>MovieId</name> <value> <string>881</string> </value> </member> <member> <name>UserRating</name> <value> <string>4</string> </value> </member> <member> <name>Watched</name> <value> <string></string> </value> </member> </struct></value> </data> </array> </value> </param> </params> </methodResponse>
    24-May-2011 04:14:55 Debug [ DatabaseManager]: UPDATING: Bad Santa
    24-May-2011 04:14:55 Debug [ DatabaseManager]: UPDATING: DBUserMovieSettings (413)
    24-May-2011 04:15:01 Debug [ DataProviderManager]: UPDATE: Title='Irving Berlin's White Christmas', Provider='imdb.com', Version=2.1.3, Result=SUCCESS
    24-May-2011 04:15:05 Debug [ DataProviderManager]: UPDATE: Title='Irving Berlin's White Christmas', Provider='themoviedb.org', Version=Internal, Result=SUCCESS
    24-May-2011 04:15:05 Debug [ DataProviderManager]: UPDATE: Title='Irving Berlin's White Christmas', Provider='Ember Media Manager (Local)', Version=1.0.2, Result=FAILED_NEED_ID
    24-May-2011 04:15:05 Info [ DBMovieInfo]: Added cover art for 'Irving Berlin's White Christmas' from: \\MOFO\Audio Video Media\Christmas Movies\Irving Berlin's White Christmas (1954)\folder.jpg
    24-May-2011 04:15:09 Fatal [ MovieImporter]: Unhandled error in MediaScanner. System.IO.IOException: The process cannot access the file 'C:\ProgramData\Team MediaPortal\MediaPortal\Thumbs\MovingPictures\Covers\FullSize\{Irving.Berlin's.White.Christmas} [1687307958].jpg' because it is being used by another process.
    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
    at System.IO.File.Delete(String path)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieResources.ImageResource.VerifyAndResize(ImageSize minSize, ImageSize maxSize)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieResources.ImageResource.FromUrl(String url, Boolean ignoreRestrictions, ImageSize minSize, ImageSize maxSize, Boolean redownload)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieResources.Cover.FromUrl(DBMovieInfo movie, String url, Boolean ignoreRestrictions, ImageLoadResults& status)
    at MediaPortal.Plugins.MovingPictures.Database.DBMovieInfo.AddCoverFromURL(String url, Boolean ignoreRestrictions)
    at MediaPortal.Plugins.MovingPictures.DataProviders.TheMovieDbProvider.GetArtwork(DBMovieInfo movie)
    at MediaPortal.Plugins.MovingPictures.DataProviders.DataProviderManager.GetArtwork(DBMovieInfo movie)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieImporter.AssignFileToMovie(IList`1 localMedia, DBMovieInfo movie, Boolean update)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieImporter.AssignAndCommit(MovieMatch match, Boolean update)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieImporter.ProcessNextApprovedMatches()
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieImporter.ScanMedia()

    I checked, and the file in question exists, but it's not in use by anything. This PC is sitting unused. I don't even have antivirus software installed yet (apart from Microsoft's), and there are no other processes that could be using this file. Heck, it's a virgin set-up with Windows 7 Pro installed, all necessary drivers, SAF 5.0, and then MP1.2beta.

    Update: It's happened again.
    I've completely removed all traces of MP after deleting all paths. I watched it delete all the movies in the database. I deleted all the images in the thumbs directory and then deleted the actual database file itself. I restarted it and it again has done the same thing. MP got all the way through 1600 movies in the first pass and has hung (on a different movie) in the 2nd pass. The messaage is "Retrieving details for: Godzilla, Mothra and King Ghidorah: Giant Monsters All-Out Attack".

    My net connection has been up the entire time.

    The log file shows:

    24-May-2011 11:13:48 Debug [ DatabaseManager]: INSERTING: Godzilla, Mothra and King Ghidorah; Giant Monsters All-Out Attack (2001).avi
    24-May-2011 11:13:49 Debug [ DataProviderManager]: UPDATE: Title='Godzilla, Mothra and King Ghidorah: Giant Monsters All-Out Attack', Provider='themoviedb.org', Version=Internal, Result=SUCCESS
    24-May-2011 11:14:02 Debug [ DataProviderManager]: UPDATE: Title='Godzilla, Mothra and King Ghidorah: Giant Monsters All-Out Attack', Provider='imdb.com', Version=2.1.3, Result=SUCCESS
    24-May-2011 11:14:02 Debug [ DataProviderManager]: UPDATE: Title='Godzilla, Mothra and King Ghidorah: Giant Monsters All-Out Attack', Provider='Ember Media Manager (Local)', Version=1.0.2, Result=FAILED_NEED_ID
    24-May-2011 11:14:02 Info [ DBMovieInfo]: Added cover art for 'Godzilla, Mothra and King Ghidorah: Giant Monsters All-Out Attack' from: \\MOFO\Audio Video Media\Movies\Godzilla, Mothra and King Ghidorah; Giant Monsters All-Out Attack (2001)\folder.jpg
    24-May-2011 11:14:08 Fatal [ MovieImporter]: Unhandled error in MediaScanner. System.IO.IOException: The process cannot access the file 'C:\ProgramData\Team MediaPortal\MediaPortal\Thumbs\MovingPictures\Covers\FullSize\{Godzilla,.Mothra.and.King.Ghidorah_.Giant.Monsters.All-Out.Attack} [-205799697].jpg' because it is being used by another process.
    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
    at System.IO.File.Delete(String path)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieResources.ImageResource.VerifyAndResize(ImageSize minSize, ImageSize maxSize)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieResources.ImageResource.FromUrl(String url, Boolean ignoreRestrictions, ImageSize minSize, ImageSize maxSize, Boolean redownload)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieResources.Cover.FromUrl(DBMovieInfo movie, String url, Boolean ignoreRestrictions, ImageLoadResults& status)
    at MediaPortal.Plugins.MovingPictures.Database.DBMovieInfo.AddCoverFromURL(String url, Boolean ignoreRestrictions)
    at MediaPortal.Plugins.MovingPictures.DataProviders.TheMovieDbProvider.GetArtwork(DBMovieInfo movie)
    at MediaPortal.Plugins.MovingPictures.DataProviders.DataProviderManager.GetArtwork(DBMovieInfo movie)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieImporter.AssignFileToMovie(IList`1 localMedia, DBMovieInfo movie, Boolean update)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieImporter.AssignAndCommit(MovieMatch match, Boolean update)
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieImporter.ProcessNextApprovedMatches()
    at MediaPortal.Plugins.MovingPictures.LocalMediaManagement.MovieImporter.ScanMedia()

    I used Process Explorer about 40 minutes after it had been sitting there not progressing. The only .jpg file open by any process is opened by Configuration.exe and is "C:\ProgramData\Team MP etc....\MovingPictures\Covers\Thumbs\{Yes.Man} [-2080981462].jpg. There's no mention of this file or movie in the log file.

    I guess if there really is something else keeping this file open, then the app really needs to simply skip that movie entry and continue on. I don't know what's opening the file - there are no networked PCs accessing that machine, it's a virgin machine with no anti-virus or any other apps installed.

    I'll zip up both log files and attach them.
     

    Attachments

    • movingpictures Dir.zip
      1.2 MB

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    I don't even have antivirus software installed yet (apart from Microsoft's)

    If you mean Microsoft Security Essentials with this statement, then you *do* have anti-virus installed. And I've found that MSE can indeed cause issues, but they are easy to solve as well, just add both Team MediaPortal folders (program and data) to the exclusion list and you are good to go.
     

    mastyman

    MP Donator
  • Premium Supporter
  • June 1, 2010
    364
    68
    Auckland
    Home Country
    New Zealand New Zealand
    i have exactly this problem every time i try to import my collection (from scratch) , "[ MovieImporter]: Unhandled error in MediaScanner"

    my workaround is this

    Once the import paths are added, close the importer, then just run mediaportal and let it run, i havent had it crash this way,

    I also opened a issue for it a few months ago here, maybe you can add your logs to it and a comment.
     

    dir

    MP Donator
  • Premium Supporter
  • August 30, 2006
    408
    36
    Home Country
    Afghanistan Afghanistan
    Ok, I've excluded c:\programdata\Team Media Portal in MS Security Essentials. Interesting idea although I would expect the entire world to have literally millions or hundreds of millions of problems from people using MSSE if it keeps files locked while applications are trying to work with them. But whatever. We'll see.

    I've also moved out all the movies from my main \Movies directory and am adding them back in one letter at a time. At least that way I can slowly make progress.

    I might try just running MP and letting it run for a few hours to see if it gets any further.

    I'll check the bug mentioned and add my logs - the importer definitely shouldn't hang forever just because an image file is in use.
     

    dir

    MP Donator
  • Premium Supporter
  • August 30, 2006
    408
    36
    Home Country
    Afghanistan Afghanistan
    Yeah but you're a Kiwi so chances are the computers simply don't work properly over there anyway.

    Been feeding it only 100 to 200 movies at a time and so far no problems. Doing a 500-movie chunk now. If that completes, I'll feed it the remaining movies (1000). If that completes, I'll rename the completed database and start a new one from scratch and do an entire large run. If that completes, I'll re-enable MS Security Essentials and do another run. If that fails, then MSSE is likely the culprit.

    p.s. Kiwi joke was just an Aussie-Kiwi thing for those non-antipodeans.
     

    dir

    MP Donator
  • Premium Supporter
  • August 30, 2006
    408
    36
    Home Country
    Afghanistan Afghanistan
    Ok, good news in a way. I threw 200 movies at it and it worked fine. I then tried 480 or so. Ok again. Then I tried throwing 779 at it and it hung in exactly trhe same way.

    This is with MS Security Essentials configured to ignore c:\programdata\team mediaportal. So it can't be that locking out the files.
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    I actually am having problems myself recently on a mass-refresh on my collection, but in my case they are OutOfMemory errors that are causing things to crash hard during import. When I select a smaller selection of movies it works fine, so now I'm wondering if it somehow relates to the same section of code. Since I got pagefile disabled, an out of memory error for me is dealt with via a hard crash, but in your case it might result in the errors you are seeing (wild guess).

    So lets help the developers narrow things down then, for example you can disable MediaInfo during import and try again, the same with Follw.it. They most likely have nothing to do with it, but the more we can eliminate and provide debug logs on, the easier it will be for the developers to narrow the problem down.

    For the same reasons disable as many plugins as possible, because it could also be that for example the Fanart Handler plugin is trying to do something with the same fanart images causing the conflict. Not saying that you have that plugin enabled, or if it is the problem, but the more we exclude the faster this will get resolved.

    So be sure to provide all the log files, not what you think is relevant.
     

    Users who are viewing this thread

    Top Bottom