IMDB+ issue | Page 3

Discussion in 'Moving Pictures' started by fredco, December 25, 2015.

  1. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    @fredco, I'm still confused by your logs. Your last one did not contain a fresh re-import of Red Sparrow, so not much for me to look at. The original one didn't appear to contain any failure. Can you verify that IMDb+ is linked to that movie as the primary scraper script, and not say the TMDB one? This is visible inside MovPic config, on the movie details tab, and after you select the movie look for primary scraper reference.



    This can be fixed via IMDb+ plugin otherwise with the "Force IMDb+" option.
     
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. fredco

    fredco Portal Pro

    Joined:
    November 13, 2010
    Messages:
    81
    Likes Received:
    5
    Gender:
    Male
    Ratings:
    +16 / 0
    Home Country:
    Netherlands Netherlands
    Show System Specs
    @RoChess: Hi, you're talking about Red Sparrow, but I don't even have that movie, so clearly there's no import for that one. Never mentioned that movie I think...
    I've made a new import and included a screenshot and a log. I've re-imported one movie: Daredevil. As you can see, all info in Moving Pictures is in English EXCEPT the summary. You can also see IMDb+ is the primary source, which is always the case. IMDb+ is always the primary scraper script. ALL my movies are scraped by your plugin.

    Moving Pictures.jpg
     

    Attached Files:

  4. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    @fredco, mixing logs up then. What your logs are missing though is DEBUG mode from MediaPortal, and please be sure IMDb+ hidden option is configured to include HTML sources.

    A single movie import (done fresh like you did Daredevil), should be like 1MB for movingpictures.log, and not 24kB (feel free to ZIP it up)

    Open file in Notepad, and search for <html> which should be prefixed by ########## and then a ton of HTML source from IMDb and end with #######.

    That way I can properly verify the RegExp results manually in Expresso and debug issue.
     
  5. fredco

    fredco Portal Pro

    Joined:
    November 13, 2010
    Messages:
    81
    Likes Received:
    5
    Gender:
    Male
    Ratings:
    +16 / 0
    Home Country:
    Netherlands Netherlands
    Show System Specs
    @RoChess: I think I understand what you're saying, but I have "Log Verbosity" ALWAYS set to DEBUG. But I changed another setting: in Options IMDb+ Scraper.xml I set the "global_options_debug_imdb_sources" value to TRUE (was FALSE). Now I get a MUCH larger movingpictures.log. I did a fresh import of my Pixar Collection. All movies were scraped by IMDb+ and ALL summaries are in Dutch. Does this help?

    Moving Pictures Pixar Import 1.jpg Moving Pictures Pixar Import 2.jpg
     

    Attached Files:

  6. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    Hi @fredco

    The log is impossible to read right now, because it reads:

    movie 1
    movie 2
    movie 3
    movie 1
    movie 4
    movie 5
    movie 2
    movie 1
    (155,000+ other lines like that)

    That makes it extremely difficult to fix it.

    Please allow MovPic to settle for a few minutes, delete the movingpictures.log file, ensure it doesn't get created again (background tasks can take a while)

    Then send 1, as in ONE movie back to the importer (easy to do from the "Movie Details" tab via that drop down gear icon menu)

    And then give me that log file.

    You created the right logs, but that would prevent me from going down a rabbit hole looking at the code for movie #1, and running expression results from movie #2, etc.
     
  7. fredco

    fredco Portal Pro

    Joined:
    November 13, 2010
    Messages:
    81
    Likes Received:
    5
    Gender:
    Male
    Ratings:
    +16 / 0
    Home Country:
    Netherlands Netherlands
    Show System Specs
    Hi RoChess, I think I did what you wanted: deleted the log and resent "Coco" back to the importer. Log is now 400 kB, still a lot of text...
     

    Attached Files:

  8. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    @fredco, weird how your log did not contain any actual scraper debug log entries on the processing of the IMDb+ script. But I was able to obtain enough information from it to realize that short/long summaries came up empty.

    Moving-Pictures on missing data will then resort to any fallback scraper-script to compliment missing data.

    Your configuration must have TMDb or some other script configured/active, which was then used to supplement the missing summary and resort to a Dutch one for that.

    Updated the IMDb+ regular expressions for v4.9.79, so you should be good to go again.

    That is... until the next time IMDb.com decides to change things again ;)
     
  9. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    To update you can use IMDb+ plugin to mass-refresh all movies (can take a while, but has benefit of refreshing any `Top 250` entries, update votes/ratings, etc.) or you go to an individual movie inside Moving-Pictures GUI and use F9 -> 1 -> 1 to update online info. Another way is to his the refresh icon within MovPic config on Movie Details tab.
     
  10. fredco

    fredco Portal Pro

    Joined:
    November 13, 2010
    Messages:
    81
    Likes Received:
    5
    Gender:
    Male
    Ratings:
    +16 / 0
    Home Country:
    Netherlands Netherlands
    Show System Specs
    Well RoChess. you did it! All summaries are back to English, like they should! THANKS! I still have some thoughts:

    1. Strange that my log does not contain what you expect, as it's a clean install of MP, Moving Pictures and Windows. Log verbosity to DEBUG and global_options_debug_imdb_sources to TRUE. Don't know what more to do?
    2. I DO have other fall-back scaper-scripts active for the rare cases IMDb+ doesn't find a match, like IMDb.com and TMDb. That should be ok, but I don't understand why they resort to Dutch?
    3. I waited quite some time for the scraper-script to automatically update to v4.9.79 as I forgot how to force the update?
    4. I've got 1.200 movies in my library and 4 instances of MP, so scraping them all takes some time, but I've done it more than once. And like you said, it refreshes all data.
    5. Lately however it seems Moving Pictures stops responding when scanning a large number of movies. And it does so quite fast, after scanning say 50 movies. I don't think it's IMDb+ related, but I don't read about this issue at all just yet. Have you experienced this?
     
  11. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    Glad it works again for you as well. It is a constant battle to keep up with IMDb.com changes, but until AI works, it is a never ending cat-and-mouse game.

    1. MediaPortal -- in debug mode (under General somewhere)
    2. Moving-Pictures -- in scraper-debug mode (shows little green bug icon inside scraper-script section)
    3. IMDb+ -- turn on hidden XML option to include HTML sources
    I think you did 1+3, but forgot/overlooked #2, no biggie, just remember it for the future, and be sure to change it back once done as creating those extreme log files has a serious impact on performance.

    That depends on their settings, so you should allow one movie to import using them and verify why they are acting up. Localized detection is a pain in the ass for this, which is the main reason why I need to completely rewrite IMDb+, but trouble with finding the time to do so.

    Default time is 24 hours to not overload the GitHub server, but simply adjust the IMDb+ XML settings file to force a check upon launch.

    Yes, even at 3 seconds per movie, that is a full hour, but you want to be sure that all the debug stuff is disabled so it doesn't take 12 seconds per movie and you are stuck waiting for 4 hours :)

    Of course I usually launch it, go to bed, and enjoy results when I wake up.

    That I'm not sure about, but the key about solving any problem is the ability to consistently reproduce an issue, because then whatever developer needs to fix it (be it MediaPortal, Moving-Pictures, or IMDb+) can step through their code until they hit the offending code-section and examine/fix the issue.

    The problem with these random or hard to define issues is that this becomes a painstaking process, but keep in mind if it is difficult for you to reproduce, it is practically impossible for a developer to fix it. Good logs are a lifesaver though, and you might be lucky enough that somebody encountered the same thing.

    One suggestion I have is that you might have an anti-virus solution interfering, which I had myself, and was forced to exclude the MediaPortal process, MediaPortal program folder, and my import folders from the default Windows Defender AV solution that ships with Windows.

    Weird thing on that was that I was hitting 100% CPU usage, but the AV solution wasn't showing in task manager. Third party tools such as ProcMonitor from SysInternals revealed what was really going on. Microsoft system resource tool probably would have worked as well, but you get the idea.
     
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!