FilmInfo+ - A german movie details scraper with auto grouping (5 Viewers)

badboyxx

Portal Pro
June 15, 2012
728
97
Home Country
Germany Germany
I also had a Movie today without Directors and with English Summary. Send it back to the Importer and all was fine.

I tried it out with 2 movies and it works. But why doesn't it work by default?
 

RoChess

Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    @badboyxx that is what a scraper-script debug log would reveal, as it could be a timing issue the first time and web request gets blocked for example
     

    badboyxx

    Portal Pro
    June 15, 2012
    728
    97
    Home Country
    Germany Germany
    Now I have otfen the problem that the director is missing. I've uploaded the log file. The scraper is trying to connect more than 5 times and fails but why? My internet connection should be stable and good enough. And what is that for an error with "xpath parsing failed"?
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    When a scraper-debug log file is not 500kB+ for a single movie import you did something wrong (although I have made a suggestion to developers that got implemented to reduce that in the future) :)

    Activate MediaPortal debug mode for more data if you want more details.

    In short, those errors of "27-Apr-2016 19:03:37 Warn [ WebGrabber]: Connection failed: Reached retry limit of 5. URL=http://ofdbgw.org/movie/284359" do not have to mean that you have a bad internet connection, but the server at ofdbgw.org might be down, show a 404 error of page not found, or "server is to busy right now, try again later" type of error. Moving-Pictures does not exactly give full HTTP header diagnostics to verify that in the moment that it happens, and when you manually try later it might work just fine.

    The fact that directly following you see "27-Apr-2016 19:03:37 Error [ ScraperNode]: Scraper Script XPATH parsing failed: Das Stammelement ist nicht vorhanden." is a FilmInfo+ oversight in that it fetches the URL, stores it in a variable, and then plans to parse the result. With the fetching failing and not verifying that, then the parsing results in an error. I have the same in IMDb+ were I was forced to add dozens and dozens of extra lines to account for all the "what if it goes wrong" BS.

    So yeah, the whole ofdbgw.org mirror is highly unreliable, because the systems behind it are nowhere near the S3/Amazon capabilities that drive a site like IMDb.com, so especially when thousands of users hit it, you will have to deal with connection issues originating from their side. Wait a few hours and try again is the best course of action then, but that can not exactly be automated in MovPic.
     

    badboyxx

    Portal Pro
    June 15, 2012
    728
    97
    Home Country
    Germany Germany
    I did a test with another movie in the late night and it didn't help. I have still the same problem. I uploaded the log file in debug mode now.
     

    badboyxx

    Portal Pro
    June 15, 2012
    728
    97
    Home Country
    Germany Germany
    Yes, you are right. But imdb, filmstarts and ofdb provides these informations. I thought it would get the data at least from one of these sites.
     

    Brudertac

    MP Donator
  • Premium Supporter
  • October 26, 2006
    978
    277
    Augsburg
    Home Country
    Germany Germany
    The ofdbgw Service is very unreliable. I have seen that often in the past and was thinking about removing this part in the Scrapper or at last insert a switch to disable it. But currently i have no time for those changes. Maybe this will change :coffee::coffee::coffee::coffee:
     

    D3ltoroxp

    MP Donator
  • Premium Supporter
  • June 1, 2008
    3,308
    205
    Home Country
    Germany Germany
    Hm since a long time new added movies have no Rating Stars anymore. Is this handle by this scraper ? There is always 0 / 0 Stars.
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    @D3ltoroxp if you can import a single movie that shows the problem with scraper-debug enabled logs active, then it will become quickly clear what logic is failing, and it will be very easy for any scraper-script developer to fix it (including me).

    For IMDb+ I wrote up a nice wiki article to explain this -- https://github.com/RoChess/IMDbPlus/wiki/Issues-Debugging

    Everything applies in that for FilmInfo+ as well, except the starting part to edit the IMDb+ XML config file.
     

    Users who are viewing this thread

    Top Bottom