FilmInfo+ - A german movie details scraper with auto grouping | Page 73

Discussion in 'Moving Pictures' started by Merlyn, September 6, 2011.

  1. badboyxx

    badboyxx Portal Pro

    Joined:
    June 15, 2012
    Messages:
    662
    Likes Received:
    45
    Gender:
    Male
    Ratings:
    +71 / 9
    Home Country:
    Germany Germany
    Show System Specs
    I tried it out with 2 movies and it works. But why doesn't it work by default?


     
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,153
    Likes Received:
    1,294
    Ratings:
    +1,659 / 2
    @badboyxx that is what a scraper-script debug log would reveal, as it could be a timing issue the first time and web request gets blocked for example
     
  4. badboyxx

    badboyxx Portal Pro

    Joined:
    June 15, 2012
    Messages:
    662
    Likes Received:
    45
    Gender:
    Male
    Ratings:
    +71 / 9
    Home Country:
    Germany Germany
    Show System Specs
    Now I have otfen the problem that the director is missing. I've uploaded the log file. The scraper is trying to connect more than 5 times and fails but why? My internet connection should be stable and good enough. And what is that for an error with "xpath parsing failed"?
     
  5. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,153
    Likes Received:
    1,294
    Ratings:
    +1,659 / 2
    When a scraper-debug log file is not 500kB+ for a single movie import you did something wrong (although I have made a suggestion to developers that got implemented to reduce that in the future) :)

    Activate MediaPortal debug mode for more data if you want more details.

    In short, those errors of "27-Apr-2016 19:03:37 Warn [ WebGrabber]: Connection failed: Reached retry limit of 5. URL=http://ofdbgw.org/movie/284359" do not have to mean that you have a bad internet connection, but the server at ofdbgw.org might be down, show a 404 error of page not found, or "server is to busy right now, try again later" type of error. Moving-Pictures does not exactly give full HTTP header diagnostics to verify that in the moment that it happens, and when you manually try later it might work just fine.

    The fact that directly following you see "27-Apr-2016 19:03:37 Error [ ScraperNode]: Scraper Script XPATH parsing failed: Das Stammelement ist nicht vorhanden." is a FilmInfo+ oversight in that it fetches the URL, stores it in a variable, and then plans to parse the result. With the fetching failing and not verifying that, then the parsing results in an error. I have the same in IMDb+ were I was forced to add dozens and dozens of extra lines to account for all the "what if it goes wrong" BS.

    So yeah, the whole ofdbgw.org mirror is highly unreliable, because the systems behind it are nowhere near the S3/Amazon capabilities that drive a site like IMDb.com, so especially when thousands of users hit it, you will have to deal with connection issues originating from their side. Wait a few hours and try again is the best course of action then, but that can not exactly be automated in MovPic.
     
    • Thank You! Thank You! x 1
  6. badboyxx

    badboyxx Portal Pro

    Joined:
    June 15, 2012
    Messages:
    662
    Likes Received:
    45
    Gender:
    Male
    Ratings:
    +71 / 9
    Home Country:
    Germany Germany
    Show System Specs
    I did a test with another movie in the late night and it didn't help. I have still the same problem. I uploaded the log file in debug mode now.
     
  7. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,153
    Likes Received:
    1,294
    Ratings:
    +1,659 / 2
  8. badboyxx

    badboyxx Portal Pro

    Joined:
    June 15, 2012
    Messages:
    662
    Likes Received:
    45
    Gender:
    Male
    Ratings:
    +71 / 9
    Home Country:
    Germany Germany
    Show System Specs
    Yes, you are right. But imdb, filmstarts and ofdb provides these informations. I thought it would get the data at least from one of these sites.
     
  9. Brudertac
    • Premium Supporter

    Brudertac MP Donator

    Joined:
    October 26, 2006
    Messages:
    978
    Likes Received:
    117
    Gender:
    Male
    Location:
    Augsburg
    Ratings:
    +277 / 1
    Home Country:
    Germany Germany
    Show System Specs
    The ofdbgw Service is very unreliable. I have seen that often in the past and was thinking about removing this part in the Scrapper or at last insert a switch to disable it. But currently i have no time for those changes. Maybe this will change :coffee::coffee::coffee::coffee:
     
  10. D3ltoroxp
    • Premium Supporter

    D3ltoroxp MP Donator

    Joined:
    June 1, 2008
    Messages:
    3,308
    Likes Received:
    133
    Gender:
    Male
    Ratings:
    +207 / 9
    Home Country:
    Germany Germany
    Show System Specs
    Hm since a long time new added movies have no Rating Stars anymore. Is this handle by this scraper ? There is always 0 / 0 Stars.
     
  11. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,153
    Likes Received:
    1,294
    Ratings:
    +1,659 / 2
    @D3ltoroxp if you can import a single movie that shows the problem with scraper-debug enabled logs active, then it will become quickly clear what logic is failing, and it will be very easy for any scraper-script developer to fix it (including me).

    For IMDb+ I wrote up a nice wiki article to explain this -- https://github.com/RoChess/IMDbPlus/wiki/Issues-Debugging

    Everything applies in that for FilmInfo+ as well, except the starting part to edit the IMDb+ XML config file.
     
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!