FilmInfo+ - A german movie details scraper with auto grouping (5 Viewers)

[badboyxx]

I also had a Movie today without Directors and with English Summary. Send it back to the Importer and all was fine.

I tried it out with 2 movies and it works. But why doesn't it work by default?

 

[RoChess]

@badboyxx that is what a scraper-script debug log would reveal, as it could be a timing issue the first time and web request gets blocked for example

 

[badboyxx]

Now I have otfen the problem that the director is missing. I've uploaded the log file. The scraper is trying to connect more than 5 times and fails but why? My internet connection should be stable and good enough. And what is that for an error with "xpath parsing failed"?

 

[RoChess]

When a scraper-debug log file is not 500kB+ for a single movie import you did something wrong (although I have made a suggestion to developers that got implemented to reduce that in the future)

Activate MediaPortal debug mode for more data if you want more details.

In short, those errors of "27-Apr-2016 19:03:37 Warn [ WebGrabber]: Connection failed: Reached retry limit of 5. URL=http://ofdbgw.org/movie/284359" do not have to mean that you have a bad internet connection, but the server at ofdbgw.org might be down, show a 404 error of page not found, or "server is to busy right now, try again later" type of error. Moving-Pictures does not exactly give full HTTP header diagnostics to verify that in the moment that it happens, and when you manually try later it might work just fine.

The fact that directly following you see "27-Apr-2016 19:03:37 Error [ ScraperNode]: Scraper Script XPATH parsing failed: Das Stammelement ist nicht vorhanden." is a FilmInfo+ oversight in that it fetches the URL, stores it in a variable, and then plans to parse the result. With the fetching failing and not verifying that, then the parsing results in an error. I have the same in IMDb+ were I was forced to add dozens and dozens of extra lines to account for all the "what if it goes wrong" BS.

So yeah, the whole ofdbgw.org mirror is highly unreliable, because the systems behind it are nowhere near the S3/Amazon capabilities that drive a site like IMDb.com, so especially when thousands of users hit it, you will have to deal with connection issues originating from their side. Wait a few hours and try again is the best course of action then, but that can not exactly be automated in MovPic.

 

[badboyxx]

I did a test with another movie in the late night and it didn't help. I have still the same problem. I uploaded the log file in debug mode now.

 

[RoChess]

Well you'll have to start contributing to the German database then -- http://ofdbgw.metawave.ch/movie/284670 looks pretty darn empty if you ask me.

 

[badboyxx]

Yes, you are right. But imdb, filmstarts and ofdb provides these informations. I thought it would get the data at least from one of these sites.

 

[Brudertac]

The ofdbgw Service is very unreliable. I have seen that often in the past and was thinking about removing this part in the Scrapper or at last insert a switch to disable it. But currently i have no time for those changes. Maybe this will change

 

[D3ltoroxp]

Hm since a long time new added movies have no Rating Stars anymore. Is this handle by this scraper ? There is always 0 / 0 Stars.

 

[RoChess]

@D3ltoroxp if you can import a single movie that shows the problem with scraper-debug enabled logs active, then it will become quickly clear what logic is failing, and it will be very easy for any scraper-script developer to fix it (including me).

For IMDb+ I wrote up a nice wiki article to explain this -- https://github.com/RoChess/IMDbPlus/wiki/Issues-Debugging

Everything applies in that for FilmInfo+ as well, except the starting part to edit the IMDb+ XML config file.