- August 4, 2007
- 861
- 99
- Home Country
-
Spain
- Moderator
- #291
Yes, of course. Otherwise the test wouldn't make sense.
Spanish Scraper FilmAffinity.com with IMDb.es bonus to get fanarts -- v2.1.0 (1 Viewer)
- Thread starter RoChess
- Start date
- March 10, 2006
- 4,434
- 1,897
- Thread starter
- Moderator
- #293
@CATRonin, the same log files, they will tell me why it works on my side, but not for you guys. I do not actually run the scraper myself, but just run the Regular Expressions inside Expresso based on raw HTML that I expect to get from FilmAffinity, but when that gets altered for some unknown reason, then I have no idea how to solve it without more insight.
It is one of those times where the fix is probably 5 minutes, but it takes 5 days to reproduce it and locate the source of the problem
It is one of those times where the fix is probably 5 minutes, but it takes 5 days to reproduce it and locate the source of the problem
- March 10, 2006
- 4,434
- 1,897
- Thread starter
- Moderator
- #295
@peque, Start Menu -> 'All Programs' -> Team MediaPortal -> 'MediaPortal' -> Use the shortcut 'User Files' -> navigate to the 'log' subfolder
As for turning on scraper-script debug mode in MovPic, the MovPic manual should explain that, but in short you go to MovPic config -> "Importer Settings" tab -> Adjust the radio button to "Manually Manage" the data source scrapers -> Use the "Movie Details Data Sources" button to open new window -> use the gear icon to select the debug option (adds little green bug icon).
Be sure to activate MediaPortal debug logs as well via MediaPortal configuration and select the 'General' menu entry. Verify or adjust that "Log verbosity" is set to 'Debug'.
Rely on the MovingPictures config shortcut to get logs to make the process easier, and remember to allow all the concurrent background stuff to finish, delete all logs, and then run a clean import on one movie, to keep the logs as simple as possible. It is frankly too much of a task if 50 things all add to the logs and I'm trying to figure out just one issue.
As for turning on scraper-script debug mode in MovPic, the MovPic manual should explain that, but in short you go to MovPic config -> "Importer Settings" tab -> Adjust the radio button to "Manually Manage" the data source scrapers -> Use the "Movie Details Data Sources" button to open new window -> use the gear icon to select the debug option (adds little green bug icon).
Be sure to activate MediaPortal debug logs as well via MediaPortal configuration and select the 'General' menu entry. Verify or adjust that "Log verbosity" is set to 'Debug'.
Rely on the MovingPictures config shortcut to get logs to make the process easier, and remember to allow all the concurrent background stuff to finish, delete all logs, and then run a clean import on one movie, to keep the logs as simple as possible. It is frankly too much of a task if 50 things all add to the logs and I'm trying to figure out just one issue.
- March 10, 2006
- 4,434
- 1,897
- Thread starter
- Moderator
- #299
@peque, you have a problem with Moving-Pictures connecting to FilmAffinity.com
You might need to install Fiddler2 or Wireshark to debug at a lower level why FilmAffinity is blocking MovPic. It could however, be as simple as user-agent filtering. If I open that link in my browser I get a response back, search results are empty per "No se han encontrado coincidencias.", but connection works.
Try to make it work in a browser on the same system, then copy the User-Agent string from that browser (use a website such as: https://www.whoishostingthis.com/tools/user-agent/ to obtain it), and copy it to Moving-Pictures configuration -> About -> Advanced Settings -> "Default User Agent" (3rd entry at the top).
03-Apr-2018 14:23:47 Warn [ WebGrabber]: Connection failed: Reached retry limit of 5. URL=http://www.filmaffinity.com/es/advsearch.php?stext=Antz%20Hormigaz%20(Spanish)%20DVD-Rip%20-%20by%20CowboysGroup%20(cowboysdelared%20es)&fromyear=0&toyear=0
You might need to install Fiddler2 or Wireshark to debug at a lower level why FilmAffinity is blocking MovPic. It could however, be as simple as user-agent filtering. If I open that link in my browser I get a response back, search results are empty per "No se han encontrado coincidencias.", but connection works.
Try to make it work in a browser on the same system, then copy the User-Agent string from that browser (use a website such as: https://www.whoishostingthis.com/tools/user-agent/ to obtain it), and copy it to Moving-Pictures configuration -> About -> Advanced Settings -> "Default User Agent" (3rd entry at the top).
I did notice there is a bug in the search-node of the scraper-script, because it does not expect connection failures it appears and still tries to connect to a non-existing movie URL, so will have a look at fixing that part at least.
If the user-agent adjustment fails, but it works in a browser on the same system, then how comfortable are you around advanced tools such as Fiddler2?
If the user-agent adjustment fails, but it works in a browser on the same system, then how comfortable are you around advanced tools such as Fiddler2?
- August 4, 2007
- 861
- 99
- Home Country
-
Spain
- Moderator
- #300
One of the things I though was about user-agent, but didn't find the option. Thanks for pointing me in the right direction.
Anyway, once I've changed user-agent, I've got same problem... What I can't understand is why it's working with a browser with same user-agent and machine, and not the search inside Moving Pictures... It's something that I, technically, can't understand.
And yes, I'm quite familiar with wireshark so I can dig a bit in the http packets...
Also I'm not 100% sure it's me, becauser theres @CATRonin with problems (can't say if exactly same problems).
@CATRonin, tenemos que conseguir que saques un log a ver si te pasa lo mismo que a mí, para descartar que sea problema mío. Te paso privado.
Anyway, once I've changed user-agent, I've got same problem... What I can't understand is why it's working with a browser with same user-agent and machine, and not the search inside Moving Pictures... It's something that I, technically, can't understand.
And yes, I'm quite familiar with wireshark so I can dig a bit in the http packets...
Also I'm not 100% sure it's me, becauser theres @CATRonin with problems (can't say if exactly same problems).
@CATRonin, tenemos que conseguir que saques un log a ver si te pasa lo mismo que a mí, para descartar que sea problema mío. Te paso privado.