Spanish Scraper FilmAffinity.com with IMDb.es bonus to get fanarts -- v2.1.0 | Page 30

Discussion in 'Moving Pictures' started by RoChess, December 28, 2009.

  1. peque
    • Premium Supporter

    peque Moderator - Spanish Forums

    Joined:
    August 4, 2007
    Messages:
    855
    Likes Received:
    88
    Ratings:
    +111 / 1
    Home Country:
    Spain Spain
    Show System Specs
    Yes, of course. Otherwise the test wouldn't make sense.


     
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. CATRonin

    CATRonin Portal Member

    Joined:
    June 10, 2008
    Messages:
    35
    Likes Received:
    1
    Ratings:
    +1 / 0
    Home Country:
    Spain Spain
    If you need me to do some additional test, you tell me and I can do it.

    Regards.
     
  4. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    @CATRonin, the same log files, they will tell me why it works on my side, but not for you guys. I do not actually run the scraper myself, but just run the Regular Expressions inside Expresso based on raw HTML that I expect to get from FilmAffinity, but when that gets altered for some unknown reason, then I have no idea how to solve it without more insight.

    It is one of those times where the fix is probably 5 minutes, but it takes 5 days to reproduce it and locate the source of the problem :)
     
  5. CATRonin

    CATRonin Portal Member

    Joined:
    June 10, 2008
    Messages:
    35
    Likes Received:
    1
    Ratings:
    +1 / 0
    Home Country:
    Spain Spain
    @peque, los logs que me pide de donde los saco?, que en eso estoy un poco verde.

    Saludos.
     
  6. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    @peque, Start Menu -> 'All Programs' -> Team MediaPortal -> 'MediaPortal' -> Use the shortcut 'User Files' -> navigate to the 'log' subfolder

    As for turning on scraper-script debug mode in MovPic, the MovPic manual should explain that, but in short you go to MovPic config -> "Importer Settings" tab -> Adjust the radio button to "Manually Manage" the data source scrapers -> Use the "Movie Details Data Sources" button to open new window -> use the gear icon to select the debug option (adds little green bug icon).

    Be sure to activate MediaPortal debug logs as well via MediaPortal configuration and select the 'General' menu entry. Verify or adjust that "Log verbosity" is set to 'Debug'.

    Rely on the MovingPictures config shortcut to get logs to make the process easier, and remember to allow all the concurrent background stuff to finish, delete all logs, and then run a clean import on one movie, to keep the logs as simple as possible. It is frankly too much of a task if 50 things all add to the logs and I'm trying to figure out just one issue.
     
  7. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    Is it all working again then?
     
  8. peque
    • Premium Supporter

    peque Moderator - Spanish Forums

    Joined:
    August 4, 2007
    Messages:
    855
    Likes Received:
    88
    Ratings:
    +111 / 1
    Home Country:
    Spain Spain
    Show System Specs
    Nop... I'm having same issue... Why do you say that?
     
  9. peque
    • Premium Supporter

    peque Moderator - Spanish Forums

    Joined:
    August 4, 2007
    Messages:
    855
    Likes Received:
    88
    Ratings:
    +111 / 1
    Home Country:
    Spain Spain
    Show System Specs
    Oh Jesus... I forgot to set MEDIAPORTAL log in Debug level, aswell as inside Moving Pictures.

    Here you have a way more detailed log.

    Sorry for that! It's not my first time generating logs... :(
     

    Attached Files:

  10. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,182
    Likes Received:
    1,304
    Ratings:
    +1,683 / 2
    @peque, you have a problem with Moving-Pictures connecting to FilmAffinity.com


    You might need to install Fiddler2 or Wireshark to debug at a lower level why FilmAffinity is blocking MovPic. It could however, be as simple as user-agent filtering. If I open that link in my browser I get a response back, search results are empty per "No se han encontrado coincidencias.", but connection works.

    Try to make it work in a browser on the same system, then copy the User-Agent string from that browser (use a website such as: https://www.whoishostingthis.com/tools/user-agent/ to obtain it), and copy it to Moving-Pictures configuration -> About -> Advanced Settings -> "Default User Agent" (3rd entry at the top).

    I did notice there is a bug in the search-node of the scraper-script, because it does not expect connection failures it appears and still tries to connect to a non-existing movie URL, so will have a look at fixing that part at least.

    If the user-agent adjustment fails, but it works in a browser on the same system, then how comfortable are you around advanced tools such as Fiddler2?​
     
  11. peque
    • Premium Supporter

    peque Moderator - Spanish Forums

    Joined:
    August 4, 2007
    Messages:
    855
    Likes Received:
    88
    Ratings:
    +111 / 1
    Home Country:
    Spain Spain
    Show System Specs
    One of the things I though was about user-agent, but didn't find the option. Thanks for pointing me in the right direction.

    Anyway, once I've changed user-agent, I've got same problem... What I can't understand is why it's working with a browser with same user-agent and machine, and not the search inside Moving Pictures... It's something that I, technically, can't understand.

    And yes, I'm quite familiar with wireshark so I can dig a bit in the http packets...

    Also I'm not 100% sure it's me, becauser theres @CATRonin with problems (can't say if exactly same problems).

    @CATRonin, tenemos que conseguir que saques un log a ver si te pasa lo mismo que a mí, para descartar que sea problema mío. Te paso privado.
     
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!