Spanish Scraper FilmAffinity.com with IMDb.es bonus to get fanarts -- v2.1.0 (1 Viewer)

peque

Moderator - Spanish Forums
  • Premium Supporter
  • August 4, 2007
    859
    98
    Spain Spain
    Country flag
    Ok. If I can do something more, just ask.
     

    CATRonin

    Portal Member
    June 10, 2008
    35
    1
    Spain Spain
    Hola.

    A mi no me ha funcionado.

    He sustituido la dll por la que hay en archivos de programa.... plugins windows y me da el error que adjunto.

    Habia hecho antes un backup de la dll y la he vuelto a poner.

    Tengo .Net 4.5

    Alguna idea?

    Edito: Tengo Windows 7

    Saludos.
     

    Attachments

    Last edited:

    peque

    Moderator - Spanish Forums
  • Premium Supporter
  • August 4, 2007
    859
    98
    Spain Spain
    Country flag
    Scrapper is totally failing for me... anyone else with problems? Any decent alternative for spanish people?

    PD: Filmaffinity web layout is quite the same, I think.

    PD2: Logs added.
     

    Attachments

    Last edited:

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,342
    1,824
    Country flag
    They probably made some HTML changes that cause the RegEx to fail.

    Should be able to fix it with the logs you provided if they contain the error in detail.

    Will check later tonight after work.
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,342
    1,824
    Country flag
    I did notice the following in your log file:

    09-Sep-2019 09:09:15 Debug [ ScraperNode]: Retrieving URL: FilmAffinity
    09-Sep-2019 09:09:15 Error [ WebGrabber]: Connection failed: URL=FilmAffinity, Status=429, Description=Too Many Requests.

    So they appear to be blocking you from hitting them with too many requests, which then invalidates any results.

    The scraper-script fall back logic then seems to fail, so instead of just aborting it ends up storing incorrect data.

    Will look into fixing that.

    In the meantime for you, try importing at a very slow rate, as FilmAffinity quickly sees you as trying to DDoS them and blocks your requests. That is a major benefit of Amazon owning IMDb.com as you can hit them with hundreds of requests per seconds and they just take it :whistle:
     

    peque

    Moderator - Spanish Forums
  • Premium Supporter
  • August 4, 2007
    859
    98
    Spain Spain
    Country flag
    I did notice the following in your log file:

    09-Sep-2019 09:09:15 Debug [ ScraperNode]: Retrieving URL: FilmAffinity
    09-Sep-2019 09:09:15 Error [ WebGrabber]: Connection failed: URL=FilmAffinity, Status=429, Description=Too Many Requests.

    So they appear to be blocking you from hitting them with too many requests, which then invalidates any results.

    The scraper-script fall back logic then seems to fail, so instead of just aborting it ends up storing incorrect data.

    Will look into fixing that.

    In the meantime for you, try importing at a very slow rate, as FilmAffinity quickly sees you as trying to DDoS them and blocks your requests. That is a major benefit of Amazon owning IMDb.com as you can hit them with hundreds of requests per seconds and they just take it :whistle:
    That makes sense, because I moved a whole dir with more than 200 films... I must think now how to do it slower.

    Desde el móvil...
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,342
    1,824
    Country flag
    Moving-Pictures advanced settings default multi-threading is 5 movies at a time if memory serves me.

    Changing that to 1 slows things down a lot, and might be enough.
     

    peque

    Moderator - Spanish Forums
  • Premium Supporter
  • August 4, 2007
    859
    98
    Spain Spain
    Country flag
    I'm back... :)

    Now I would say I'm not "blacklisted", but I can't retrieve movie details. I attach log.

    It seems it's retrieving search results Ok, but then fails trying to get film details. It seems film details url is not correct.
     

    Attachments

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,342
    1,824
    Country flag
    Dealing with issues at work, so will look at it tomorrow
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,342
    1,824
    Country flag
    Irritated while waiting for NOC to call me back, so perfect time to calm down looking at regular expressions :whistle:

    Try 2.1.8 which is available now. FilmAffinity changed the way they reference URLs so the RegEx was failing.
     

    Users Who Are Viewing This Thread (Users: 0, Guests: 1)

    Top Bottom