Adult movie scraper (details+covers) -- v2.0.7 (1 Viewer)

no1bossman

Portal Member
April 30, 2009
24
0
RoChess, I used your suggestion a few years back regarding using the parental filtering option. Long story short works fine, but I never see any fanart for these movies. Is this by design, or is there something you suggest I should check first? Cheers.
 

RoChess

Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,345
    1,824
    Country flag
    • Thread starter
    • Moderator
    • #72
    CDUniverse offers no fanarts/backdrops. TMDb has limited support for 'adult' movies, and requires the IMDb tt-ID reference to work.

    So the only way possible would be to manually add IMDb tt-ID to the filename or NFO file and then it will work (if you reimport).

    Example: http://www.imdb.com/title/tt1733302/
    TMDb = http://www.themoviedb.org/movie/50019-this-ain-t-avatar-xxx

    You can also use TMDb scraper obviously, but they have a very limited 'adult' collection, which is user-managed so you can always add missing ones.
     

    Matt Kirby

    Portal Member
    June 14, 2009
    43
    8
    United Kingdom United Kingdom
    RoChess,

    Firstly, thanks for you work on this script and the IMDB+ one - great work!

    However, I think that the scraping detection at CD Universe has been tightened to stop your script from working.

    I noticed that I didn't get any results during an import, and subsequently when visiting CD Universe I get a block warning:


    Almost there!

    CDU Error Code: DS for IP <redacted>

    We're sorry but we have blocked your page request due to possible unusual activity on your computer.

    Please click here to continue to our website.

    If you are getting this message in error, please contact us at:
    Email: abuse@cduniverse.com
    CD Universe
    101 N Plains Ind Rd
    Wallingford, CT 06492
    USA
    Phone: 203-294-1648
    Electronic data feeds containing our product information can be made available to approved partners.
    Specialized pages for real time checking on pricing and availability are also available.
    For more information, contact partners@cduniverse.com.

    So, maybe I hit it with too many requests in one go - fair enough.

    However, I have access to a couple of VPNs, so I can easily change my IP address.

    To test if it's an automatic block here's what I do:
    • I set all files in the importer to be ignored, and restart the importer for good measure to ensure that nothing is pending
    • I connect a VPN and check that my external IP address has actually changed (icanhazip.com - very useful!)
    • I go to the CD Universe website to check I'm not weirdly blocked already - website comes up
    • I release 1 file from the ignored list and restart the importer - no results
    • I revisit the CD Universe website and get the block message above
    So, I suspect that they are now blocking any requests that look like they are not from a web browser - whereas previously it appeared to let things through up to a certain point and then block.

    Not sure if there's actually much you can do - if CD Universe are blocking it outright there's probably not a work-around to continue using them without becoming an "approved partner" - and I can't see that happening! Using another site could be an option, but I know that will need a rewrite and could end up with a similar situation down the line. Just thought I'd post this in case others are having issues, and to let you know.
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,345
    1,824
    Country flag
    • Thread starter
    • Moderator
    • #75
    MovPic allows for adjustment of the user-agent string sent to the site, so I could play with that.

    It could also be that they caught on to the way in which I auto-bypass the 18+ validation screen.

    Haven't imported any new adult media in a while, so I never noticed. Will have to run some tests, to see what I can do.
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,345
    1,824
    Country flag
    • Thread starter
    • Moderator
    • #77
    Hi RoChess,
    Have you had a chance to investigate to see if there's anything that can be done?
    Not in great detail, at first view it seems they have a pretty airtight solution. Have a few more ideas, and if those all fail, will consider looking into a full rewrite with another source. First I want to get my new HTPC build, my old one is on its last legs and I've been sitting on these 'new' parts long enough that it is time, and then I'll be better equiped to test new scenarios as well.
     

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,698
    7,083
    Sydney
    Australia Australia
    Country flag
    You can also use TMDb scraper obviously, but they have a very limited 'adult' collection, which is user-managed so you can always add missing ones.
    I added support at the API level but I did not enable it by default in MovingPictures. The reason for that is I didnt want adult movies being accidentally chosen or seen in list of possible matches.

    I think what might be best is to add an advanced setting which allows the inclusion of TMDb Adult titles in search / match results.
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,345
    1,824
    Country flag
    • Thread starter
    • Moderator
    • #80
    You can also use TMDb scraper obviously, but they have a very limited 'adult' collection, which is user-managed so you can always add missing ones.
    I added support at the API level but I did not enable it by default in MovingPictures. The reason for that is I didnt want adult movies being accidentally chosen or seen in list of possible matches.

    I think what might be best is to add an advanced setting which allows the inclusion of TMDb Adult titles in search / match results.
    Yeah, pretty much every blockbuster movie has an xXx variant on the title, so that would complicate matches. But if a correct IMDb tt-ID entry is added to the filename/NFO and imported as such then the TMDb backdrop scraper does work. Maybe I just need to work on adding a fuzzy IMDb match system to it, the same way I did for Spanish FilmInfo scraper. Just have to work out the conversion between CDUniverse title format and the way IMDb does it.

    At the same time there are other things I need to look at. Just haven't had much time, and getting sick with flu again didn't help this weekend.
     

    Users Who Are Viewing This Thread (Users: 0, Guests: 1)

    OP Similar threads Forum Replies Date
    toricred Using TMM, How to Stop Importer From Going to The Movie Database for Data General 1
    DHosein 1.25 Upgrade and movie covers/backdrops General Support 9
    doskabouter [Help Me!] Playback of iTunes Movie Trailers Developer Discussions 0
    D [solved] Movie artwork download stopped working Submit: Bug Reports 9
    G Movie release date in guide Electronic Program Guide 2
    J [fixed] Import issue with alternate movie title matching the name of another movie Submit: Bug Reports 7
    tano_grasa Movie cover not showing when starting a movie aMPed 0
    R [further infos missing] some movies freezing evry seconds Submit: Bug Reports 9
    J [fixed] Importer confused on specific movies with the same title, different years and different NFO's Submit: Bug Reports 3
    B How to Search MP2 for Movies only in 7 day EPG that comes from the Antenna General 3
    A MP2.2.2, understanding movies/videos/tv and WTV recordings. General 16
    N [solved] Movies starten nicht Allgemeines Support- und Diskussionsforum 16
    J How to save recorded Movies and TV Series to sub-folders? Newcomers Forum 1
    S Movie navigation issues in MP2 General 8
    D media portal 2.2 have movies shown on genre not name General 6
    D Ripped movies not importing unless IMDB code is included in folder name General 4
    M 4k movies General 2
    D [solved] Active Movie window General 2
    fredco [solved] Large import not working Moving Pictures 15
    M [solved] MP2 Not discovering all movies Newcomers Forum 2
    M 4k usergroup rule for them movies only watch/edit Videos 1
    S [solved] Movie SRT subtitle didn't display on client 2.2 Archive 18
    S [solved] Some movie didn't appear General 21
    F [Solved] Movies View with Thumbnail Layou General Support 4
    Terra Crash when searching movies, reproducable Moving Pictures 2
    Similar threads

























    Top Bottom