home
products
contribute
download
documentation
forum
Home
Forums
New posts
Search forums
What's new
New posts
All posts
Latest activity
Members
Registered members
Current visitors
Donate
Log in
Register
What's new
Search
Search
Search titles only
By:
New posts
Search forums
Search titles only
By:
Menu
Log in
Register
Navigation
Install the app
Install
More options
Contact us
Close Menu
Forums
MediaPortal 1
MediaPortal 1 Plugins
Popular Plugins
Moving Pictures
Adult movie scraper (details+covers) -- v2.0.7
Contact us
RSS
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Reply to thread
Message
<blockquote data-quote="Matt Kirby" data-source="post: 1018990" data-attributes="member: 95689"><p>RoChess,</p><p> </p><p>Firstly, thanks for you work on this script and the IMDB+ one - great work!</p><p> </p><p>However, I think that the scraping detection at CD Universe has been tightened to stop your script from working.</p><p> </p><p>I noticed that I didn't get any results during an import, and subsequently when visiting CD Universe I get a block warning:</p><p> </p><p></p><p></p><p> </p><p> </p><p>So, maybe I hit it with too many requests in one go - fair enough.</p><p> </p><p>However, I have access to a couple of VPNs, so I can easily change my IP address. </p><p> </p><p>To test if it's an automatic block here's what I do:</p><ul> <li data-xf-list-type="ul">I set all files in the importer to be ignored, and restart the importer for good measure to ensure that nothing is pending</li> <li data-xf-list-type="ul">I connect a VPN and check that my external IP address has actually changed (icanhazip.com - very useful!)</li> <li data-xf-list-type="ul">I go to the CD Universe website to check I'm not weirdly blocked already - website comes up</li> <li data-xf-list-type="ul">I release 1 file from the ignored list and restart the importer - no results</li> <li data-xf-list-type="ul">I revisit the CD Universe website and get the block message above</li> </ul><p>So, I suspect that they are now blocking any requests that look like they are not from a web browser - whereas previously it appeared to let things through up to a certain point and then block.</p><p> </p><p>Not sure if there's actually much you can do - if CD Universe are blocking it outright there's probably not a work-around to continue using them without becoming an "approved partner" - and I can't see that happening! Using another site could be an option, but I know that will need a rewrite and could end up with a similar situation down the line. Just thought I'd post this in case others are having issues, and to let you know.</p></blockquote><p></p>
[QUOTE="Matt Kirby, post: 1018990, member: 95689"] RoChess, Firstly, thanks for you work on this script and the IMDB+ one - great work! However, I think that the scraping detection at CD Universe has been tightened to stop your script from working. I noticed that I didn't get any results during an import, and subsequently when visiting CD Universe I get a block warning: [SIZE=6][B] [/B][/SIZE] So, maybe I hit it with too many requests in one go - fair enough. However, I have access to a couple of VPNs, so I can easily change my IP address. To test if it's an automatic block here's what I do: [LIST] [*]I set all files in the importer to be ignored, and restart the importer for good measure to ensure that nothing is pending [*]I connect a VPN and check that my external IP address has actually changed (icanhazip.com - very useful!) [*]I go to the CD Universe website to check I'm not weirdly blocked already - website comes up [*]I release 1 file from the ignored list and restart the importer - no results [*]I revisit the CD Universe website and get the block message above [/LIST] So, I suspect that they are now blocking any requests that look like they are not from a web browser - whereas previously it appeared to let things through up to a certain point and then block. Not sure if there's actually much you can do - if CD Universe are blocking it outright there's probably not a work-around to continue using them without becoming an "approved partner" - and I can't see that happening! Using another site could be an option, but I know that will need a rewrite and could end up with a similar situation down the line. Just thought I'd post this in case others are having issues, and to let you know. [/QUOTE]
Insert quotes…
Verification
Post reply
Forums
MediaPortal 1
MediaPortal 1 Plugins
Popular Plugins
Moving Pictures
Adult movie scraper (details+covers) -- v2.0.7
Contact us
RSS
Top
Bottom