Receiving some Forbidden responses when doing imports (1 Viewer)

[Dj_Moren]

Hello,

When I do some imports (for example 10 films at the same time) I receive a lot of Forbidden responses from Filmaffinity. I suspect that this is because Filmaffinity has some kind of "abusing" protection when it receives a lot of request from the same IP, but I'm not sure. For example, I put in the importer 10 films, and 3 films fail with Forbidden response. I close the importer, reopen it and the 3 films are imported successfully.

Is there any way to force a delay between each request?

Thank you!

 

[RoChess]

Yes, advanced settings on "threads".

Default is 5 threads if memory serves me, so that means 5 movies are imported simultaniously (well that's the idea anyway).

Changing this to 1, will do them all nice in sequential ordering, which is ideal for websites that monitor for IP hammering.

 

[Dj_Moren]

Hmmm I have changed that setting to 1 but it is responding the same... I think that I need more time between each request. I'm going to do a test script to measure this and post back the results

 

[Dj_Moren]

OK, after trying to force a 403 (Forbidden) response with Filmaffinity, I have achieved it with a little script. Sometimes it gives me a 403 response after 100 continued requests and other times after just 5 requests... So, I have put an sleep of 1 second between each request and all the responses (200 continued requests) have been of type 200 (OK).
My idea is to do this in any way in Moving pictures, but i think that this is not implemented...
I have downloaded the source code and tried to launch in development enviroment to see how much could be for me to do this change, but I don't know how to launch the project. It gives me an error saying that i can not start directly a project with a result of class library (translated from spanish). Can you help me?

 

[RoChess]

As for helping u get your development environment up and running, check the web.archive.org results for the old moving-pictures.tv domain as it had a development section on there.

Regarding the code to slow down the import, simply look at the existing code that slows down an import when the mediafile in question is locked (still being copied/etc). Expanding on that code should make it easy, but simply adding a Thread.Sleep(1000); in the import processing function will do the trick nicely.

Another way is to adjust the XML based scraper-script, and come up with a work-around (there are options to delay retrieves, or you can use proxies, etc).