TVSeries No Longer Connects to TVDB... (1 Viewer)

[SkydeOption]

Hey,-

I'm a huge TVSeries fan. This plug-in is what really makes MediaPortal

Since about a couple of weeks I'm having troubles getting TVSeries to connect to the TVDB site.

The configuration opens up by logging two failures to connect to http://www.thetvdb.com/interfaces/GetMirrors.php and GetLanguages.php respectively. The return code listed is "(403) Forbidden". Executing an Import run results in loads of similar failed attempts and return codes.

Opening the URL http://www.thetvdb.com/interfaces/GetMirrors.php in a browser is successful.

I have noted that www.thetvdb.com resolves to 74.220.207.129. While opening http://www.thetvdb.com by name in a web browser is successful, using http://74.220.207.129 shows a HostMonster error page ("no site configured on this address") and opening http://74.220.207.129/interfaces/GetMirrors.php gives a 404 error.

Changing the "Main Mirror" string (I've tried http://tvdb.zsori.com/interfaces/) in the configuration gives identical results.

All possible firewall issues have been ruled out.

Any assistance or advise would be greatly appreciated.

Thanks and Rgds.

(Running current version: 1.2.9.34660, but the latest SVN version 1.2.9.30423 (not sure why the number is lower?), shows same behaviour)

 

[Inker]

My guess is a prosy issue, do you use any proxies that require authentication?

 

[SkydeOption]

Don't use proxy...

I have a direct internet connection (should have mentioned that, sorry)

Thanks for the suggestion, though. Appreciated

I have been spending the morning in the company of my good friend "HTTP Analyzer V2" and he's given some insights.

First it became clear that the issue is not limited to TVSeries. Exactly the same happens for IMDB lookup in Movie Database, for Weather lookup and for "Now Playing" lookups (none of which I use very often).

Secondly it became clear that the 403 error was accompanied by a little text reading:

Access Denied
Your system was configured to deny access to this URL.Because this url was and improper/obscene website or your package is localnet
For assistance, contact your network support team.

This text - it appears (after a quick google) is courtesy of the MICT (Ministry of Information and Communication Technology) which governs the InterWebs in the country where i reside.

Clearly there is nothing improper or obscene about TVDB, IMDB, LastFM or the weatherchannel, and this is confirmed by the fact that all the requests that are denied to MediaPortals various parts, are all served correctly if issuing them through a webbrowser (IE or FF was tested). MediaPortal is NOT allowed to talk to these sites, but a web browser IS allowed, meaning it cannot be the hostnames that are causing the problem, it MUST be something else

Looking at either of the various request shows a major difference between what MP and for example FF sends. Example:

FireFox Request:

GET /interfaces/GetLanguages.php HTTP/1.1
Host: www.thetvdb.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

MediaPortal (TVSeries) Request:

GET /interfaces/GetLanguages.php HTTP/1.1
Host: www.thetvdb.com
Connection: Keep-Alive

Obviously the governments packet filters feels something is wrong,- or rather that something is missing,- from the MP request and my first buck in on the User-Agent string....

So,- how do I go about enhancing the headers (adding a User-Agent string would probably do) of MediaPortal's HTTP request ?

Am I screwed ?

Any help is greatly appreciated.

Thanks and Rgds.

 

[Patric]

I think you'll have to modify the ZsoriParser class and extend its request creation by the additional headers. Probably this could point you in the right direction: http://msdn2.microsoft.com/en-us/library/system.net.webrequest.headers(VS.85).aspx
Something like

request.Headers.Add("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11");

or

request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11"

could work...

Greets

 

[Inker]

Thats really weird....are you saying your government filters every internet connection? From googling this seems to be Thailand.....its exactly why the net needs to remain free.......but enough about that!

We shouldn't have to fake the user agent string, are u even sure its the user agent thats the problem?

You can try changing it urself if you feel confortable doing so, else I can provide a couple of test builds for you to narrow the problem down. Just not today as I'm not feeling too well ;-) Make sure you report back if you find out more.

 

[SkydeOption]

This is where it stands...

Hi Guys,-

I been wrecking my brain over this and experimenting back and forth and I can't get past this.

The requests are all served correctly if the headers are "fullblown".... If the headers are minimal (as MP provides them) the requests are blocked.

Using HTTPAnalyzers request builder, I can clone a failed request from MP, add a User-Agent string (I initially used the FF string) and the request goes through and is served beautifully. Now,- here's the kicker: the actual contents of the User-Agent string doesn't matter, as long as it is there. (See attachments)

It all keeps shouting at me: "the missing User-Agent is your problem".

Whether this is all a part of a government conspiracy directed against MediaPortal users, I cannot say My guess is that this is either an attempt to force HTTP traffic to be real "Web Browsing", or simply due to badly configured packet filters. The fact that the content of the User-Agent string seems irrelevant, sort of suggest the latter.

As for me modifying the MP/TVSeries sourcecode.... I simply don't have the talent

If you can find the time to help me out, I would be extremely grateful.

The responses to my plea for help, have been excellent. Another reason to love MediaPortal.

Thanks and Rgds.

 

[Inker]

Please test the build I have pmed you about. Provided it does solve the problem for you I can commit the changes to SVN so everyone benefits (althought only you seem affected) and more importantly the changes get carried on along with other changes.

Please report back.

 

[SkydeOption]

Hi Inker,-

Thank You, Thank You, Thank You.. And may I add: U Da Man !

This was just what the doctor ordered. Your efforts are deeply appreciated. (And I'm so happy that I wasn't leading you down a completely wrong path )

The added User-Agent header is working its magic and TVSeries is talking to TVDB again.

There is however one but... (isn't there always ) I have attached a couple of screenshots which show the requests for JPGs (banners) are still missing the User-Agent header (and so they fail with the infamous 403 and accompanying tiresome message). I'm assuming that the TVSeries plug-in requests JPGs from a separate section of code, that would need to be modified similarly to what you already did.

I hate to bother you unnecessarily, but...

Thanks for the first rate, primo, platinum support.

Rgds,-

 

[Inker]

Hi,

You are correct, banners are handled differently. I have added the user-agent string their too. Changes are all in latest SVN build. Let me know if there is something that's still blocked.

Inker