WebEPG foxtel grabbers... (2 Viewers)

[squidina]

You can easily check what URL TVHarvest is using via the config file, but yes by default it uses ninemsn.

I have TVHarvest working on both FTA and Optus Cable. It creates two different xml files and then combines them for uploading into MP. It seems to work pretty well.

One thing I have noticed is that it fails every now and then when trying to retrieve Saturday and/or Sunday's guide. It seems this is a ninemsn problem as the pages just don't exist straight away.

Apart from that it works beautifully.

 

[winsys11]

Hi James,

I agree the foxtel.tv pages are "encrypted", but yesterday I discovered separate web sites for each Pay channel, that have guides, and are plain text.

I guess they would be suitable for web scrapping?

look at:
http://www.animalplanet.com.au/_ont...APAN&schedule_date=20051106&language_code=ENG

or


http://www.discoverychannel.com.au/...e_code=ENG&target_page=/_listings/index.shtml

for examples.

I hope this can help us aussies get guides for foxtel....

Mark

 

[James]

Hi winsys11,

Yeah, I was very surpised when I saw what foxtel did to the text on their tv guide site. The funny thing is the detailed pages are note encripted, so if you took everylink you can get the pain text. However, I'm sure they would change that pretty quick.

Yes the sites which you posted are fine.

I don't have time right now to do all the configuration files, but am happy to help anyone who wants to try. I wrote the program so that the site config files could the developed by anyone (with a little bit of HTML knowledge).

Also I have heard that all the pay TV channels are available from yahoo. They limit access to the descriptions, but these sites don't seem to have descriptions anyway. However, it is also good to have multiple sources.

Cheers,

/James

 

[SiLenTYL]

ok..ive found a working foxtel/foxtel digital/optus/austar tv guide at au.tv.yahoo.com

seems like all the show times are identical even in different states...ill see if i have a chance to write it up for u guys..then lemme know how it goes

 

[winsys11]

we need to be careful hitting the foxtel.tv site, they use Red Sherriff to keep track of page access (for ratings, I believe)

On another note, do other Aussies think we should take this up with the ACCC? As a Foxtel customer, don't I have the right to the EPG? ( I pay for it....)

We all know the reason the pages are "encrypted" is to specifically stop what we are trying to do, to increase the relative worth of their IQ product.

Doesn't that smack of anti-competetive behaviour, ie a monopoly?

Just a thought.....

 

[winsys11]

Just another thought...

If we got a web grabber going, and we ALL grabbed the pages, that would maybe amount to some noticable traffic.

What if..... one person grabbed all the pages, and the XML file was made available to all?

Foxtel would never know it was happening, and would therefore be less likely to change things.

Mark

 

[Callifo]

Your not actually allowed to grab EPG data thats why the P2P EPG idea is basically on hold.

 

[BKCH]

ugg! Back to the drawing board - it seems ninemsn have now encrypted their pages - tvharvest no longer working...

Anybody got any ideas? (Hard to live without an epg)

Thread at XPMediaCenter.com.au: http://www.xpmediacentre.com.au/community/showthread.php?t=4291

 

[Dordo]

From what I understand ninemsn uses a simple ROT13 encryption which should be extremely easy to get around http://en.wikipedia.org/wiki/ROT13

Also did I hear P2P EPG as a possibility? This is a great idea works very well both for MP users and the websites. Why exactly is this unfeasable, Im not sure why we aren't allowed to grab the EPG data, hell even if this was written as a very much distanced and unofficial plugin to take the focus off MP?

 

[James]

Hi Dordo,

Rather then hard coding the ninemsn encription. It is far better to use the javascript provided by the site to decode the text. This way if they change the javascript, the grabber will not stop working.

/James