DUGG TV guide grabber - Updated [02.06.06] (1 Viewer)

[angusmann]

Yahoo scraping is VERY difficult. Without giving away the details, just believe me....as good as impossible.

 

[gwood]

not sure if you have looked into your tv.com.au (remove the space)

there screens look ok and you can get daily emails of shows, maybe you could process the data at the email level into xml??

 

[angusmann]

That web site makes extensive use of cookies and form submissions to deliver the correct pages. Coding these with my limited knowledge is difficult, but probably with some research I could figure it out.

I plan to devote a bit more time to the MSN issue and if it goes nowhere I'll look at alternatives like this.

 

[gwood]

these aussies look interesting

http://minnie.tuhs.org/twiki/bin/view/Guidedata/WebHome

To solve this deficiency, WarrenToomey and KevinLentin have written a Perl-based system called Wktivoguide to create Australian TV guide data for the TiVo.

 

[zombiepig]

That web site makes extensive use of cookies and form submissions to deliver the correct pages. Coding these with my limited knowledge is difficult, but probably with some research I could figure it out.

I plan to devote a bit more time to the MSN issue and if it goes nowhere I'll look at alternatives like this.

actually your tv.com.au blocks access after about 10-15 sucessive page hits. you'd have to grab REALLY slowly to be able to get a complete list from them. i managed to cobble together a grabber for their site once but my ip got blocked really quickly

 

[James]

retrieving from Yahoo might be easier/safer?

Yahoo is currently supported by WebEPG. However, they limit the number of requests per IP. So if you try to grab too much, then you will be blocked for about 24 hours.

 

[Nightmare77]

is it perhaps not an option for tugg to grab say 14 days of Fox and FTA tv and then the results being uploaded to team media portal site for everyone then to grab the xml file from that site so msn only get hits once every 14 days?

 

[petejo]

null pointer got it going

Angusmann,

Seems NullPointer of Webscheduler fame has got his guide working ?
http://forums.dvbowners.com/index.php?showtopic=6287

Maybe you two should get together and get smart about this data mining and sort it out so there is minimal twawling of the source sites ?

 

[Callifo]

That solution seems to work ok (used to use it) but it only produces ~8 word descriptions. In the same amount of time, DUGG produces full descriptions.

 

[petejo]

That solution seems to work ok (used to use it) but it only produces ~8 word descriptions. In the same amount of time, DUGG produces full descriptions.

At the moment it produces 8 words more than DUGG then !!