DUGG TV guide grabber - Updated [02.06.06]

angusmann · 2005-11-24, 07:02

Hi all. I have written a guide grabber for Australia.

It can grab all the pay TV stations and also FTA for more than 20 different Australian regions.

A simple editable initialization file allows you to set your region, timezone and paytv choices, as well as the number of days grabbed etc.

It can grab encrypted data.

It outputs in XML and an easily read text-file is created also. It grabs the program start time and, duration and calculates the stop time, coping properly with programs that span midnight etc. It grabs the channel, program title, subtitle,long description, rating and genre.

I wrote the prog. to format the XML for my needs, but if anybody can point to a more "universal" format for this kind of data I could easily modify the XML output.

The XML is "cleaned up" to remove or replace foriegn and unprintable characters. For example "&" is converted to "and" because "&" in an XML document can cause problems.

Find the program here : http://dugg.ueuo.com

If you are interested, or can suggest an XML format, PM me.

Taipan · 2005-11-24, 07:05

Hi angusmann,

This sounds interesting - where are you grabbing the TV Guide data from?

angusmann · 2005-11-24, 07:07

I will only say in public that is is the best source.

Dont want to get in trouble.

PM me and I can email more info.

James · 2005-11-24, 08:32

Hi Angusmann,

What language did you write your grabber in ? I'm guessing Java?

Cheers,

/James

Taipan · 2005-11-25, 21:28

Quote:

Originally Posted by angusmann

PM me and I can email more info.

OK, I PM'd you, but haven't yet received a reply....

PJ · 2005-11-27, 01:55

If you want to use & in an XML doc can't you just used the escaped character &? Maybe I'm getting this confused with xsl instead.

Dordo · 2005-11-28, 05:00

I have used this and I must say its very good, quick easy to use and grabs quite detailed information. There are still a few problems to be ironed out and a few features which would be nice to see added but considering how new this is I must say on the whole its very impressive.

angusmann · 2005-11-28, 11:00

Thanks to all the testers. Even over the course of 1 day I have improved the program using all your suggestions and I am very pleased with the way it's working now.

guytpetj · 2005-12-02, 05:41

Angusman,

I've PM'ed you several days ago, but no reply

regrards,
Peter

winsys11 · 2005-12-03, 22:48

just thought y'all might like to read this.......
Who knows if it's legally enforcable, but I for one don't have the $$$ to contest it with Foxtel.

quote :
"The Listing Information is not to be, in any way, downloaded or used in conjunction with any device, such as customer equipment or a set top box, which permits manipulation of the program listings, or which permits scheduled program recording."

Of course thay don't want this - it's competition for their IQ product. By explicitly stating this, I think Foxtel are wide open to investigaton by the ACCC for anti-competitive behaviour.

Anyway, here's the amusing little warning in full :

FOXTEL is the copyright owner in respect of:

â€¢ the listing information for each of the channels it owns and operates (â€˜Channel Listingsâ€™);
â€¢ the overall compilation of listing information for all the channels it provides, and the compilation of movies called â€œA-Z MOVIESâ€ (â€˜Compilationsâ€™),
(together, â€˜Listing Informationâ€™)

Copyright information

Copyright in the Channel Listings and the Compilations remain at all times the property of FOXTEL and no unauthorised person must in any way challenge FOXTELâ€™s claim to copyright in the Channel Listings and the Compilations.

The Listing Information is not to be, in any way, downloaded or used in conjunction with any device, such as customer equipment or a set top box, which permits manipulation of the program listings, or which permits scheduled program recording.

A person (unless otherwise authorised) may only use the Listing Information for the purpose of viewing online for the person's personal use, and must not sell, supply or otherwise provide the listing information to any third parties.

The user of the Listing Information agrees not to:

â€¢ adapt or alter all or any part of the Listing Information;
â€¢ copy, publish, exploit, split, record, re-encode, redirect, relay, alter, retransmit, resell, redistribute or otherwise offer or supply all or any part of the Listing Information in any way, including via the internet or any other means of communication; or
â€¢ exhibit, or permit or facilitate the viewing of the all or any part of the Listing Information in a public viewing area.

2005-11-24, 07:02	#1 (permalink)
angusmann Portal Member Join Date: Oct 2005 Posts: 88 Thanks: 1 Thanked 3 Times in 1 Post	DUGG TV guide grabber - Updated [02.06.06] Hi all. I have written a guide grabber for Australia. It can grab all the pay TV stations and also FTA for more than 20 different Australian regions. A simple editable initialization file allows you to set your region, timezone and paytv choices, as well as the number of days grabbed etc. It can grab encrypted data. It outputs in XML and an easily read text-file is created also. It grabs the program start time and, duration and calculates the stop time, coping properly with programs that span midnight etc. It grabs the channel, program title, subtitle,long description, rating and genre. I wrote the prog. to format the XML for my needs, but if anybody can point to a more "universal" format for this kind of data I could easily modify the XML output. The XML is "cleaned up" to remove or replace foriegn and unprintable characters. For example "&" is converted to "and" because "&" in an XML document can cause problems. Find the program here : http://dugg.ueuo.com If you are interested, or can suggest an XML format, PM me.

2005-12-02, 05:41	#9 (permalink)
guytpetj MP Donator Join Date: Mar 2005 Location: Brisbane, Australia Age: 44 Posts: 264 Thanks: 0 Thanked 3 Times in 3 Posts Country: My System	Angusman, I've PM'ed you several days ago, but no reply regrards, Peter __________________ Peter HTCP_1 (Media Server) CM Stacker 830 ; XP-Pro ; PD 3.4 ; 2GB ram ; 2970GB HDD ; 6 Tuners HTCP_2 TT Mozart SX ; XP-Home ; Core2duo ; 2GB ram ; 400GB HDD ; 2 Tuners ; MCE Remote & KB HTCP_3 CM-260 ; XP-Home ; P4 3.0 ; 512MB ram ; 250GB HDD ; MCE Remote [HTPC_4 CM-260 ; XP-Pro ; Core2duo ; 2GB ram ; 330GB HDD ; 1 Tuner ; MCE Remote

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
WebEPG issues	ronald2oo2	WebEPG	28	2007-04-29 18:55
(original thread) US/Canadian/Mexico EPG Zap2it Plugin Released	bradsjm	zap2it (discontinued)	513	2007-02-21 15:07
Any idea why WebEPG is putting the wrong date in tvguide.xml	koach	MediaPortal 1 Talk	3	2006-11-07 00:36
Leadtek TV2000 RM Gives Errors - Can't do anything	slmckenzie	General Support	1	2006-09-15 07:35
A problem creating spanish TDT grabber for WEBEPG	pegial	WebEPG	7	2006-06-04 13:14

2005-11-24, 07:05	#2 (permalink)
Taipan Portal Tester Join Date: Feb 2005 Location: Melbourne Posts: 1,929 Thanks: 4 Thanked 15 Times in 15 Posts Country: My System	Hi angusmann, This sounds interesting - where are you grabbing the TV Guide data from?

2005-11-24, 07:07	#3 (permalink)
angusmann Portal Member Join Date: Oct 2005 Posts: 88 Thanks: 1 Thanked 3 Times in 1 Post	I will only say in public that is is the best source. Dont want to get in trouble. PM me and I can email more info.

2005-11-24, 08:32	#4 (permalink)
James Portal Developer Join Date: May 2005 Location: Switzerland Posts: 1,348 Thanks: 4 Thanked 55 Times in 34 Posts	Hi Angusmann, What language did you write your grabber in ? I'm guessing Java? Cheers, /James

2005-11-27, 01:55	#6 (permalink)
PJ Portal Member Join Date: Sep 2004 Location: Melbourne, Australia Posts: 15 Thanks: 1 Thanked 0 Times in 0 Posts Country:	If you want to use & in an XML doc can't you just used the escaped character &? Maybe I'm getting this confused with xsl instead.

2005-11-28, 05:00	#7 (permalink)
Dordo Portal Member Join Date: Dec 2004 Posts: 81 Thanks: 1 Thanked 0 Times in 0 Posts	I have used this and I must say its very good, quick easy to use and grabs quite detailed information. There are still a few problems to be ironed out and a few features which would be nice to see added but considering how new this is I must say on the whole its very impressive.

2005-11-28, 11:00	#8 (permalink)
angusmann Portal Member Join Date: Oct 2005 Posts: 88 Thanks: 1 Thanked 3 Times in 1 Post	Thanks to all the testers. Even over the course of 1 day I have improved the program using all your suggestions and I am very pleased with the way it's working now.

2005-12-03, 22:48	#10 (permalink)
winsys11 Portal Member Join Date: Sep 2005 Location: Melbourne Posts: 44 Thanks: 0 Thanked 0 Times in 0 Posts	just thought y'all might like to read this....... Who knows if it's legally enforcable, but I for one don't have the $$$ to contest it with Foxtel. quote : "The Listing Information is not to be, in any way, downloaded or used in conjunction with any device, such as customer equipment or a set top box, which permits manipulation of the program listings, or which permits scheduled program recording." Of course thay don't want this - it's competition for their IQ product. By explicitly stating this, I think Foxtel are wide open to investigaton by the ACCC for anti-competitive behaviour. Anyway, here's the amusing little warning in full : FOXTEL is the copyright owner in respect of: â€¢ the listing information for each of the channels it owns and operates (â€˜Channel Listingsâ€™); â€¢ the overall compilation of listing information for all the channels it provides, and the compilation of movies called â€œA-Z MOVIESâ€ (â€˜Compilationsâ€™), (together, â€˜Listing Informationâ€™) Copyright information Copyright in the Channel Listings and the Compilations remain at all times the property of FOXTEL and no unauthorised person must in any way challenge FOXTELâ€™s claim to copyright in the Channel Listings and the Compilations. The Listing Information is not to be, in any way, downloaded or used in conjunction with any device, such as customer equipment or a set top box, which permits manipulation of the program listings, or which permits scheduled program recording. A person (unless otherwise authorised) may only use the Listing Information for the purpose of viewing online for the person's personal use, and must not sell, supply or otherwise provide the listing information to any third parties. The user of the Listing Information agrees not to: â€¢ adapt or alter all or any part of the Listing Information; â€¢ copy, publish, exploit, split, record, re-encode, redirect, relay, alter, retransmit, resell, redistribute or otherwise offer or supply all or any part of the Listing Information in any way, including via the internet or any other means of communication; or â€¢ exhibit, or permit or facilitate the viewing of the all or any part of the Listing Information in a public viewing area.