TV Guide for Aussies (3 Viewers)

[Taipan]

Hi James,

I tried your WebEPG a few weeks ago and was impressed by how easy it was to use and quick to grab the data. However, I found that only some of the data that was available at yahoo was actually retreived - refer here for the details.

Is it possible to modify WebEPG (with yahoo) so that it retrieves:-

* the full description (where it exists)
* the genre
* the rating (G, PG or M)
* and when a programm overlaps from one day's schedule to the next, it treats it as a continuous program? This may not be an issue for the yahoo data, as it does not appear to break programs at a particular time in the day?

I use ourguide.com, which goes from midnight to midnight - but if a program goes past midnight, it shows up in MediaPortal as 2 separate programs, which makes it tricky to record as one continuous program.

I am happy to make the necessary mods myself, but I really have no idea where to start.....

 

[James]

Hi Taipan,

Description and genre should be supported. However, current I have not put in support for rating.

For yahoo all this extra data is contained on a linked page. So you must enable this option in the config and set a time between which you want linked pages. (0 to 23 is the whole day) The default is currently set to 18 to 23, or prime time. This is to cut down the requests on the sites for information that may not be used / required. I was also thinking of feathure to have a list of strings (like "friends" etc) that could be matched with the title to then grab the linked page.

However, there seems to be something at yahoo the blocks a number of requests for linked pages and block the IP address for a certain time. At first it seemed to be time based if the request came to quickly. So I set a delay of 5seconds between these grabs. This seemed to fix the problem for my test with 1 channel and a few days. However, it seems to have another limit. Maybe set of the total number of pages loaded.

What would be really good. If you can find out want they are doing here. What the limit is. How long the IP is blocked for, etc. If possible any way to get around the limit. I was thinking of testing with different browser string (program, OS, etc). So that it hopefully thinks that different computers behind a NAT firewall with the same IP are accesses the site?

Any information on this would help.

Cheers,

/James

 

[Taipan]

Description and genre should be supported. However, current I have not put in support for rating. For yahoo all this extra data is contained on a linked page. So you must enable this option in the config and set a time between which you want linked pages. (0 to 23 is the whole day)

Ah, I see that now - I didn't realise that was how you got the descriptions downloaded...

However, there seems to be something at yahoo the blocks a number of requests for linked pages and block the IP address for a certain time. What would be really good. If you can find out want they are doing here. What the limit is. How long the IP is blocked for, etc.

OK, I will do some testing to see if I can figure out the limit.

If possible any way to get around the limit. I was thinking of testing with different browser string (program, OS, etc). So that it hopefully thinks that different computers behind a NAT firewall with the same IP are accesses the site?

Mmmm- not sure I know enough about all this web stuff to figure out a way to trick Yahoo....

Any information on this would help.

OK, I will see what I can discover.

 

[Taipan]

Hi James,

From the testing I have done so far, it looks like the limit is 27 description requests within a 24 hour period. After the 27th request has been received, I get the following error (in the error.log file):-

7/09/2005 5:44:21 PM Error retreiving WebPage: http://au.tv.yahoo.com/event.html?ev=12637422 Encoding:iso8859-1 err:The remote server returned an error: (999) Unable to process request at this time -- error 999. stack:   at System.Net.HttpWebRequest.CheckFinalStatus()
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.Net.HttpWebRequest.GetResponse()
   at MediaPortal.Util.HTMLPage.LoadPage(String strURL, Boolean isHTML)

I have asked Yahoo support what the meaning of error 999 is, but they are avoiding the question, so far!

After I get this error, I am locked out until the next day. It looks like the yahoo application is working on the IP address, as I am behind the firewall of my router and when this "999" error occurs, I get it from all of my PCs - not just the one running WebEPG. The only IP address that yahoo would know about is the IP address of my router.

However, I have discovered that au.tv.yahoo.com get their TV schedule data from http://www.yourtv.com.au/guide/. So, maybe it would be easier to get the EPG data directly from yourtv.com.au - as it is quite possible that they don't impose any download limits?

I am happy to make the changes to WebEPG to use yourtv.com.au, but I need some instructions on what changes are required - do you have something like that available for a non-programmer/developer like me?

 

[jazZman]

Taipan,
Have you had any luck, with modifying WebEPG to scrape data from http://www.yourtv.com.au/guide/ ??
While the southern end of the country seems to be adequately represented with this marvellous piece of software , your poorer northern cousins (yes we do have electricity & TV up here in Queensland ) are not.

I understand that catching the latest game of aerial ping pong or the ongoing escapades of Sam Newman are important to the citizens of your fair city. To us uncultured folk of Brisbane, would like to know what is on our local channels :wink:

Yourtv.com.au supports Brisbane listings, so if you have a way to integrate this site and are able to share it with me, I would be grateful

James, If your listening, any input from you would also be cheerfully accepted.

Cheers

Jamie

 

[SiLenTYL]

my new xml's include ourguide.com.au, yahoo.com.au......both with all metro cities

Ive updated for all Australian Metro cities for WebEPG ...tried 0.4pre4 and my new xmls are included, but I only see 1-www.SBS.com.au at the moment....hopefully James will have time to get AU working nicely?

When it works, WebEPG will have functional grabbers for :
1-www.sbs.com.au
2-www.ourguide.com.au
3-www.d1.com.au
4-www.abc.net.au
5-au.tv.yahoo.com

all with all Metro cities

 

[jazZman]

SiLenTYL,
Thankyou, I shall give it a try this evening
I understand from Tiapan's post above that there are some issues with pulling down discriptions with yahoo.com.au so I'll give "ourguide" ago .

thanks again

Jamie

 

[timx]

hey thought i would post in here cause it was aussie tv talk

does anyone have an idea where to get the info for the - hdtv channels sbs and abc?? i have the epg going great (using tvxb) but i dont have info for the hdtv channels

timx

 

[Johnny Utah]

I'm using tvxb to download epg info for foxtel digital, and am running into a few minor problems with timings/channels displaying the wrong info.

Some of the 2 hr delayed channels (eg The LifeStyle Channel +2) display the same data as the original lifestyle channel, program for program - when in fact it should be the same shows, but lagging 2 hours behind the original The LifeStyle Channel. Is there an easy way of adjusting the tvxb.ini file to manually correct for this?

I also have a problem whereby "The LifeStyle Channel" displays program information which is actually that of "The History Channel". Is there any way to easily adjust the output file/the way the info is gathered via tvxb so that these 2 channels can be interchanged?

Finally, is there a reasonable guide on integrating 2 xmltv files created by tvxb into one - i.e integrating my digital free to air channel guide with the foxtel tvguide.xml - ?

Thanks in advance for all of your assistance. I'm amazed at how far MP has come in such a short span of time - thanks to the efforts of this community.

 

[timx]

mate open up your tvguide.xml output file in ur favourite editor and have a look around. its pretty easy to work out whats going on and you can have a go at changing some stuff.

1. play around with the timezone setting. you might need to set up an individual .ini with the timezone -200hrs for those channels. Other than that you can manually go through and adjust each program in the .xml file by two hours.

2. make sure the channels names in your .ini file match up with your channel names in media portal, so you correctly match the tv channels with their programs. also make sure your the channel names in your .ini file match up with their source i.e so your not extracting abc program guide info into a channel named seven. look at htmlstart and htmlend lines in the .ini file. it should give you some idea at what is being extracted for each channel

3. to combine your foxtel with free to air just combine both of your .ini together into a single .ini file (see section 2.3.2 in the installation guide). You could also combine each .xml file into a single file - simple copy and paste.

timx

edit:
check out my edits if you already read this