new US imdb_com grabber file, adds SubTitle support (4 Viewers)

[RonD]

The updated version of the US www_imdb_com.xml grabber file is included with MediaPortal 1.7.0 and later and you do NOT need to download a new grabber file.

You still need to customize the Channel IDs for you local city and set your zip code as described MediaPortal wiki WebEPG setup guide. See the following link.

http://wiki.team-mediaportal.com/1_...ides/2_TV_Setup/4_TV_Guide_Setup_(EPG)/WebEPG

If you are using an earlier version of MediaPortal, you still need to download the grabber file in this message.
*******************************
Attached is a new www_imdb_com.xml grabber file based on the current (Jan 2008) US imdb_com file, With this grabber you do not need to set a "tv_prov". This grabber works for my stations and I see StartTime, Title, SubTitle, Description in the MediaPortal TV Guide. An example of the tvguide.xml output is shown below. The www_imdb_com grabber does not have support for Genres.

<programme start="20120525233000 -0700" channel="KTVU-HD-KTVU-HD">
<title>How I Met Your Mother</title>
<sub-title>The Goat</sub-title>
<desc>When Barney breaks the ``bro code,'' he struggles with the consequences of his actions.</desc>
</programme>

The newest new www_imdb_com_Nov2012.xml grabber file fixes a timezone and daylight savings bug.

I have 2 wiki guides with help for setting up US EPG.

US EPG setup using WebEPG and imdb.com, use this if you are trying to setup WebEPG with this grabber file
http://wiki.team-mediaportal.com/User:rond/US_EPG_setup_using_WebEPG_and_imdb.com

US EPG setup using XmlTv and mc2xml, use this if you want to use mc2xml and xmltv.
http://wiki.team-mediaportal.com/User:rond/US_EPG_Setup_using_XmlTv_and_mc2xml

The "imdb_com_readme.txt" is the original text instructions for setting up the original May2012 version of the grabber file.

If you are aleady use the May2012 grabber file, copy your file to a safe place, download the new Nov2012 grabber file and replace your current grabber file. You can copy/paste your TV Channel information from your saved file to the new grabber, and replace ZIPCODE with your local zipcode. You also need to start the TV Server Configuration program, redo the channel mapping, and save the new mappings. You should double check that you are using the correct grabber file.

For debug you can do the following to flush the old EPG info from the TV Guide database to make sure you are seeing the new grabber data.

"TV Server Configuration -> manual control" press "Refresh DVB EPG" to flush the current TV Guide EPG info before doing "Grab now!"

change info:

Nov 4, 2012: uploaded new www_imdb_com_Nov2012.xml grabber file with fixes for timezone and daylight savings bug.

July 11 2012: I made a wiki page with setup info for WebEPG and the new grabber file, link posted below.
http://wiki.team-mediaportal.com/User:rond/US_EPG_setup_using_WebEPG_and_imdb.com

May 31 2012: uploaded a new imdb_com_readme.txt file with editorial changes to help answer some questions from this thread.

 

[TLD]

So looking at the readme the "siteid" is now the whole ulr as in (http://www.imdb.com/tvgrid/2012-05-24/KTVUDT/) or just the (KTVUDT) in your case?

Also you have (<Info language="en" availableDays="14" timezone="Eastern Standard Time" version="2.0" />) I am in the Pacific time zone so I would change that to (<Info language="en" availableDays="14" timezone="Pacific Standard Time" version="2.0" />),

We are in daylight savings time now is there going to be a time offset somehow?

Thanks RonD

 

[RonD]

I'm also in the Pacific Timezone and the WebEPG grabber gets the correct times. I've never looked at the WebEPG code but the tv.log files show the following and I guess the code does the following.

2012-05-25 09:24:23.203125 [WebEPGImporter(7)]: WebEPG: TimeZone, Local: Pacific Standard Time
2012-05-25 09:24:23.203125 [WebEPGImporter(7)]: WebEPG: TimeZone, Site : Eastern Standard Time

If you don't set a zipcode in the imdb_com URL, imdb sends TV guide times for a default Eastern Timezone. WebEPG uses the SiteTimezone=Eastern, LocalTimezone=Pacific and does the needed timezone correction. If the imdb_com URL has a zipcode (Pacific), imdb sends TV guide times for the Pacific Timezone and the grabber could/did use timezone="".

The siteId="KTVUDT" should just have the imdb_com channel info (KTVUDT) and nothing else.

The (http://www.imdb.com/tvgrid/2012-05-24/KTVUDT/ was intended as a sample of what imdb sends for a 1 day single channel listing and an example for the siteId="KTVUDT". Maybe I should change the wording in the readme file to avoid confusion.

 

[TLD]

After i reread the instructions a couple times i got it about the (The (http://www.imdb.com/tvgrid/2012-05-24/KTVUDT/ was intended as a sample of what imdb sends for a 1 day single channel listing and an example for the siteId="KTVUDT".)

You don.t want to change the time zone to (<Info language="en" availableDays="14" timezone="Pacific Standard Time" version="2.0" />) as it will put everything a hour off (<Info language="en" availableDays="14" timezone="Eastern Standard Time" version="2.0" />) works fine in the Pacific Time zone.

Looks like you got it right as it works fine for me with subtitles in the EPG.

Thanks a lot and great work RonD

 

[RonD]

TLD,

Glad this update works for you. Have you checked some of your known problem channels/shows for correct SubTitle/Episode info. This grabber works for me but this always works better for the guy messing with the script than for others.

 

[TLD]

Yes I've checked all the issues i had and everything works great and will be sticking with WebEPG now. This should help a lot of people from the USA out, I know i've helped a lot of folks that were having problems with the EPG in the USA.

Thanks again RonD great work.
Ted

EDIT: I'm using Comcast Cable.

 

[2.0]

I just used this new grabber and I can say that it does work.

I used Notepad to edit the file. It was easy to copy all my channel info from the old file, then I had Notepad remove the zip & tv provider part from each line. I had my new grabber file set up in just two minutes. I went to WebEPG setup and mapped all my channels and all went well. I saved it and hit Grab Now and watched it work. With the old grabber file, it took four or five hours to get 14 days of schedule for 50 channels. This new grabber did it in only twelve minutes! I checked my EPG in MP and all the correct info was there. The times were all correct. I checked the tv.log and saw that it used the Eastern time zone which is where I am.

Excellent work, RonD. We just need more people to test this.

 

[RonD]

2.0, glad this works for you. I think the reason the old grabber took so long was the time spent doing sublinks and generated a lot more web traffic. With the (find start="Genre:" and change "Genre:" to "Genres:") problem the sublink didn't do anything useful.

If you look in your tv.log files you should have a line like the following. I'm curious how much traffic the old vs new grabbers generate. My data is for 14 channels and 14 days, takes around 2.5 minutes and 8.5 MB of traffic.

HTTP Statistics: Site www.imdb.com : Pages 168 : Bytes 8480174 : Total Time 00:02:32.0520000 : Av. Rate 55.77154 KBps

You might need to look at the tv.bak log file to see the old data.

 

[2.0]

New log:

HTTP Statistics: Site www.imdb.com : Pages 686 : Bytes 33302229 : Total Time 00:09:14.7647780 : Av. Rate 60.02946 KBps

Old log:

HTTP Statistics: Site www.imdb.com : Pages 6355 : Bytes 552047707 : Total Time 03:28:16.7987485 : Av. Rate 44.17513 KBps

Yeah, quite a difference.

So with this new grabber file, each user still has to customize it for his or her own channel list. But if I understand correctly how it works, in theory it is possible to make one enormous master grabber file with every channel listed on the IMDB website. Since this new file doesn't require zip codes or providers, and works regardless of one's time zone, one giant list of all channels should work for everyone in the US. As long as your channels are mapped correctly it would work, right? Putting together a list like that would be quite a job, though.

 

[TLD]

Problem is that if your on cable like me (without a cable Box) when i scan channels all i get is 6 digit numbers and have to preview each channel to see what it is then rename them to the proper channel.