new US imdb_com grabber file, adds SubTitle support (1 Viewer)

RonD

Test Group
  • Team MediaPortal
  • December 20, 2011
    911
    278
    SillyValley CA
    Home Country
    United States of America United States of America
    So each or your EPG runs downloaded 552 MB of data but really wasted 519 MB since the "Genre grabber" did not find anything. How often did you run the EPG download?

    Don't think the idea of giant master channel list would work to well. The channel names I find with a OTA channel scan are close but never really match the IMDB channel names. With mc2xml/xmltv its a pain to map my 30-40 "All Channels" with the ~100 channels from mc2xml.

    To make WebEPG a bit more user friendly, the code would need changes to do something like

    1. let the user enter a zip code, send this to IMDB and get a list of providers
    2. let the user select a provider, send this to IMDB to get a list of channels
    3. give the user a xmltv-like "auto close match" and with a mapping option to fix channels mapping problems.

    I'm not a C# programmer so I can't estimate the effort make these changes. One of these days I'll download the source and figure out what compiler SW to download/install to play with the code.
     

    2.0

    MP Donator
  • Premium Supporter
  • January 18, 2012
    106
    19
    New York
    Home Country
    United States of America United States of America
    I ran that download every Sunday morning while I exercise. This was also the time I had my HTPC perform its scheduled maintenance functions (Disk Cleanup, TRIM, etc.) There's nothing good on TV on Sunday mornings anyway so I knew I'd never have to record anything while it was doing all this.

    Your idea of a WebEPG feature that would check the IMDB website for channel matches sounds good, but I see a potential problem. If IMDB ever decides to change its website in even the slightest, it could render this feature of WebEPG useless. Since WebEPG is built-in to MediaPortal, I think it can't be updated without installing a new version of MediaPortal. And let's not forget that WebEPG would need to work with all of the different TV listings websites in other countries as well. Instead, how about a separate program/plugin that does what you said? This program can generate a customized grabber script based on what it finds on the IMDB website after you input your zip code and tv provider, then let you map each of your own channels to what it finds. If the IMDB website ever changes its format, this program or plugin could be easily updated to accomodate the changes.

    As for a "master" grabber file with all channels, I think it would work well if it used standardized channel names. The xml file has both a channel id and a site id for each channel. The site id must of course match whatever is on IMDB. But the channel id can be something standard that everyone would recognize. Different TV providers may call the channels by their own odd names and numbers, but each channel still has an "official" name that it uses nationally (HBO, Antenna TV, CBS, Telemundo, etc.) The xml file can use these official names as the channel id. Then as long as the user names each of his or her channels by these same names, auto mapping should work. Even if it doesn't, manual mapping should take care of the few odd misses.
     

    TLD

    Portal Pro
    October 26, 2007
    954
    387
    Rainy Washington
    Home Country
    United States of America United States of America
    Different TV providers may call the channels by their own odd names and numbers, but each channel still has an "official" name that it uses nationally (HBO, Antenna TV, CBS, Telemundo, etc.)

    While this may be true in some areas or OTA it is not true here on cable even when i had a Digital cable box as now for instance the channels you mentioned like HBO on the guide at IMDB.com here we have HBO 1, HBO 2 or HBO 3, Antenna TV is KZJODT3, Telemundo is TeleP so i can imagine that there are many different call letters/numbers depending on where you are located in the IMDB.com guide depending on which network is broadcasting the particular station.

    Instead, how about a separate program/plugin that does what you said? This program can generate a customized grabber script based on what it finds on the IMDB website after you input your zip code and tv provider, then let you map each of your own channels to what it finds. If the IMDB website ever changes its format, this program or plugin could be easily updated to accomodate the changes.
    This seems like a good solution, having a plugin that could parse the local official channel call letters/numbers (site IDs) from IMDB.com would make it easy for anyone to get WebEPG working for them.

    The instructions that RonD has written up are not that hard to follow and most people could manage it in my opinion.
     

    HeatherBecker

    New Member
    May 25, 2012
    2
    0
    Rossville, GA
    Home Country
    United States of America United States of America
    Like TLD, I'm on cable without a box, and when the channels were scanned, I just got a numeric value. I went into the MediaPortal TV Server Configuration (TV Channels --> right-click a channel --> Edit channel) and gave each channel a name.

    So, now I'm not clear on what value should actually be used for the channel id in the imdb file. If someone could point me to the exact location in the Configuration app to locate the correct value for channel id, or even the right table + column to use from the MySql database, I would be greatly appreciative!
     

    RonD

    Test Group
  • Team MediaPortal
  • December 20, 2011
    911
    278
    SillyValley CA
    Home Country
    United States of America United States of America
    I'm not 100% sure I understand the question since I use OTA. If you goto "WebEPG->TV Mapping", select a channel group and press "import" this will load your channels. The "EPG Name" is your local name "Channel ID". Your www_imdb_com file needs a list a mappings between your names and IMDB Channel Name

    Channel id="EPG Name" siteId="IMDB Channel Name".

    picture shows my setup after I do the "import"

    You can also goto the "import/export", set the check boxes for "TV Channels" and "TV Groups" and export the info to a xml file to get a list of "Channel ids" in your channel groups.
     

    Attachments

    • WebEPG_TV_Mappings.jpg
      WebEPG_TV_Mappings.jpg
      74.3 KB
    Last edited:

    TLD

    Portal Pro
    October 26, 2007
    954
    387
    Rainy Washington
    Home Country
    United States of America United States of America
    Like TLD, I'm on cable without a box, and when the channels were scanned, I just got a numeric value. I went into the MediaPortal TV Server Configuration (TV Channels --> right-click a channel --> Edit channel) and gave each channel a name.

    So, now I'm not clear on what value should actually be used for the channel id in the imdb file. If someone could point me to the exact location in the Configuration app to locate the correct value for channel id, or even the right table + column to use from the MySql database, I would be greatly appreciative!

    Below is my working "www_imdb_com.xml" file, The "Channel id" is the name i gave to the channels in TV Server.
    The "siteId" is the name that " http://www.imdb.com/tvgrid/2012-05-26/2000/ " shows in the ULR after i put my zip code in and picked my TV Provider: and got to my TV channel line up and then clicked the channel so it was showing what was on that channel for that day. (It's not the same as what is in the Left hand column on the full scheduled)

    The ULR will look something like this: http://www.imdb.com/tvgrid/2012-05-26/DSCP/ this particular one is for Discovery Channel.
    If you look at the "www_imdb_com.xml" file below you can see it's channel "8" on my TV server.


    working"www_imdb_com.xml" file for where i live.
    <?xml version="1.0" encoding="utf-8"?>
    <Grabber>
    <Info language="en" availableDays="14" timezone="Eastern Standard Time" version="2.0" />


    <Channels>
    <Channel id="2" siteId="NWCN" />
    <Channel id="3" siteId="KWPXDT" />
    <Channel id="3 CB" siteId="TBS" />
    <Channel id="4" siteId="KOMO" />
    <Channel id="5" siteId="KING" />
    <Channel id="6" siteId="KONG" />
    <Channel id="7" siteId="KIRO" />
    <Channel id="8" siteId="DSCP" />
    <Channel id="9" siteId="KCTS" />
    <Channel id="10" siteId="KZJO" />
    <Channel id="11" siteId="KSTW" />
    <Channel id="12" siteId="KBTC" />
    <Channel id="13" siteId="KCPQ" />
    <Channel id="14" siteId="KBCB" />
    <Channel id="16" siteId="QVC" />
    <Channel id="17" siteId="HSN" />
    <Channel id="19" siteId="HALLP" />
    <Channel id="20" siteId="KTBW" />
    <Channel id="22" siteId="KPTV" />
    <Channel id="23" siteId="TVW" />
    <Channel id="24" siteId="CSPAN" />
    <Channel id="26" siteId="EDAC026" />
    <Channel id="27" siteId="TELEP" />
    <Channel id="28" siteId="KUNS" />
    <Channel id="29" siteId="CBUT" />
    <Channel id="77" siteId="UWTV" />
    <Channel id="78" siteId="TWC" />
    <Channel id="72 KVOSDT" siteId="KVOSDT" />
    <Channel id="104 KOMODT" siteId="KOMODT" />
    <Channel id="105 KINGDT" siteId="KINGDT" />
    <Channel id="106 KONGDT" siteId="KONGDT" />
    <Channel id="107 KIRODT" siteId="KIRODT" />
    <Channel id="109 KCTSDT" siteId="KCTSDT" />
    <Channel id="110 KZJODT" siteId="KZJODT" />
    <Channel id="111 KSTWDT" siteId="KSTWDT" />
    <Channel id="112 KCTSDT3" siteId="KCTSDT3" />
    <Channel id="113 KCPQDT" siteId="KCPQDT" />
    <Channel id="114 KOMODT2" siteId="KOMODT2" />
    <Channel id="115 KINGDT2" siteId="KINGDT2" />
    <Channel id="116 KZJODT3" siteId="KZJODT3" />
    <Channel id="117 KIRODT2" siteId="KIRODT2" />

    </Channels>

    <Listing type="Html">
    <Site url="http://www.imdb.com/tvgrid/[YYYY]-[MM]-[DD]/[ID]&amp;start_date=[YYYY]-[MM]-[DD]" post="" external="false" encoding="" />
    <Html>
    <Template name="default" start="single_station_container">
    <SectionTemplate tags="TB">
    <TemplateText>
    &lt;tr&gt;&lt;td&gt;&lt;a&gt;&lt;#START&gt;&lt;/a&gt;&lt;/td&gt;&lt;td&gt;
    &lt;b&gt;&lt;a&gt;&lt;#TITLE&gt;&lt;/a&gt;&lt;/b&gt;&lt;i&gt;
    &lt;a&gt;&lt;/a&gt;&lt;/i&gt;&lt;#DESCRIPTION&gt;&lt;/td&gt;&lt;/tr&gt;
    </TemplateText>
    </SectionTemplate>
    </Template>

    <Searches>

    <Search match="(?&lt;=&lt;i&gt;(?!&lt;a))(.*?)(?=&amp;nbsp;&lt;\/i&gt;)" field="#SUBTITLE" remove="true" />
    <Search match="(?&lt;=&lt;i&gt;&lt;a href=&quot;/title/tt\d{7}/&quot;&gt;)(.*?)(?=&lt;\/a&gt;)" field="#SUBTITLE" remove="true" />
    <Search match="&amp;nbsp;-&amp;nbsp;" field="#JUNK" remove="true" />
    <Search match=":&amp;nbsp;" field="#JUNK" remove="true" />

    </Searches>

    </Html>
    </Listing>
    </Grabber>
     

    RonD

    Test Group
  • Team MediaPortal
  • December 20, 2011
    911
    278
    SillyValley CA
    Home Country
    United States of America United States of America
    Charlie,

    I did not investigate adding Season/Episode numbers to the grabber. This version of the grabber uses the per channel 1-day web page(s) that does not include the season/episode info. I added support for Episode Names and use the TV Series pluggin to keep track of episode info.
     

    Charlie Sparks

    New Member
    July 19, 2012
    4
    0
    46
    Morrisville, North Carolina
    Home Country
    United States of America United States of America
    Hi Ron,
    Thanks for the follow-up... First of all, thanks for the grabber. It works great! It must be something trivial l I am missing but I can't get MPTVSeries to grab any TV data other than what is supplied by the grabber to obtain Fanart for the MP TV Series plugin. MPTVSeries appears to require episode and series numbers to obtain data. It ignores the subtitle. It isn't a deal breaker if I can't get fanart but it sure makes the experience nice. :) I did notice the series and episode are actually in the source code for theper channel 1-day web page IMDBTV site. I just don't know enough about how to make that available for the grabber. It follows just under the subtitle... In this example it is &nbsp;(#7.5) after Field Trip. Any thoughts would be greatly appreciated!

    </div>
    <div class="tv_phantom">
    <table id="CBS800PM_info"
    style="left: 53px;display:none"
    class="tv_info"
    cellpadding="0" cellspacing="0">
    <td class="tv_info_inner">
    <span class="tv_info_title">
    <a href="/title/tt0460649/">How I Met Your Mother</a>
    </span>
    <span class="tv_info_text">
    [TV14]
    </span>
    <br/>
    <a href="/title/tt2071800/">"Field Trip"</a>
    &nbsp;(#7.5)
    <br/>
    <br/>
    8:00 PM - 8:30 PM<br>
    User Rating: 7.6/10 (513 votes)
    <br/><br/>
    Marshall decides to step in after seeing his boss's approach to settling with a big company. <br/><br/>
    <span class="tv_info_links">
    <a href="/title/tt0460649/board/">Message Board</a>
    </span>
    </td>
    <td id="8:00 PM+CBSpr" class="rshadow"></td></tr>
    <tr><td id="8:00 PM+CBSpb" class="bshadow"></td>
    <td id="8:00 PM+CBSpx" class="xshadow"></td></tr>
    </table>
    </div>
     

    Users who are viewing this thread

    Top Bottom