ziggo.nl grabber - running into webepg limitations (1 Viewer)

benjerry

MP Donator
  • Premium Supporter
  • September 26, 2007
    167
    10
    Home Country
    Netherlands Netherlands
    Hi,

    I'm busy with developing grabbers for the Netherlands at moment. I'm preparing a tvgids.upc.nl grabber(147 channels) and an updated TVGids.nl grabber(104 channels).

    I also would like to add a grabber for another big dutch cable company: www.ziggo.nl

    I've done research for their online tvguide (https://www.ziggo.nl/spotlight/programmagids) and by using a packet sniffer I've managed to get the hidden structure.

    It's like follows:

    <Site url="https://www.ziggo.nl/proxy?pipe=programmagids&url=%2Fprogrammagids%2F[YYYY]-[MM]-[DD]T[HOUR]00%3Ftip%3Dtrue%26favoriteProgram%3D%26favoriteGenre%3D%26channelId%3D[ID]&" post="" external="false" encoding="" />

    start="<ol class="programs">" end="</ol>" tags="ALS"

    template:
    <li class="program" style="left: <#SPIX>px; width: <#EPIX>px;" title="<#PTITLE>" zp-program="<#LINKCODE>">
    <a>
    <span><#START></span>
    <span><#TITLE></span>
    </a>
    </li>

    <Sublink url="https://www.ziggo.nl/proxy?pipe=programmagids&url=%2Fprogrammagids%2FprogrammaPopup%2F[LINKCODE]&" post="" external="false" encoding="" />

    However, I have 2 problems with this structure.

    1) [HOUR] - does not exists in WebEPG. The site url gives program information for 4 hours only. So for 1 day it needs to be grabed 6 times with some counter: 00, 04, 08, 12, 16, 20

    2) Sublink url - it's not possible to generate a sublink url using tags/fields from program info.

    gr,
    Gijs


    edit: test run without hour/sublink

    Code:
    2010-06-09 19:44:46.004648 [WebEPGImporter(7)]: WebEPG: Getting Channel ID: ned3.nl
    2010-06-09 19:44:46.005648 [WebEPGImporter(7)]:         [1 of 36]
    2010-06-09 19:44:46.007648 [WebEPGImporter(7)]: WebEPG: Opening NL\4-www_ziggo_nl.xml
    2010-06-09 19:44:46.008648 [WebEPGImporter(7)]: WebEPG: Version: 2.0
    2010-06-09 19:44:46.009648 [WebEPGImporter(7)]: WebEPG: TimeZone, Local: W. Europe Standard Time
    2010-06-09 19:44:46.009648 [WebEPGImporter(7)]: WebEPG: TimeZone, Site : W. Europe Standard Time
    2010-06-09 19:44:46.036650 [WebEPGImporter(7)]: WebEPG: ChannelId: ned3.nl
    2010-06-09 19:44:46.039650 [WebEPGImporter(7)]: WebEPG: Grab Start 7:44 PM 6/9/2010
    2010-06-09 19:44:46.248662 [WebEPGImporter(7)]: WebEPG: Reading https://www.ziggo.nl/proxy?pipe=programmagids&url=%2Fprogrammagids%2F2010-06-09T1200%3Ftip%3Dtrue%26favoriteProgram%3D%26favoriteGenre%3D%26channelId%3D8003& POST: 
    2010-06-09 19:44:47.534736 [WebEPGImporter(7)]: HTMLPage: GetInternal encoding: 
    2010-06-09 19:44:47.535736 [WebEPGImporter(7)]: WebEPG: Listing Count 14
    2010-06-09 19:44:47.551737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609115000 - De drukke wereld van Richard Scarry (VPRO)
    2010-06-09 19:44:47.552737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.552737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609121000 - Olivia (VPRO)
    2010-06-09 19:44:47.553737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.553737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609122000 - Angelina Ballerina (EO)
    2010-06-09 19:44:47.554737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.555737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609123000 - De wereld van K3 (TROS)
    2010-06-09 19:44:47.555737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.556737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609130000 - De wereld van K3 (TROS)
    2010-06-09 19:44:47.556737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.557737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609132500 - Mijn liefste vriendjes Teigetje en Poeh
    2010-06-09 19:44:47.558737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.560737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609135000 - Het lelijke eendje en ik (KRO)
    2010-06-09 19:44:47.560737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.561737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609141500 - De Smurfen (VARA)
    2010-06-09 19:44:47.562737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.562737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609142500 - De Smurfen (VARA)
    2010-06-09 19:44:47.563737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.563737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609143900 - Z@pp
    2010-06-09 19:44:47.564737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.565737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609144000 - Bernard (KRO)
    2010-06-09 19:44:47.565737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.566737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609144500 - De 3 vrienden en Jerry (NCRV)
    2010-06-09 19:44:47.566737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.567737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609151000 - NOS Jeugdjournaal Verkiezingsuitzending
    2010-06-09 19:44:47.567737 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.568737 [WebEPGImporter(7)]: WebEPG: Guide, Program Info: 20100609160500 - De BZT show (NCRV)
    2010-06-09 19:44:47.569738 [WebEPGImporter(7)]: WebEPG: Program starts in the past, ignoring it.
    2010-06-09 19:44:47.569738 [WebEPGImporter(7)]: WebEPG: Program Count (0), Listing Count (14), Discard Count (14)
     

    benjerry

    MP Donator
  • Premium Supporter
  • September 26, 2007
    167
    10
    Home Country
    Netherlands Netherlands
    2) Sublink url - it's not possible to generate a sublink url using tags/fields from program info.

    <Sublinks>
    <Sublink search="" template="">
    <Link url="" post="" external="" encoding="" />
    </Sublink>
    /Sublinks>

    Possible part of sollution would be:

    1) search="" (can be omitted), so no match finding in program data
    2) allways replace tags in link url from program data (I guess near line 197 in WebParser.cs)
    3) need also to implement processing custom fields from template. they would not used for output but can be used for manipulation like here in link url or in replace actions (for instance you have 2 description fields and can add them together in an action by replacing tags in the valuestring).

    I shall see if I can setup a developer environment here.
    I've programmed in many languages, but my skills are a bit old. C# is not among them, but have done some C/C++.
     

    benjerry

    MP Donator
  • Premium Supporter
  • September 26, 2007
    167
    10
    Home Country
    Netherlands Netherlands
    When you do have something working, you can post a patch in Submit: code patches (MediaPortal/TV-Server/etc.) - MediaPortal Forum
    Bear in mind that due to feature-freeze the patch won't be included in 1.1.0 but it can be in 1.2.0. I you have any questions, or need help don't hesitate to ask.

    I'll see what I can do. :)

    So far, I've succeeded to get a legal Visual Studio 2008 installed and did a succesfull compile. I've done some debugging of WebEPG to get more familiar with the code.

    I'm still busy with some grabbers and it occurres to me that some behaviour of the z-tag is strange. I think I will take a look to see what's happening.
     

    Users who are viewing this thread

    Top Bottom