XmlTV: Parse the <new /> tag to obtain air date when <date> tag is missing? (1 Viewer)

ChaosMageX

Portal Member
October 23, 2015
10
0
United States of America United States of America
Country flag
I would like support for parsing the <new /> tag in <programme> blocks in XmlTV EPG data in order to set the original air date meta-data for a program when that program's <programme> block does not have a <date> tag.

I believe this could be easily achieved by simply adding the following code to the switch block that parses XmlTV tags in the TVEngine.XMLTVImport.Import() function, around lines 430 to 490 in XMLTVImport.cs

case "new":
if (nodeDate == null && nodeStart != null && nodeStart.Length > 7) nodeDate = nodeStart.Substring(0, 8);
else xmlProg.Skip();
break;


I haven't tried testing this code myself, but in theory it should work. In the case that the <programme> block contains both a <date> tag and a <new /> tag, the <date> tag would override the <new /> or vice-versa, depending on which comes first.

In my experience looking over XmlTV files obtained by mc2xml, the <new /> tag always seems to be the very last tag in the <programme> block whenever the program is new, and when it is there, the <date> tag is missing, and hence the original air date of many programs is often missing from the EPG data, a critical piece of data needed for setting automated recordings of new episodes of TV shows, and being able to visually determine whether an episode is new when manually scheduling recordings.
 
Last edited:

mm1352000

Development Group
  • Team MediaPortal
  • September 1, 2008
    21,571
    8,219
    New Zealand New Zealand
    Country flag
    Hello and welcome! :)

    I'm just a little confused by your suggestion, because the new tag doesn't have an associated date. The XMLTV DTD (format standard) says:
    <!-- New. This is the first screened programme from a new show that
    has never been shown on television before - if not worldwide then at
    least never before in this country. After the first episode or
    programme has been shown, subsequent ones are no longer 'new'.
    Similarly the second series of an established programme is not 'new'.

    Note that this does not mean 'new season' or 'new episode' of an
    existing show. You can express part of that using the episode-num
    stuff.
    -->
    <!ELEMENT new EMPTY>
    So, what we'd expect to find in an XMLTV file is "<new />", "<new></new>"... or nothing at all.

    Further, as the standard says, the "new" tag/attribute is only meant to be applied to the first episode of the first season. It doesn't really mean "new" in the sense that you seem to be referring to.

    Regards,
    mm
     

    ChaosMageX

    Portal Member
    October 23, 2015
    10
    0
    United States of America United States of America
    Country flag
    I'm just a little confused by your suggestion, because the new tag doesn't have an associated date.
    Well, what I meant by "parsing" the <new /> tag is that if that tag is detected, it is used to set the program's original air date from the start date/time data that has already been obtained from the attributes of the <programme> block. I apologize if using the term parsing was confusing because the <new /> tag doesn't contain any data to parse, but I couldn't think of a better term to use.

    Further, as the standard says, the "new" tag/attribute is only meant to be applied to the first episode of the first season. It doesn't really mean "new" in the sense that you seem to be referring to.
    Well, unfortunately, I fear this is not the case, at least as far as mc2xml is concerned, because the XmlTV file that it outputs almost always uses the <new /> tag in place of a <date> tag in the <programme> block whenever the program's air date matches its start date. That's why I want this code implemented, in order to compensate for that.

    Besides, there is really no harm that I can see in implementing this code. If, in other cases, the <new /> tag is only used for the first episode of the first season of a new TV show on the first time it airs, this code wouldn't make a difference because the program's <date> would already match the date portion of the "start" attribute of its <programme> tag block anyway, and again it's likely that the <new /> tag would come after the <date> tag and wouldn't override it anyway. Aside from that, yes, in those cases the code would become superfluous, but I doubt it would have any significant impact on the processing time for parsing XmlTV data, considering it's just another case in a switch statement, which I believe is O(1) in C# no matter how many cases it has. The only time it might affect the processing time is in cases where the <new /> tag is used more liberally than specified in the DTD and thus many programs would have both a <date> tag and a <new /> tag.
     
    Last edited:

    mm1352000

    Development Group
  • Team MediaPortal
  • September 1, 2008
    21,571
    8,219
    New Zealand New Zealand
    Country flag
    Well, what I meant...
    Ahh, okay - I see what you mean now. Thanks. :)

    Well, unfortunately...
    Yes, I understand. If mc2xml have chosen to ignore the standard that is unfortunate. We'd prefer not to do that.

    Besides, there is really no harm that I can see in implementing this code...
    I understand what you're saying. Your assumption is that the start date/time is expected to match the original air date when the new tag is present. Unfortunately I can't agree with that assumption.

    Consider the extremely common situation of overseas syndication. For example, a show is produced by NBC for showing in the US. A broadcaster from New Zealand also pays for the rights to show the show in New Zealand. Very often the show will be aired days, weeks, months or even years later in New Zealand compared to the US. The correct original air date for the program in US and New Zealand EPG data is the date it was aired in the US, not the date it was aired in New Zealand. If we applied the change you're suggesting, the original air date would be detected wrongly for the New Zealand EPG data.


    Putting all this aside, if I've understood correctly you probably are looking for a way to have MediaPortal only record new episodes. If that's the case, I suggest you try either:
    1. Enabling the "check episode..." option as described here. Note that will only prevent recording if you've already recorded the episode. That may not be exactly what you want.
    2. Use the mc2xml "-a * new" option to mark "new" programs with an asterisk in the title, and then create your schedules for only those programs. The result will be that MediaPortal will record programs named [for example] "Modern Family*" but not "Modern Family".
     

    ChaosMageX

    Portal Member
    October 23, 2015
    10
    0
    United States of America United States of America
    Country flag
    I understand what you're saying. Your assumption is that the start date/time is expected to match the original air date when the new tag is present. Unfortunately I can't agree with that assumption.

    Consider the extremely common situation of overseas syndication. For example, a show is produced by NBC for showing in the US. A broadcaster from New Zealand also pays for the rights to show the show in New Zealand. Very often the show will be aired days, weeks, months or even years later in New Zealand compared to the US. The correct original air date for the program in US and New Zealand EPG data is the date it was aired in the US, not the date it was aired in New Zealand. If we applied the change you're suggesting, the original air date would be detected wrongly for the New Zealand EPG data.
    Ah, I understand. I did not take that into account because I did not know that could be the case.

    Putting all this aside, if I've understood correctly you probably are looking for a way to have MediaPortal only record new episodes. If that's the case, I suggest you try either:
    Thank you for these suggestions. I will most likely try out number one, since my main goal is to prevent automatically recording an episode that I have already recorded.

    HOWEVER, I was looking over the code for the TVEngine.XMLTVImport.Import() function, and I now have a new question:

    Why isn't the <date> tag actually parsed into a valid DateTime and used in the TvDatabase.Program class constructor?

    Instead I see that the <date> string is disregarded entirely even when it is a valid string and System.Data.SqlTypes.SqlDateTime.MinValue.Value is used as the argument for originalAirDate constructor parameter instead. Why can't you implement code that parses it in a similar way the start date/time and end date/time are parsed?

    Something like this could work:

    long originalAirDate = datetolong(System.Data.SqlTypes.SqlDateTime.MinValue.Value);
    if (nodeDate != null && nodeDate.Length >= 8)
    {
    originalAirDate = 1000000 * Int64.Parse(node.Substring(0, 8)); //20040331
    }
    originalAirDate = CorrectIllegalDateTime(originalAirDate);


    Then you could send longtodate(originalAirDate) as the argument for the originalAirDate parameter in the TvDatabase.Program class constructor, unless there's a reason not to assume that the <date> tag's data will be in the format YYYYMMDD. Is there a reason not to assume that? Are there examples of the <date> string being in a different format?
     

    mm1352000

    Development Group
  • Team MediaPortal
  • September 1, 2008
    21,571
    8,219
    New Zealand New Zealand
    Country flag
    Ah, I understand. I did not take that into account because I did not know that could be the case.
    Thanks for your understanding. :)

    Thank you for these suggestions. I will most likely try out number one, since my main goal is to prevent automatically recording an episode that I have already recorded.
    Sure. Don't hesitate to ask if you have any further questions about this. :)

    HOWEVER, I was looking over the code for the TVEngine.XMLTVImport.Import() function, and I now have a new question:

    Why isn't the <date> tag actually parsed into a valid DateTime and used in the TvDatabase.Program class constructor?
    Good question! Seems to be an oversight... :oops::whistle:
    We'll have to have a look at that.
     

    ChaosMageX

    Portal Member
    October 23, 2015
    10
    0
    United States of America United States of America
    Country flag
    Good question! Seems to be an oversight... :oops::whistle:
    We'll have to have a look at that.
    While we're on the subject of oversight in the code, I also noticed that the switch statement in TVEngine.XMLTVImport.Import() is parsing the <previously-shown> tag in the <programme> block, but not doing anything with it.

    Also, could the option be added to concatenate multiple <category> tags instead of just parsing the first and dropping the rest? You could add the option for the user to set a concatenation string (which the code would set the value of the catSep string to before the main parsing loop begins), and then change the code in the switch statement to something like this:

    case "category":
    if (nodeCategory == null) nodeCategory = xmlProg.ReadString();
    else if (catSep != null) nodeCategory = string.Concat(nodeCategory, catSep, xmlProg.ReadString());
    else xmlProg.Skip();
    break;


    I ask because sometimes the XmlTv file can divide the category up between multiple category tags, and the first category tag isn't enough information to properly translate the category in the Kodi MediaPortal PVR add-on for genre coloration and other things that use the genre information. For instance, in the XmlTv files retrieved by mc2xml, I sometimes get category tags like this:

    <category lang="en">Other</category>
    <category lang="en">comedy</category>


    or like this:

    <category lang="en">Other</category>
    <category lang="en">action/adventure</category>


    You get the idea. I apologize if this is starting to veer off from the original topic of this thread, and if necessary I can make this request in a separate thread. I just thought of this request while I was continuing to look over the code for the XmlTv parsing plugin for MediaPortal, as well as the latest XmlTv file that mc2xml downloaded to see examples of what the code parses.
     
    Last edited:

    Users Who Are Viewing This Thread (Users: 0, Guests: 1)

    OP Similar threads Forum Replies Date
    framug No color in TVGuide when XMLTV without DVB EPG MediaPortal 1 Talk 59
    B xmltv - cannot import tvguide.xml australia Installation, configuration support 0
    geotwticher TV Guide EPG Grabber for Australia 2020 RIP freepg.tv General 4
    P Timeshifting in XMLTV General 2
    C Can the XmlTv Plugin handle 600+ channels? General 3
    JJDoherty 1.22.0 XMLTV not raising the following OnImportEpgPrograms event... 1.22 0
    lacanox XMLTV not writing the tv schedule Electronic Program Guide 3
    A [solved] XmlTv not importing programs General 8
    dcapslock TV Server xmltv IWakeupHandler Bug Television (MyTV frontend and TV-Server) 6
    T No data available using xmltv and mc2xml xmlTV 2
    W XMLTVImport duplicate item error xmlTV 5
    J Schedules Direct - XmlTV Plugin xmlTV 6
    LeeKelly Error parsing action result document and error starttimeshift General 2
    U Parsing invalid episode names My TVSeries 10
    O Can't browse network share on new build Newcomers Forum 4
    morpheus_xx [WiP] "What's New" plugin for MP2 upgrades MediaPortal 2 20
    F Problems with new install of MP2 Newcomers Forum 3
    morpheus_xx [WiP] OnlineVideos: New browser based playback support using WebDriver, new Amazon siteutil MediaPortal 2 14
    P [solved] MP2 new install, 2 x hdhomerun tuners, client on same PC unable to view TV & recordings Newcomers Forum 8
    J Help with new grabber file. WebEPG 1
    W Not finding/matching new series - thetvdb issue? My TVSeries 3
    caveman [solved] Cannot download new version My TVSeries 5
    wizard123 New Skin Template ? MediaPortal 2 8
    J New computer with win10 and how will i set Mediaportal up ? Newcomers Forum 55
    R [solved] Files moved to new location - Easy way to update database? My TVSeries 7
    Similar threads

























    Top Bottom