XmlTV: Parse the <new /> tag to obtain air date when <date> tag is missing?

Discussion in 'TV-Server Plugins' started by ChaosMageX, October 23, 2015.

  1. ChaosMageX

    ChaosMageX Portal Member

    Joined:
    October 23, 2015
    Messages:
    10
    Likes Received:
    0
    Gender:
    Male
    Ratings:
    +0 / 0
    Home Country:
    United States of America United States of America
    I would like support for parsing the <new /> tag in <programme> blocks in XmlTV EPG data in order to set the original air date meta-data for a program when that program's <programme> block does not have a <date> tag.



    I believe this could be easily achieved by simply adding the following code to the switch block that parses XmlTV tags in the TVEngine.XMLTVImport.Import() function, around lines 430 to 490 in XMLTVImport.cs

    case "new":
    if (nodeDate == null && nodeStart != null && nodeStart.Length > 7) nodeDate = nodeStart.Substring(0, 8);
    else xmlProg.Skip();
    break;


    I haven't tried testing this code myself, but in theory it should work. In the case that the <programme> block contains both a <date> tag and a <new /> tag, the <date> tag would override the <new /> or vice-versa, depending on which comes first.

    In my experience looking over XmlTV files obtained by mc2xml, the <new /> tag always seems to be the very last tag in the <programme> block whenever the program is new, and when it is there, the <date> tag is missing, and hence the original air date of many programs is often missing from the EPG data, a critical piece of data needed for setting automated recordings of new episodes of TV shows, and being able to visually determine whether an episode is new when manually scheduling recordings.
     
    Last edited: October 24, 2015
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. mm1352000
    • Team MediaPortal

    mm1352000 Development Group

    Joined:
    September 1, 2008
    Messages:
    21,527
    Likes Received:
    4,738
    Ratings:
    +8,205 / 17
    Home Country:
    New Zealand New Zealand
    Hello and welcome! :)

    I'm just a little confused by your suggestion, because the new tag doesn't have an associated date. The XMLTV DTD (format standard) says:
    So, what we'd expect to find in an XMLTV file is "<new />", "<new></new>"... or nothing at all.

    Further, as the standard says, the "new" tag/attribute is only meant to be applied to the first episode of the first season. It doesn't really mean "new" in the sense that you seem to be referring to.

    Regards,
    mm
     
  4. ChaosMageX

    ChaosMageX Portal Member

    Joined:
    October 23, 2015
    Messages:
    10
    Likes Received:
    0
    Gender:
    Male
    Ratings:
    +0 / 0
    Home Country:
    United States of America United States of America
    Well, what I meant by "parsing" the <new /> tag is that if that tag is detected, it is used to set the program's original air date from the start date/time data that has already been obtained from the attributes of the <programme> block. I apologize if using the term parsing was confusing because the <new /> tag doesn't contain any data to parse, but I couldn't think of a better term to use.

    Well, unfortunately, I fear this is not the case, at least as far as mc2xml is concerned, because the XmlTV file that it outputs almost always uses the <new /> tag in place of a <date> tag in the <programme> block whenever the program's air date matches its start date. That's why I want this code implemented, in order to compensate for that.

    Besides, there is really no harm that I can see in implementing this code. If, in other cases, the <new /> tag is only used for the first episode of the first season of a new TV show on the first time it airs, this code wouldn't make a difference because the program's <date> would already match the date portion of the "start" attribute of its <programme> tag block anyway, and again it's likely that the <new /> tag would come after the <date> tag and wouldn't override it anyway. Aside from that, yes, in those cases the code would become superfluous, but I doubt it would have any significant impact on the processing time for parsing XmlTV data, considering it's just another case in a switch statement, which I believe is O(1) in C# no matter how many cases it has. The only time it might affect the processing time is in cases where the <new /> tag is used more liberally than specified in the DTD and thus many programs would have both a <date> tag and a <new /> tag.
     
    Last edited: October 24, 2015
  5. mm1352000
    • Team MediaPortal

    mm1352000 Development Group

    Joined:
    September 1, 2008
    Messages:
    21,527
    Likes Received:
    4,738
    Ratings:
    +8,205 / 17
    Home Country:
    New Zealand New Zealand
    Ahh, okay - I see what you mean now. Thanks. :)

    Yes, I understand. If mc2xml have chosen to ignore the standard that is unfortunate. We'd prefer not to do that.

    I understand what you're saying. Your assumption is that the start date/time is expected to match the original air date when the new tag is present. Unfortunately I can't agree with that assumption.

    Consider the extremely common situation of overseas syndication. For example, a show is produced by NBC for showing in the US. A broadcaster from New Zealand also pays for the rights to show the show in New Zealand. Very often the show will be aired days, weeks, months or even years later in New Zealand compared to the US. The correct original air date for the program in US and New Zealand EPG data is the date it was aired in the US, not the date it was aired in New Zealand. If we applied the change you're suggesting, the original air date would be detected wrongly for the New Zealand EPG data.


    Putting all this aside, if I've understood correctly you probably are looking for a way to have MediaPortal only record new episodes. If that's the case, I suggest you try either:
    1. Enabling the "check episode..." option as described here. Note that will only prevent recording if you've already recorded the episode. That may not be exactly what you want.
    2. Use the mc2xml "-a * new" option to mark "new" programs with an asterisk in the title, and then create your schedules for only those programs. The result will be that MediaPortal will record programs named [for example] "Modern Family*" but not "Modern Family".
     
  6. ChaosMageX

    ChaosMageX Portal Member

    Joined:
    October 23, 2015
    Messages:
    10
    Likes Received:
    0
    Gender:
    Male
    Ratings:
    +0 / 0
    Home Country:
    United States of America United States of America
    Ah, I understand. I did not take that into account because I did not know that could be the case.

    Thank you for these suggestions. I will most likely try out number one, since my main goal is to prevent automatically recording an episode that I have already recorded.

    HOWEVER, I was looking over the code for the TVEngine.XMLTVImport.Import() function, and I now have a new question:

    Why isn't the <date> tag actually parsed into a valid DateTime and used in the TvDatabase.Program class constructor?

    Instead I see that the <date> string is disregarded entirely even when it is a valid string and System.Data.SqlTypes.SqlDateTime.MinValue.Value is used as the argument for originalAirDate constructor parameter instead. Why can't you implement code that parses it in a similar way the start date/time and end date/time are parsed?

    Something like this could work:

    long originalAirDate = datetolong(System.Data.SqlTypes.SqlDateTime.MinValue.Value);
    if (nodeDate != null && nodeDate.Length >= 8)
    {
    originalAirDate = 1000000 * Int64.Parse(node.Substring(0, 8)); //20040331
    }
    originalAirDate = CorrectIllegalDateTime(originalAirDate);


    Then you could send longtodate(originalAirDate) as the argument for the originalAirDate parameter in the TvDatabase.Program class constructor, unless there's a reason not to assume that the <date> tag's data will be in the format YYYYMMDD. Is there a reason not to assume that? Are there examples of the <date> string being in a different format?
     
  7. mm1352000
    • Team MediaPortal

    mm1352000 Development Group

    Joined:
    September 1, 2008
    Messages:
    21,527
    Likes Received:
    4,738
    Ratings:
    +8,205 / 17
    Home Country:
    New Zealand New Zealand
    Thanks for your understanding. :)

    Sure. Don't hesitate to ask if you have any further questions about this. :)

    Good question! Seems to be an oversight... :oops::whistle:
    We'll have to have a look at that.
     
  8. ChaosMageX

    ChaosMageX Portal Member

    Joined:
    October 23, 2015
    Messages:
    10
    Likes Received:
    0
    Gender:
    Male
    Ratings:
    +0 / 0
    Home Country:
    United States of America United States of America
    While we're on the subject of oversight in the code, I also noticed that the switch statement in TVEngine.XMLTVImport.Import() is parsing the <previously-shown> tag in the <programme> block, but not doing anything with it.

    Also, could the option be added to concatenate multiple <category> tags instead of just parsing the first and dropping the rest? You could add the option for the user to set a concatenation string (which the code would set the value of the catSep string to before the main parsing loop begins), and then change the code in the switch statement to something like this:

    case "category":
    if (nodeCategory == null) nodeCategory = xmlProg.ReadString();
    else if (catSep != null) nodeCategory = string.Concat(nodeCategory, catSep, xmlProg.ReadString());
    else xmlProg.Skip();
    break;


    I ask because sometimes the XmlTv file can divide the category up between multiple category tags, and the first category tag isn't enough information to properly translate the category in the Kodi MediaPortal PVR add-on for genre coloration and other things that use the genre information. For instance, in the XmlTv files retrieved by mc2xml, I sometimes get category tags like this:

    <category lang="en">Other</category>
    <category lang="en">comedy</category>


    or like this:

    <category lang="en">Other</category>
    <category lang="en">action/adventure</category>


    You get the idea. I apologize if this is starting to veer off from the original topic of this thread, and if necessary I can make this request in a separate thread. I just thought of this request while I was continuing to look over the code for the XmlTv parsing plugin for MediaPortal, as well as the latest XmlTv file that mc2xml downloaded to see examples of what the code parses.
     
    Last edited: October 24, 2015
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!