UK - BBC2 EPG suddenly has several programs daily with 'No information' (1 Viewer)

Anthony Vaughan

MP Donator
  • Premium Supporter
  • June 25, 2015
    566
    292
    Home Country
    United Kingdom United Kingdom
    Hi CyberSimian

    Thank you for that detailed and informative reply. I will try what you suggest; i.e. setting the interval for grabbing when idle of 1440 minutes. I think it makes sense to grab the EPG when nothing else is going on anyway.

    I suppose all of this depends upon what is the definition of idle; is 'idle' when no MP client is active (no streaming going on) and no programs are being recorded? Would you please confirm?

    Tony

    UPDATE: I've just changed the EPG settings to 1440 minutes interval and timeout of 15 when idle and the grabbing started as soon as I hit Ok, despite one of the tuners being used to view a program on one of my client machines. So, a definition of idle would be most useful.
     
    Last edited:

    CyberSimian

    Test Group
  • Team MediaPortal
  • June 10, 2013
    2,849
    1,770
    Southampton
    Home Country
    United Kingdom United Kingdom
    the grabbing started as soon as I hit Ok, despite one of the tuners being used to view a program on one of my client machines.
    That is not expected. :confused: The definition of "idle" is that no tuner is in use. So an MP client watching live TV would prevent the idle-grabber grabbing the EPG. In contrast, an MP client watching recorded TV (or video) does not use a tuner, and so the idle-grabber could run in that case.

    Actually, one of the criticisms that has been made about "TV Server" in the past is that the idle-grabber ought to grab the EPG if any tuner is idle, whereas it currently only grabs the EPG if all tuners are idle. EPG grabbing has lowest priority, so it stops immediately if the tuner is required for live TV or a scheduled recording.

    How did you determine that EPG grabbing was occurring? Also, did you remember to disable the timeshift-grabber?

    -- from CyberSimian in the UK
     

    Anthony Vaughan

    MP Donator
  • Premium Supporter
  • June 25, 2015
    566
    292
    Home Country
    United Kingdom United Kingdom
    Hi CyberSimian

    Yes, I did disable to timeshift-grabber and I recognized that the grabber had been invoked because the channel records, that have a field called lastGrabTime, showed that they had been updated at exactly the time I changed the EPG setting.

    I think I can understand why the TV Service would only operate in idle mode when no tuners are active, because the EPG grabber would be competing for resources with the tuners. So, a pretty guaranteed time that would be idle under all circumstances would be overnight - in most households anyway.

    I have encountered another problem with trying to use the 'Always try to update' option: it updates the state column in the program table that indicates whether a program is scheduled to record. I'm not sure when, or how often, MP realigns the schedule data with the program table, but I need certainty for my system to work because when I rename a program that starts with 'New:', to remove that phrase, that also has been scheduled to record, I synchronize the program names of all scheduled programs with the entries in the schedule table. For example, if we have a newly imported program called 'New: Blindspot', it is renamed to 'Blindspot'. The issue is that when the EPG updates an already imported program called 'Blindspot' back to 'New: Blindspot', the state field is reset to 0 and the schedule name, 'Blindspot'. will not match the reset program name and therefore it will not record. Additionally to this, my app also extracts the series information, like seriesNum and episodeNum, from the programs description field, and places them into the seriesNum and episodeNum fields. Obviously, these changes get lost when the EPG is refreshed.

    For these reasons I am going to go back to using WebEPG because it is more predictable, both in when it runs and which channels is grabs - it only grabs channels that are set as visibleInGuide.

    The only issue I can think of about using WebEPG is that, when the EPG goes wrong, I need to remember to use the 'Refresh DVB EPG'. The problem with this is that I don't know when MP realigns the schedule with the programs to set the state of schedules programs; I need to my app to rename the program names to remove 'New:' before MP tries to set programs as scheduled.

    Do you know how often, or when, 'TVService' synchronizes schedule data with programs? If 'TVService' performs the synchronization very regularly, then I won't have an issue because my app runs hourly and the synchronization should happen before most programs that are scheduled get recorded (mainly in the evening).

    Tony
     

    CyberSimian

    Test Group
  • Team MediaPortal
  • June 10, 2013
    2,849
    1,770
    Southampton
    Home Country
    United Kingdom United Kingdom
    Do you know how often, or when, 'TVService' synchronizes schedule data with programs?
    Part of the difficulty here is that neither of us knows the logic that "TV Server" uses when grabbing the broadcast EPG, or importing an internet EPG, or marking programmes to record. So devising workarounds is a bit hit and miss. Unfortunately, the developer who really knows about all of this is no longer an active member. :(

    As to when programmes in the updated EPG are marked for recording, I have always assumed that "TV Server" does this as part of acquiring the EPG, but this is just a guess. It might be that "TV Server" checks whether a new EPG entry is scheduled to record before inserting it in the database, or it might be that it performs these checks at the conclusion of inserting the new and changed entries in the database. But in either case, if you are using only an internet EPG, you can control when the import occurs, and hence run your EPG-edit tool afterwards. So that seems like a workable solution. :)

    So we end up where you started(!) The gaps that you found in the WebEPG must have been due to errors or omissions in the WebEPG data, and you will need to be aware that this might occur again in the future. If it does occur again, you could perform a manual once-only load of the broadcast EPG in "TV Server Config", and then manually run the EPG-edit tool.

    -- from CyberSimian in the UK
     

    Anthony Vaughan

    MP Donator
  • Premium Supporter
  • June 25, 2015
    566
    292
    Home Country
    United Kingdom United Kingdom
    Hi CyberSimian

    Thanks for that. I'll have a look at the code to see if I can work out what's going on when I get a chance. I'll let you know what I find; might be a few days though.

    Tony
     

    Anthony Vaughan

    MP Donator
  • Premium Supporter
  • June 25, 2015
    566
    292
    Home Country
    United Kingdom United Kingdom
    Hi CyberSimian

    I've done some extensive testing using my test system and did the following:
    • Installed a completely fresh copy of MP 1.21;
    • Used WebEPG to load the EPG;
    • Ran my app to update program names and extract series information, then stopped the app;
    • Then set up a schedule for one of the programs whose name started with 'New:'
    • Ran Reload DVB EPG;
    • About an hour later, to ensure the load had finished, I ran my app to reset the names and series information. At this point, the state for the program record that was scheduled earlier was 0; meaning not now scheduled;
    • Reviewed the same program about a half-hour later and the state had been set to scheduled; i.e. 64, presumably by the TV Service (in its own time).
    So, I have a working method for resetting the EPG and getting all scheduled programs back to their previous state. However, it takes about and hour an hour and a half to ensure that all is restored as it was before. NOTE: it is vital to perform this routine when none of the programs that have been scheduled to record, that started with 'New:', are scheduled to record during this process or MP will fail to record it (because the schedule and program names will not match meaning that the state is not set by the TV Service).

    I have looked the EPGs produced by WebEPG and by the broadcast media and, so far as I can see, they seem to be the same except that the WebEPG sometimes truncates the program name and places the text exceeding the truncation limit at the beginning of the description field with '...' and the end of the program name and at the beginning o the description (to show that the program name is continued in the description field). As it happens my app already handles this behaviour by moving the truncated text in the description field back to the program name field. For example:

    'House of Lords - Medicines and Medic...', with the description starting with '... al Procedures.', would become 'House of Lords - Medicines and Medical Procedures'.

    When I get a chance I'll try to find the code in the TV Service that updates the state to see how often it runs.

    Phew! I'm pleased that we seem to have that sorted with a workaround.

    Tony
    .
     

    Anthony Vaughan

    MP Donator
  • Premium Supporter
  • June 25, 2015
    566
    292
    Home Country
    United Kingdom United Kingdom
    Hi CyberSimian

    It occurs to me that all I have to do is clear the program table to force a complete rebuild of the EPG the next time the EPG loads (by WebEPG). Everything else should then happen automatically. So long as I schedule this to happen overnight, and nothing is scheduled to record at that time, the rebuild should be straight forward.

    I'll need to test this thoroughly first, of course.

    Tony
     

    Anthony Vaughan

    MP Donator
  • Premium Supporter
  • June 25, 2015
    566
    292
    Home Country
    United Kingdom United Kingdom
    Hi CyberSimian

    Well, I've done a fair bit of testing and have found that things are not quite as I expected.

    I developed an option in my app to reset the epg; i.e. clear the program table. Of course, I was doing this so that I could let it run overnight and expected WebEPG to start loading the EPG at the scheduled time (as defined in the WebEPG plugin).

    I was very surprised to find that the time defined by the plugin to start downloading the EPG is not actually used, or does not seem to be according to my testing. In fact, the WebEPG plugin loads each channel separately and, so far as I can tell, at indeterminate times. On my production system the EPG id schedule to load at 01:10, but the first channel to be loaded was loaded at 01:15, the next at 02:16, and the last channel was loaded at 06:57.

    On my development system, I schedule the EPG to load at 09:45. It is now 10:05 and there is still no EPG load.

    I'll update this post when the development process has finished.

    Tony

    P.S. my app now works for SQL Server, EXPRESS, MySQL and SQLite (MP2).
     

    CyberSimian

    Test Group
  • Team MediaPortal
  • June 10, 2013
    2,849
    1,770
    Southampton
    Home Country
    United Kingdom United Kingdom
    I developed an option in my app to reset the epg; i.e. clear the program table. Of course, I was doing this so that I could let it run overnight and expected WebEPG to start loading the EPG at the scheduled time (as defined in the WebEPG plugin).
    I have never used the WebEPG tool, so I don't know whether this is normal. o_O

    If you are happy to spend the time investigating the EPG behaviour and refining your apps, that is fine. (y) But if you just want a solution, I will mention some possibilities that might do some of what you want.

    (1) There is a tool called EPG Cleaner & Schedule Repair written by user hurley. This is a "TV Server" plugin, and I think that it does similar things to what you are attempting. I have never used this tool, but if it is a "TV Server" plugin, it may work better than the post-process that you are currently using. Also, it should work regardless of the origin of the EPG (so it should work with the broadcast EPG or with an internet EPG). :)

    (2) Another user has also had an attempt at cleaning the EPG, in this thread. This is intended for MP2, but since MP2 uses the same TV server as MP1 (but packaged differently), any solution for MP2 might also be usable for MP1. However, as far as I can see, this is not yet a working solution. :(

    (3) There is an existing plugin called "TV Wishlist". This may do some of what you want. But as with all of these suggestions, I have never used it. :( (I am very conservative in my use of plugins. :eek:)

    -- from CyberSimian in the UK
     

    Anthony Vaughan

    MP Donator
  • Premium Supporter
  • June 25, 2015
    566
    292
    Home Country
    United Kingdom United Kingdom
    Hi CyberSimian

    Thank you for these suggestions. I would be delighted to try any existing solutions if one of them provides a fix for this issue.

    Bilusg's contributions talk about exactly what I have done in my app and he is right that it would be better to be able to apply those to the EPG load processing than to update the program table after an EPG download. I think that the three options you suggest all operate on the program data after EPG downloads have occurred; so I doubt they will help.

    I think I have figured out what I may have done wrong in my testing this morning. I'll try again tomorrow morning and report back.

    Thanks again for all of your efforts.

    Tony
     

    Users who are viewing this thread

    Top Bottom