EPG (both DVB and WebEPH) stopped working? (1 Viewer)

Chris Melville

Portal Member
August 2, 2016
10
0
45
Home Country
United Kingdom United Kingdom
Dear MediaPortal experts,

I use MediaPortal TV server as a back-end to Kodi. Until yesterday, it was working almost fine (I think).

I have both the DVB EPG and the WebEPG plugin enabled. I have never been too sure how these work together: I assume that they work in tandem, writing data to the same database, and entries are updated by whichever of the two ran more recently. Can someone confirm if this is correct?

I would notice that there appeared to be a delay in the database actually updating the programme information. For example, if I noticed that a channel was mapped incorrectly in WebEPG (and therefore the wrong listings were appearing in Kodi), I would go and re-map that channel; click "Grab Now"; wait until it finished grabbing; save; and then re-load Kodi (with the option of not caching EPG data and always refreshing it from the database). However Kodi would not give me the new listings immediately, and I might have to wait some time before it did. I do not know what caused this. However I could live with it, which is why I said it worked "almost" fine.

Anyway - today I saw that BBC Four EPG information gave a message saying that the user should re-tune / re-scan channels in advance of the Olympics. I therefore went into the MediaPortal TV Server configuration and scanned for new TV channels.

Next thing I know, Kodi shows no EPG information at all. It's all blank. It's as if refreshing the TV channel list has disabled the EPG from working. I went back to the WebEPG plugin to re-scan - which completed successfully (saying that it imported 102 channels and 3217 programmes) - however despite this, the actual EPG still shows as blank.

Moreover, when I go to the "Manual Control" screen of the configuration and click "Refresh DVB EPG", it is now no longer working. It pops up the message "EPG Grabber will restart in a few seconds..." - however nothing happens. I do not see any activity on my TV card. It was working before, but now it is not.

Does anyone have a clue what might be happening here? Could it be a corruption of the database in which the actual EPG information is stored? I have tried to investigate, but I have no idea where the location would actually be on my hard drive. I poked around in the "Team MediaPortal" folders in AppData and ProgramData (I'm on Windows 10), however I can't find any database files. A Google search for MediaPortal EPG database location only returns results related to setting up the EPG, but neglects to mention where the database is actually stored.

Please help!

Thanks.
 

mm1352000

Retired Team Member
  • Premium Supporter
  • September 1, 2008
    21,577
    8,224
    Home Country
    New Zealand New Zealand
    Hello and welcome Chris :)

    I have both the DVB EPG and the WebEPG plugin enabled. I have never been too sure how these work together: I assume that they work in tandem, writing data to the same database, and entries are updated by whichever of the two ran more recently. Can someone confirm if this is correct?
    Yes, I confirm that's roughly correct.

    I would notice...
    TV Server has no cache for EPG data, so there is no possible explanation in that direction.

    If (...and this is a huge IF!) TV Server is responsible for this delay, the only plausible explanation I can come with would be as follows...

    Updating EPG updating can be a resource-intensive task, so in most cases the import process runs as a background task. The intention behind that is to avoid stalling/overloading the system. As a result EPG data updates over time, not in one big hit. If you were starting with zero EPG data and using MediaPortal you'd be able to see the guide filling in progressively channel by channel. Unless you have a lot of channels and/or many days worth of data, the import shouldn't take much more than a few minutes. Obviously the more channels you have, the longer it will take.

    Now, the order of updates for channels is not guaranteed. The channels you're attempting to fix could be first... or they could be last. Furthermore, if most of the channels already have correct data, you probably wouldn't even notice them being updated. Therefore to me it seems very plausible that you may think nothing is happening for some time (ie. there's some random delay), but in reality [with the greatest respect :) ] you may just be expecting too much and/or not being patient enough. :)

    If this behaviour is a significant nuisance for you and you want to take this further, you'll need to reproduce the situation with MediaPortal and provide relevant detail (log files, screenshots etc.). I'm completely unable to assist with KODI or KODI's TV Server plugin.

    Next thing I know, Kodi shows no EPG information at all. It's all blank...
    Again, it's not possible for me to help with KODI, and again I'd be happy to assist if you can reproduce with MediaPortal.

    Moreover, when I go to the "Manual Control" screen of the configuration and click "Refresh DVB EPG"...
    Please be aware that clicking "refresh DVB EPG" will cause all EPG data to be deleted. Considering the delay you've reported in seeing EPG data updated in KODI, the combination of this refresh function and the KODI cache could be a partial explanation for why you see no EPG data.

    ...it is now no longer working. It pops up the message "EPG Grabber will restart in a few seconds..." - however nothing happens. I do not see any activity on my TV card. It was working before, but now it is not.
    It's basically impossible to help with this part of the problem with zero log files and zero knowledge of your EPG settings.

    Perhaps all your EPG data has been coming from the WebEPG plugin?
    Then:
    1. No activity could be normal.
    2. Your EPG data disappeared because you clicked "refresh DVB EPG".
    3. You haven't done a WebEPG import after refreshing.
    4. ==> no data

    Does anyone have a clue what might be happening here?
    Not really. That's partly because you're using KODI, but mostly because while you've described the problems really well, you haven't provided the detail (screenshots of settings, full log files etc.) that would be needed to really understand what's going on.

    Could it be a corruption of the database in which the actual EPG information is stored?
    Unlikely.

    I have tried to investigate, but I have no idea where...
    EPG data is stored in TV Server's database - which is either a MySQL or SQL Server database - along with almost all other TV Server settings and data (channels, schedules etc.). Therefore you won't be able to locate a file on your HDD that you could simply open and view/edit with a text editor or web browser. Investigating would require that you install an appropriate database editor tool for a start, and you'd probably need to learn some rudimentary SQL in order to actually examine the contents of the database. If you're confident with technology and insist that you want to go down this route then feel free to do that and see what you can dig up. EPG data is stored in the Program table, linked to channels (Channel table) through the channel identifier (IdChannel). It's not practical to provide more detailed instructions or explanation.

    Regards,
    mm
     

    CyberSimian

    Test Group
  • Team MediaPortal
  • June 10, 2013
    2,850
    1,772
    Southampton
    Home Country
    United Kingdom United Kingdom
    I have both the DVB EPG and the WebEPG plugin enabled.
    In the UK, the simplest EPG to get working is the EPG broadcast with the TV signals. It gives you seven full days of EPG for all channels. Although you get a description for each programme, you don't get other info, such as cast lists for films, indicators for repeats, original broadcast dates, series and episode numbers.

    Other EPG sources have to be downloaded from the internet, and may give you more programme information, but they are more difficult to setup, and may charge a monthly or annual fee. I use the broadcast EPG.

    For the UK, the broadcast EPG will give you a full EPG (starting from empty) in about 8-10 minutes. It varies, depending on programme schedules, so sometimes you may get a full EPG in 6 minutes, while at other times it may not be complete even after 10 minutes. If you look at the panel shown on this page in the Wiki:

    http://wiki.team-mediaportal.com/1_...figuration/TV-Server_Configuration/05_DVB_EPG

    you will see that the default timeout for "EPG grabbing while idle" is set at 10 minutes. That is a good default, and is the value that I use. To set up the broadcast EPG, I would suggest that you look at the following post from mm (and remember to disable WebEPG!):

    https://forum.team-mediaportal.com/threads/tv-not-recording.131759/#post-1149034

    -- from CyberSimian in the UK
     

    Chris Melville

    Portal Member
    August 2, 2016
    10
    0
    45
    Home Country
    United Kingdom United Kingdom
    Hello and welcome Chris :)

    Thank you for taking note of this thread, and being willing to help :) (Sorry, I don't know how to mark-up another level of quotation here...)

    Updating EPG updating can be a resource-intensive task, so in most cases the import process runs as a background task. The intention behind that is to avoid stalling/overloading the system. As a result EPG data updates over time, not in one big hit. If you were starting with zero EPG data and using MediaPortal you'd be able to see the guide filling in progressively channel by channel. Unless you have a lot of channels and/or many days worth of data, the import shouldn't take much more than a few minutes. Obviously the more channels you have, the longer it will take.

    Now, the order of updates for channels is not guaranteed. The channels you're attempting to fix could be first... or they could be last. Furthermore, if most of the channels already have correct data, you probably wouldn't even notice them being updated. Therefore to me it seems very plausible that you may think nothing is happening for some time (ie. there's some random delay), but in reality [with the greatest respect :) ] you may just be expecting too much and/or not being patient enough. :)

    When it used to work - after I click "Refresh EPG" (under Manual Control), I would wait until the relevant card had stopped grabbing EPG data before looking at the EPG. So one would assume that there was no more process to complete. Unless there are further background processes which take more time, even after the Manual Control screen has apparently completed its grabbing??

    If this behaviour is a significant nuisance for you and you want to take this further, you'll need to reproduce the situation with MediaPortal and provide relevant detail (log files, screenshots etc.). I'm completely unable to assist with KODI or KODI's TV Server plugin.

    Okay, I appreciate that. Currently I only have the TV Serve back-end installed. If necessary, further down the line I can also install the MP front-end.

    Please be aware that clicking "refresh DVB EPG" will cause all EPG data to be deleted. Considering the delay you've reported in seeing EPG data updated in KODI, the combination of this refresh function and the KODI cache could be a partial explanation for why you see no EPG data.

    Okay - so I understand that it would delete all EPG data. However there is no explanation for why, since yesterday, I see no activity whatsoever after the "EPG Grabber will restart in a few seconds" message :(

    It's basically impossible to help with this part of the problem with zero log files and zero knowledge of your EPG settings.

    I have now uploaded a log file to my original post. This was logged today. Note that there are some errors in WebEPG not finding sources (for some channels). That was run at 15:09. However then at 15:14 I clicked "Refresh DVB EPG" (Manual Control), expecting there to be a further log entry to show that activity - but there was nothing. No more data written to the log file. This was the most recently-modified log file in the directory. Can you explain why the manual DVB EPG refresh operation would produce no activity? It's as if the commend simply never gets executed.

    Perhaps all your EPG data has been coming from the WebEPG plugin?
    Then:
    1. No activity could be normal.
    2. Your EPG data disappeared because you clicked "refresh DVB EPG".
    3. You haven't done a WebEPG import after refreshing.
    4. ==> no data
    If I do a WebEPG import, and let it run to completion, and do *not* try to manually refresh the DVB EGP, the guide is still empty.

    EPG data is stored in TV Server's database - which is either a MySQL or SQL Server database - along with almost all other TV Server settings and data (channels, schedules etc.). Therefore you won't be able to locate a file on your HDD that you could simply open and view/edit with a text editor or web browser. Investigating would require that you install an appropriate database editor tool for a start, and you'd probably need to learn some rudimentary SQL in order to actually examine the contents of the database. If you're confident with technology and insist that you want to go down this route then feel free to do that and see what you can dig up. EPG data is stored in the Program table, linked to channels (Channel table) through the channel identifier (IdChannel). It's not practical to provide more detailed instructions or explanation.

    I do have a working knowledge of SQL. But I want to know where the database is stored. Whatever the file type (.mdb or otherwise) I am happy to investigate, but I first need to know where the file actually is. I have been given no clues so far. I do understand that the channels, EPG etc are stored on a local database server (I am running single-seat). But that SQL server was installed as part of the MediaPortal installation package, and set up automatically from there. I have been presented with no options as to how to actually access that database. I used default settings. Grateful if you could let me know where the actual data files would have been installed.

    Edit - it's ok, I found the .ibd files in the MySQL Server directory under ProgramData.

    Thanks :)
     
    Last edited:

    mm1352000

    Retired Team Member
  • Premium Supporter
  • September 1, 2008
    21,577
    8,224
    Home Country
    New Zealand New Zealand
    Thank you for taking note of this thread, and being willing to help :)
    You're very welcome, any time. :)

    When it used to work...
    Off the top of my head, I think the tuner is held in use (ie. appears to still be grabbing) until the import process completes, but I could be wrong about that.

    Okay - so I understand that it would delete all EPG data. However there is no explanation for why, since yesterday, I see no activity whatsoever after the "EPG Grabber will restart in a few seconds" message :(
    As I said in my previous reply, I couldn't offer you any explanation for anything unless you provide additional detail in the form of log files, screenshots etc.

    I have now uploaded a log file to my original post.
    ...
    Can you explain why the manual DVB EPG refresh operation would produce no activity?
    No, I can't. The single log file you provided is inadequate for assisting you with that particular problem.

    Please provide:
    1. All the log files in that folder, not just the most recently modified.
    2. Screenshots of:
      1. Your tuner list.
      2. The DVB EPG grabber section.
      3. The DVB EPG -> TV EPG Grabber sub-section.
      4. The DVB EPG -> Radio EPG Grabber sub-section.
    If I do a WebEPG import, and let it run to completion, and do *not* try to manually refresh the DVB EGP, the guide is still empty.
    The log file you provided contains repeated messages like this:
    [2016-08-03 15:08:29,195] [Log ] [WebEPGImporter] [INFO ] - WebEPG: Program Count (17) < Listing Count (28) - Discard Count (7), possible template error

    There are still some database inserts:
    [2016-08-03 15:08:30,410] [Log ] [SQL EPG importer] [DEBUG] - BusinessLayer: Inserted 17 programs to the database

    I would interpret this to mean that the WebEPG grabber script may be almost completely broken... but I'm not a WebEPG expert. Note that I'm also aware of recent significant changes in relation to RadioTimes and DigiGuide:
    https://forum.team-mediaportal.com/threads/uk-grabbers-are-any-working-at-the-moment.134615/

    I had thought most of that affected people who used the XMLTV Import plugin, but it's entirely possible that there are flow-on effects to the WebEPG plugin if website formats/content has changed.

    I do have a working knowledge of SQL. But I want to know where the database is stored.
    As I tried to explain, the specific location of database data files is irrelevant for MySQL and SQL Server databases. To access the contents of the database, you need to know which machine is hosting the database... but beyond that, location is completely irrelevant.
     

    Chris Melville

    Portal Member
    August 2, 2016
    10
    0
    45
    Home Country
    United Kingdom United Kingdom
    UPDATE:

    Via checking the log files myself, I saw that when the EPG refresh was working, an event was logged for each time the grabber started and stopped. However after it apparently stopped working, no events of any kind were logged - even when I clicked the button to manually refresh.

    I still have no idea what caused this.

    Anyway - I did a complete reinstallation of MediaPortal (dropping the previous database), and it is working now. I did not enable the WebEPG plugin: all the data is now coming from the DVB EPG. No idea if the two plugins working together had anything to do with the bug.

    Thanks for your help anyway, MM and CyberSimian.
     

    Users who are viewing this thread

    Top Bottom