Scheduled recordings not always highlighted in guide (3 Viewers)

[mm1352000]

Hi folks

Just a quick note to say I've seen the update, but don't have time to make any fuller response now. In general though I'm not familiar with how the EPG updating and highlighting on the client side is handled, so somebody else may need to step in to take this to the next step.

Regards,
mm

 

[CyberSimian]

For the last two weeks I have been trying a slightly-modified TV Server configuration. This modified configuration does seem to reduce the occurrence of missing EPG highlighting.

In my previous post I described the problem that I noticed for the first time last week, namely vanishing EPG highlighting. In this post I want to describe a workaround for the problem that I noticed soon after starting to use MP several months ago, namely missing EPG highlighting. This workaround may (or may not) also fix the problem of vanishing EPG highlighting. First let us have a recap of where we are with this problem:

(1) This thread is but the most recent of several earlier threads concerning this problem (see earlier post).

(2) Someone in an earlier thread (not me!) wondered if the problem was caused by the SQL query timing out.

(3) @RobNorthcott commented that he was experiencing this problem on an HTPC that he considered "weak", and I commented that my HTPC had a slow operating-system disk (the one used for the SQL databases).

(4) @mm1352000 examined Rob's logs and observed that his system was grabbing the EPG at the time of the problem.

(5) The next time this problem occurred on my system, I examined the TV Server logs and also found that EPG grabbing was occurring at that time.

So the indications seem to be that the problem occurs when the system is grabbing the broadcast EPG on a "weak" HTPC.

Some weeks ago I modified my EPG-grabbing settings in order to cure an unrelated problem (with help from mm ); see this thread:

https://forum.team-mediaportal.com/threads/recording-failed-to-start.128150/

I ended up with: grab when idle, grab when timeshifting/recording, grab from one 24-hour channel per MUX. However, we did not discuss the Grab EPG only for channels on same transponder setting, and it is enabling this setting that seems to avoid the problem of missing EPG highlighting. So what is going on here?

The reason that I believe this works for the UK EPG is the way in which the data is transmitted. Instead of broadcasting the entire EPG for channel 1, followed by the entire EPG for channel 2, and so on, the data for different channels is interleaved. Consequently, if the data for channels in other MUXes is ignored, it significantly reduces the rate at which SQL updates are generated, and hence reduces the load on a "weak" system.

If (say) it takes 5 minutes to transmit once the entire EPG for the 6 SD MUXes in the UK, grabbing the EPG for all MUXes would capture the entire EPG in just 5 minutes, but produce a very high SQL transaction rate.

In contrast, ignoring the EPG for channels in other MUXes means that the EPG Grabber must tune in turn a channel in each MUX in order to capture the EPG for that MUX, so the entire EPG would be captured in (5 minutes) x (6 MUXes) = 30 minutes, but the SQL transaction rate would be reduced by a factor of 6.

In practice the improvement would not be as big as a factor of 6, due to another facet of the way that the EPG is broadcast in the UK: each MUX favours its own channels. If there were no favouritism, each MUX's EPG would occupy 100/6 = 17% of the EPG bandwidth in a MUX. With favouritism, the MUX's own channels will occupy more than 17%, with less being available for the other MUXes. If the MUX's own channels occupied as much as 50% of the bandwidth, that still results in the SQL transaction rate being reduced by a factor of 2; a 30% occupation would result in a reduction by a factor of 3.

But the free lunches are not being distributed just yet. The UK has quite a dynamic EPG, meaning that the major broadcasters try to update it with last-minute schedule changes. And indeed, in the four months that I have been using MP, there have been three occasions when MP made two simultaneous recordings of the same programme, but displaced by 5 minutes. These programmes followed the main evening news, which had been extended by 5 minutes due to the amount/severity of the news that day. The broadcasters had updated the EPG, and MP had by coincidence performed an EPG grab before the programmes started, and so recorded each programme twice (at the original scheduled time, and at the revised scheduled time). Live sports programmes also often overrun, affecting subsequent programmes. However, the problem with ignoring the EPG for channels in other MUXes is that these last-minute EPG changes may go unnoticed.

So, in summary, these EPG grabber settings should eliminate missing EPG highlighting:

(a) Grab when idle.
(b) Grab when timeshifting/recording.
(c) Grab only for channels in the same MUX.
(d) Grab from one 24-hour channel per MUX.
(e) Store data for all channels in MUX.

But the cost is that your EPG may not receive last-minute schedule changes.

-- from CyberSimian in the UK

Edit: Added (d) and (e) for completeness.

 

[RobNorthcott]

That's interesting @CyberSimian
Unfortunately, I'm mostly using FreeSat, so all the EPG comes from one channel (freesatinfo radio channel, or whatever it's called). I do have a terrestrial tuner as well though (mostly used as a backup when the wind moves the dish), which is also grabbing EPG, so perhaps your suggestion may be worth me trying to take a bit of load off my system.

Although having said that, I haven't noticed any missing highlights for quite some time anyway - may be just luck or maybe some update to MP and/or Windows has made something more efficient and cured whatever bottleneck used to exist. I've moved loads of stuff out of the recordings directory as well, which has probably reduced the load on MP's indexing and thumbnail checking processes.

 

[CyberSimian]

Unfortunately, I'm mostly using FreeSat, so all the EPG comes from one channel (freesatinfo radio channel, or whatever it's called).

I don't have any experience using a satellite tuner in an HTPC, so it is probable that some or all of the explanation in my previous post is inapplicable to a satellite system.

I do have a terrestrial tuner as well though (mostly used as a backup when the wind moves the dish), which is also grabbing EPG, so perhaps your suggestion may be worth me trying to take a bit of load off my system.

If you grab the EPG for all of the terrestrial channels (instead of just those channels missing from satellite), you could encounter the missing EPG highlights problem due to the high SQL transaction rate.

Although having said that, I haven't noticed any missing highlights for quite some time anyway - may be just luck.

It is unpredictable when missing EPG highlights will occur, because it depends on when "TV Server" decides to grab the EPG. On my system, I habitually wake the HTPC first thing in the morning to check existing recordings and schedule new ones, then listen to a radio recording whilst having a cup of coffee. Since I don't use "PowerScheduler" to grab the EPG overnight, the EPG is always overdue for a grabbing, and so I almost always encounter the missing highlights first thing in the morning. But the missing highlights can occur at any time.

-- from CyberSimian in the UK

 

[RobNorthcott]

I don't have any experience using a satellite tuner in an HTPC, so it is probable that some or all of the explanation in my previous post is inapplicable to a satellite system

All the EPG for freesat comes from a single channel (MP detects it as a "radio" channel). So I couldn't use the "only grab EPG for this transponder" idea because the EPG comes from a channel that isn't on the same transponder as most of the normal channels.

If you grab the EPG for all of the terrestrial channels (instead of just those channels missing from satellite), you could encounter the missing EPG highlights problem due to the high SQL transaction rate

There's only limited terrestrial freeview where I am (only two of the MUXes), hence why I mostly use freesat and just use the terrestrial tuner as a backup if the satellite dish gets moved by a storm. I assume though that MP is still grabbing EPG for all the channels I can't get, so setting the terrestrial EPG grabber to only grab stuff for the two available transponders may be a good thing. (My channels aren't linked between DVB-S and DVB-T, I've got a separate "terrestrial" group so I can force it to use that tuner if I need to, so I need to grab terrestrial EPG as well even though there's nothing on there that I can't get on freesat).