1.11.0 Video & Audio Stuttering for short period of time for LiveTV only after S3 or S5 (1 Viewer)

[marttoma]

Hi Sebastii,

here is my proposed text for Wiki (for http://wiki.team-mediaportal.com/1_...figuration/TV-Server_Configuration/05_DVB_EPG).

I would propose to add:

• red text to the Description
• section EPG grabbing in general
• section for EPG grabbing while timeshifting/recording
• section for EPG grabbing when idle

Description
DVB (Digital Video Broadcast) channels have the possibility to receive program information embedded in the TV signal-stream.
This system is called EPG (Electronic Program Guide).
The EPG information that is received depends on the tv broadcaster. Some TV channels include very detailled EPG information, some only a very basic EPG and some don't transmit EPG at all. Some TV channels includes EPG not only for that same TV channel, but also for other TV channels transmitted even on the same TV multiplex or also for (all or almost all) other TV channels comming from TV brodcaster.
Besides receiving the EPG through the DVB stream you can also download the information from various internet sources by using the WebEPG plugin.


EPG grabbing from TV broadcasting in general
Thera are three options how to keep updated EPG from TV broadcasting in Media Portal:

• EPG grabbing while timeshifting/recording
• EPG grabbing when idle
• Both above combinations

If you dont use any from above 3 options, than your TV Server can not received any EPG which is usefull for set up TV recording or for information, what specific TV program is about. You can set up any option from above.

If you dont let TV Server to go to sleep and your TV Server is in idle mode quite often, probably EPG grabbing when idle is good choice.

If you dont have too many TV channels and your TV Server is going quite often to sleep, than EPG grabbing while timeshifting/recording option + EPG grabbing when idle could be good choice.

If you want to be "saved" - i.e. to have EPG regularly updated, then EPG grabbing based on Power Scheduler every day (e.g. during night hours) could be good choice as well.


EPG grabbing while timeshifting/recording
This option grabs EPG only if TV Server is in timeshifting/recording mode - i.e.:

• TV Server is playing any live TV/Radio (timeshifting status) OR
• TV Server is recording any live TV/Radio content (recording status)

Enabled:
This checkbox switch ON/OFF the EPG grabbing feature "EPG grabbing while timeshifting/recording".

The trigger to start EPG grabbing while timeshifting/recording is based on 1 from 2 possible events:

• a switch ON TV/Radio channel
• a TV/Radio channel change to another TV/Radio channel

If you change TV/Radio channel before EPG grabbing is completed (approx. 2-3 minutes - but this strongly depends on the number of TV/Radio channels selected for EPG grabbing in TV EPG Grabber and Radio EPG Grabber settings) then EPG grabbing would be cancelled.


Timeout:
This is a time setting how long TV Server will try to start EPG grabbing. Default is 2 minutes.


Refresh every:
There is no "Refresh every" feature like in "EPG grabbing when idle" settings. If you stay on the same TV/Radio channel for a long time then EPG will eventually run out (due to no EPG refresh).


Notes:
Be carefour with TV EPG Grabber and Radio EPG Grabber settings, if you select to many TV or Radio channels for EPG grabbing, TV/Radio stuffering can appeared due to too long processing time for EPG data grabbing.



EPG grabbing when idle
This option grabs EPG only if TV Server is in idle mode - i.e.:

• all tunners are in idle (all tunners are inactive - i.e. no live TV/Radio or TV/Radio recording)

Enabled:
This checkbox switch ON/OFF the EPG grabbing feature "EPG grabbing when idle".

The trigger to start EPG grabbing when idle is based on 1 event only:

• All tuners have to be in idle and the time since the last EPG grab completed exceeds the "refresh every" time.

Timeout:
This is a time setting how long TV Server will try to start EPG grabbing. Default is 10 minutes.


Refresh every:
TV Server will refresh the EPG as often as you specify using the "refresh every" setting.

 

[mm1352000]

But how to find there is such channel, which contains data for other channels as well?

Usually the full EPG is either carried only on the provider's "home" transmitter, or it is carried on each transmitter. The provider's website may tell you this. If it doesn't, there's no way to know except with trial and error. The starting point should usually be to tick one channel from each frequency.

How did you find that this "Barrandov TV HD" contains EPG data for other TV channels as well? Which log file did you use to find this information?

The log section (from TV service log) that told me this is included in my previous post. As you can see:
[2015-06-28 21:12:41,064] [EPG ] [21 ] [INFO ] - dvb:dvb ready.EPG 183 channels

So, I knew that EPG for lots of channels is available from that frequency. However, sometimes the EPG may only be for the current and next programmes (or X hours), so I also took notice of these:
[2015-06-28 21:12:44,683] [EPG ] [EPG Update thread] [INFO ] - - Inserted 247 epg entries for channel CT 1
[2015-06-28 21:12:45,375] [EPG ] [EPG Update thread] [INFO ] - - Inserted 265 epg entries for channel CT 2

The number of entries is large, which tells me that full EPG is available.

Regards,
mm

 

[marttoma]

TV EPG Grabber

There are 4 options how each provider can send EPG data:

Option 1:

EPG data are sent with each TV channel for all TV channels from that provider. This means that you will only need to check one TV channel from each provider. Due to fact it takes some time to receive EPG data from one TV channel (in minutes drive), it is recommended to select just one specific TV channel including EPG data for all TV channels to save processing time in order to avoid stuffering (if EPG grabbing while timeshifting/recording).
​

Option 2:

EPG data are sent with one TV channel for all TV channels from that provider. EPG data of other TV channels are sent only for these specific TV channels.
In this case recommendation is to check only such specific one TV channel from each provider, which contains EPG data for all TV channels from such provider to save EPG processing time in order to avoid stuffering (if EPG grabbing while timeshifting/recording).​

Option 3:

EPG data are sent with each TV channel only for such specific TV channel from that provider. In this case you have to check all TV channel from that provider to secure you have EPG data available for all TV channels from that provider -> this setting can bring stuffering (if EPG grabbing while timeshifting/recording) if you select a lot of TV channels for EPG grabbing.​

Option 4:

EPG data are not sent at specific TV channel at all.

​

In general the recomendation for option 1 and 2 is to select only these TV channels, which contain EPG data for all TV channels to reduce EPG processing time as much as possible.


Store data only for the selected channels:

If "Store data only for the selected channels" is checked, it will store all available EPG data from all these checked TV channels (also EPG data for all (!) other TV channels from specific provider as decribed in option 1 and 2).

If "Store data only for the selected channels" is unchecked, it will store all available EPG data for all TV channels it find EPG for from the TV channel list (checked boxes for TV channels are not relevant for this case).


Note: Some channels are sending bad EPG that can make TV Server crash

 

[marttoma]

Hi mm,

finally I hvae good news. With the "Barandov HD" TV channel with full EPG the stuffering disappered if EPG while timeshifting. It confirm (of course) your statement about my previous EPG setting.

So now I have both EPG when idle (based on PS) and EPG while timeshifting and it works fine

So no driver updates, configuration setting testing any more

Are you OK with EPG "wiki" text?

Best regards,
marttoma

 

[CyberSimian]

I have been following this thread with some interest, as I am not 100% convinced that I have the optimal EPG options for my system. May I ask a couple of questions?

(1) The meanings of the two timeouts on the "DVB EPG" tab in "TV Server Config" are not currently described in the Wiki; see this page:

http://wiki.team-mediaportal.com/1_...figuration/TV-Server_Configuration/05_DVB_EPG

I think that I remember from other threads that the timeouts have the following meanings:

(a) The timeshifting timeout is the time that elapses between switching to a different channel, and the EPG grabber starting to grab the EPG. True? (There is some sense in not starting EPG grabbing if the user is switching rapidly through the channels trying to find something interesting to watch.)

(b) The idle timeout is the maximum time that the EPG grabber spends grabbing from a single MUX. True?

(2) If statement (b) is correct, I was wondering whether there is a "smart" limit on the time spent in idle EPG grabbing? The EPG is broadcast continually and repeatedly, so if the grabber grabs for long enough, the EPG will start coming round again -- like a carousel. A smart EPG grabber would keep some record of how much of the EPG it had grabbed during the current session, and when it had grabbed 100% it would stop grabbing. So if I specified an idle timeout of 60 minutes, the grabber might stop grabbing after (say) 12 minutes (for DVB-T the UK). Does it work this way?

However, keeping track of how much of the EPG has been grabbed may be too complex a task for the EPG grabber to perform. If this is the case, the grabber presumably grabs for the specified timeout and then stops. In which case, is it possible for the user to determine how long to grab for? Not so short that some of the EPG is missed, but not so long that the entire EPG is grabbed several times over. This would determine what value to specify for the idle timeout setting.

Thank you.

-- from CyberSimian in the UK

 

[mm1352000]

@marttoma

So now I have both EPG when idle (based on PS) and EPG while timeshifting and it works fine

While I'm happy for you, I think you may find that you will see stuttering if you view the Barandov HD channel. It's up to you if you leave the timeshifting EPG grabber enabled.

Are you OK with EPG "wiki" text?

I need some more time to check it. Some things like "Note: Some channels are sending bad EPG that can make TV Server crash" are definitely wrong.

Regards,
mm

 

[mm1352000]

@CyberSimian

May I ask a couple of questions?

Yes, always.

(a) The timeshifting timeout is the time that elapses between switching to a different channel, and the EPG grabber starting to grab the EPG. True?

No, wrong. It's the time that the timeshifting grabber is allocated to complete grabbing of the available EPG.
If the timeout expires before the grabber completes grabbing then TV Server takes the EPG that was acquired (if any).
If the user changes channel or stops TV before the grabber completes then no EPG is acquired.

(There is some sense in not starting EPG grabbing if the user is switching rapidly through the channels trying to find something interesting to watch.)

I'm not sure that I agree. In any case, it's certainly not how TV Server does things.

(b) The idle timeout is the maximum time that the EPG grabber spends grabbing from a single MUX. True?

Yes. The idle timeout is the same as described above, except for the idle grabber rather than the timeshifting grabber.

(2) If statement (b) is correct, I was wondering whether there is a "smart" limit on the time spent in idle EPG grabbing? ... Does it work this way?

Yes of course. Grabbing is determined to be complete if 60 seconds elapse and new (not previously encountered) EPG is not seen.

However, keeping track of how much of the EPG has been grabbed may be too complex a task for the EPG grabber to perform. If this is the case, the grabber presumably grabs for the specified timeout and then stops.

As above: this is not what the EPG grabber does.

In which case, is it possible for the user to determine how long to grab for? Not so short that some of the EPG is missed, but not so long that the entire EPG is grabbed several times over. This would determine what value to specify for the idle timeout setting.

I don't think your question is something that the user should be worried about (due to the fact that the grabber supports a "smart limit"), but the direct answer to your question is: not easily.

The user could set the limit extremely high and use the log to check how long grabbing actually takes (search the TsWriter log for "epg: epg received"). I would expect the time to vary from provider to provider, transmitter to transmitter, and even somewhat from grab to grab on the same transmitter. That would be a lot of testing to do for most people... and I honestly can't imagine what they'd have to gain from doing this.

You might also be interested to read ETSI TS 101 211 (freely available; formerly known as TR 101 211), which sets out recommended repetition rates for EPG.
TL;DR: section 4.4 recommends minimum repetition rates...
EIT P/F* actual** = 2 seconds
EIT P/F* other** = 10 seconds cable, satellite; 20 seconds terrestrial

The repetition rates for further EIT tables will depend greatly on the number of services and the quantity of related SI information. The following transmission intervals should be followed if practicable but they may be increased as the use of EIT tables is increased. The times are the consequence of a compromise between the acceptable provision of data to a viewer and the use of multiplex bandwidth.

EIT schedule* actual** first day = 10 seconds
EIT schedule* other** first day = 10 seconds cable, satellite; 60 seconds terrestrial
EIT schedule* actual** next 7 days = 10 seconds cable, satellite; 30 seconds terrestrial
EIT schedule* other** next 7 days = 10 seconds cable, satellite; 300 seconds terrestrial
EIT schedule* actual** beyond 8 days = 30 seconds
EIT schedule* other** beyond 8 days = 30 seconds cable, satellite; 300 seconds terrestrial

*P/F = current and next programmes only; schedule = full programme list
**actual = channels transmitted by the tuned transmitter; other = channels transmitted by other transmmitters

Broadcasters may choose to accept and adhere to these recommendations... or not.
Both EIT P/F and schedule are optional, as stated in ETSI EN 300 468 section 5.2.4.

Make of that what you will.

 

[CyberSimian]

The timeshifting timeout is the time that elapses between switching to a different channel, and the EPG grabber starting to grab the EPG.

No, wrong. It's the time that the timeshifting grabber is allocated to complete grabbing of the available EPG.

Thank you for correcting my misunderstanding of this timeout.

I was wondering whether there is a "smart" limit on the time spent in idle EPG grabbing?

Yes of course. Grabbing is determined to be complete if 60 seconds elapse and new (not previously encountered) EPG is not seen.

That is interesting. As marttoma had had some success with timeshift grabbing disabled, I thought that I would try that. My TV was off, but the HTPC was recording a programme, and was overdue for an idle EPG grab. I made a note of the time interval between the recording ending and the HTPC hibernating; it took 90 minutes.

I am using MP 1.9.0 pre with debug logs, so I had a look at the TV Server log (attached). You can see that the idle grabber grabbed from 8 MUXes, with each MUX timing out after 10 minutes. So the smart limit on grabbing was never triggered. The quote from the ETSI document suggests that the EPG should have been complete after 300 seconds, so with a further 60 seconds for no new EPG data to be encountered, idle grabbing should have stopped after 6 minutes.

The UK DVB-T system has acquired many more channels over the years (possibly more than originally envisaged when the ETSI document was written), so perhaps the 300 seconds is now too short for a complete EPG grab. This is why I postulated grabbing for 60 minutes, with the grab ending after perhaps 12 minutes. I will try 60 minutes and see how long it takes.

A couple of other points:

(1) From the log, the grab from each MUX seems to include both SD channels and HD channels. This is contrary to my previous understanding that the SD MUXes did not contain the EPG for the HD channels. So it looks as though in the UK any MUX can be used. However, I think that it is still true that the HD EPG is transmitted in an encoded form, which is why WMC can display the EPG for the SD channels, but not HD channels.

(2) If you look at what PowerScheduler is doing, there seems to be a window for something horrible to happen. During the EPG grab, a tuner is active and PowerScheduler correctly detects every 15 seconds that the system should not sleep/hibernate. But when the grab for a MUX ends, on my system it takes 90 seconds to update the SQL database(s), during which time PowerScheduler thinks that it is OK to sleep/hibernate. Surely there is a check missing here? PowerScheduler should be checking whether SQL updates are being performed and (if they are) prevent sleep/hibernation.

You may think that this is academic, but on my system I use a very short hibernation timeout, controlled by my own application. With WMC I used a hibernation timeout of 10 seconds, but that does not work with MP because of PowerScheduler's 15 second heartbeat. I currently use a timeout of 1 minute, but that results in recording failures about 5% of the time because of another window when PowerScheduler fails to prevent sleep/hibernation (as the recording is just about to start). So I use 1 minute normally, and 5 minutes for unattended recordings. I have never noticed a problem from this sleep/hibernation window at the end of grabbing, so perhaps it is benign. Nevertheless, one must conclude that PowerScheduler's processing is not "watertight".

-- from CyberSimian in the UK

Attached is the TV Server log; look at entries for 2015-07-03:

15:55:06.619 EPG grabbing initialised
15:55:10.099 Grabbing starts on "BBC1 HD"
16:05:10.357 Grabbing times out
16:05:11.283 Database update starts
16:06:43.491 Database update ends

This is then repeated for each of the other 7 MUXes.

 

[mm1352000]

I'm not sure how much value I can add to your investigation.

The quote from the ETSI document suggests that the EPG should have been complete after 300 seconds, so with a further 60 seconds for no new EPG data to be encountered, idle grabbing should have stopped after 6 minutes.

As I said previously...

The ETSI document is just a guideline; an informative recommendation.
Broadcasters can and do completely ignore it.

I can't emphasize that enough.
Your expectation that idle grabbing should have stopped after 6 minutes is only reasonable if the broadcasts you're receiving are actually intending to comply with the recommendations.

The UK DVB-T system has acquired many more channels over the years (possibly more than originally envisaged when the ETSI document was written), so perhaps the 300 seconds is now too short for a complete EPG grab.

In case you hadn't noticed, the ETSI document was last updated in 2013.
The simplest explanation is that the UK DVB-T system does not follow the recommendations.

(1) From the log, the grab from each MUX...

As far as I'm concerned anything is possible, and I make no claim about what the situation is or is not. There are "a million and one" configuration options for broadcasters when it comes to EPG. Subsets of channels, subsets of time, subsets of event attributes, custom text encodings... a combination of all of the above etc..

However, I think that it is still true that the HD EPG is transmitted in an encoded form, which is why WMC can display the EPG for the SD channels, but not HD channels.

As above: anything is possible. I'm not familiar enough with UK TV or WMC to make an educated comment.

(2) If you look at what PowerScheduler is doing...

I'd be reasonably sure that PS doesn't have any ability to check whether SQL updates are being performed. TV Server doesn't provide that level of insight to plugins. However, the tuner is active for the duration of the database update, so I don't see why that wouldn't prevent standby.

Beyond that I think it would be unwise for me to comment further. I'm not at all knowledgeable when it comes to PS or power configuration, and I don't even have access to your PS settings.

 

[CyberSimian]

You can see that the idle grabber grabbed from 8 MUXes, with each MUX timing out after 10 minutes. So the smart limit on grabbing was never triggered.

Your expectation that idle grabbing should have stopped after 6 minutes is only reasonable if the broadcasts you're receiving are actually intending to comply with the recommendations.

I performed a test overnight... with different results.

(1) I had previously disabled the timeshift grabber and the idle grabber so that the EPG would be overdue for grabbing.

(2) The TV Server log included in my previous post showed that in the UK it is not necessary to grab from one 24-hour TV channel per MUX; it is sufficient to grab from a single 24-hour TV channel. So I deselected all grabbing channels except for "BBC1 HD".

(3) I set the refresh interval for the idle grabber to 24 hours, and the timeout to 12 hours.

(4) Finally, I rebooted, and then enabled the idle grabber.

Looking at the TV Server log file this morning reveals two differences from the previous log:

(a) The grabber's smart limiter operated, with grabbing (including database update) taking only 7 minutes 8 seconds. This is not that different from my previous "guesstimate" of 6 minutes based on the ETSI document.

(b) For this grab, PowerScheduler correctly identified that it should not allow the system to sleep/hibernate whilst the database updates were being performed (the exact opposite of the situation shown in the previous log).

Well, this is all very puzzling , and I cannot point to any particular factor that might explain the difference between this EPG grab and the one shown in the log in my previous post.

I am beginning to think that my HTPC is one of those systems that benefits from a daily reboot, presumably because it reduces the accumulation of RAM bit errors caused by cosmic rays.

Going forward, I will use: idle grabber only, one channel only, refresh interval of one hour, grab timeout of 30 minutes.

-- from CyberSimian in the UK