Multituner und EPG Grabber (15 Viewers)

[mm1352000]

Hi, i think the timeout always is used also when epg is received for a channel...

You mean there is a [timeout * 1 minute] delay between end of grabbing for one channel and start of grabbing for the next?

Those screenshots looks really nice, much better then in TVE3!

Thanks.
The changes made sense to me, but I'm always nervous about what the community might think. So, your positive feedback is reassuring.

Are the changes already in the latest available native TV build for MP2? Taking from the screenshots you did them in Visual Studio and not from a running TV-Server?

You're right - the screenshots are from VS design view. They're not in the latest available Native TV build, or even publicly committed/pushed yet. As previously mentioned, I have to fix a few things that I broke before I could push them. It will take some time, but I'm trying to complete as fast as possible without creating more bugs than I fix.

 

[Vasilich]

when Vasilich says "...other programs don't have such a long delay before grabbing starts", I think he must have been confused... because the setting should not have any relationship to a start delay.

well, that was bad/wrong wording selected by me. Actually grabber starts immediately, but the grabbing results you will see only after timeout expires (because only then TVE writes it to the DB and MP client can see that).
And saying that "other software don't have such delays" i actually meant the delay between tuning to a channel and seeing EPG info in GUI, so you don't need to wait several minutes to get EPG for the channel you are currently watching.
I am still living with that patch and set it to 15 seconds, because i don't have any channel/transponder that has so many EPG info that it needs longer that 15 seconds to transmit it all.

 

[FreakyJ]

I like your rework mm But I see one issue.
How do you prevent the inbuilt epg grabber from overwriting xmltv entries? Because I can't tell it anymore: don't store the data for this channel...

 

[mm1352000]

well, that was bad/wrong wording selected by me...

Ahhh, I see - okay.
Well the issue there is actually probably the design of EPG grabber. Especially the code in TsWriter.

First, the grabber only returns data when it thinks it has received all data or timeout is reached. This is probably the single biggest cause of the difference you see between MP and [for example] DVBViewer.

Also, the way the grabber detects when the EPG is complete currently is to wait until no new data - detected by CRC comparisons - is seen for at least 60 seconds (hard-coded!!!). I have completely rewritten [most - not quite finished] the EIT parser in TsWriter to use section_number, last_section_number, last_table_id etc. to try to improve the completion detection. However it is difficult because those fields are per-service, and there doesn't seem to be an easy way to know which services actually have EPG. In other words, completion doesn't seem to be deterministic.

Having said all that, the only reason I can think of that you could experience this in the first place is if your provider only broadcasts EPG for a few hours, and your PC sleeps for a few hours without a chance to update the data. IMO that isn't (or shouldn't be) a common situation, because [good] providers usually like to broadcast ~7 days of data.

So, I wonder whether your grabbing is configured correctly (ie. to get the "long range" guide data) and/or whether your short timeout is actually preventing you from getting the "long range" data that would solve the problem.

Hope that makes sense?

 

[mm1352000]

I like your rework mm But I see one issue.
How do you prevent the inbuilt epg grabber from overwriting xmltv entries? Because I can't tell it anymore: don't store the data for this channel...

Easy: simply don't store data for channels that have an ExternalId value.
ExternalId is the column that XMLTV, WebEPG, SchedulesDirect etc. use to link the channel to a channel from their guide data source. If the column has a value, a plugin is responsible for providing the EPG for the channel.

 

[FreakyJ]

Ah didn't know that the plugin actually changes the db.
Thanks for clarification

 

[snowball]

You mean there is a [timeout * 1 minute] delay between end of grabbing for one channel and start of grabbing for the next?

Yes that is what i observed. But you discussed with vasilich already the background. As far as i can remeber (analysis is 2-3 years old): The simple ch.refresh() i put into the loop helped to reduce time also. Otherwise the grapper was doing graps on each channel instead of 1 per transponder.

 

[CyberSimian]

Added grabbing of series and episode identifiers (UK Freeview/Freesat, NZ Freeview etc.) so series link schedules may be supported in future.

Brilliant!

Now the EPG grabber will simply grab once from each transmitter with at least one visible channel (rather than once for each enabled channel on each transmitter!) and store all available data.

Good idea! (The same thought had occurred to me when thinking about the missing EPG highlights problem.)

Removed "grab EPG only for channels on the same transponder" setting.

In my experiments with different EPG settings, enabling this option was the only setting that avoided the missing EPG highlights problem. Hopefully the changes elsewhere will prevent missing EPG highlights. (And I must be frank and admit that I have reverted to not using that setting, as it seems to delay EPG grabbing too much.)

The idle EPG grabber is currently broken. When I fix it, I intend to enable it to use idle tuners even if timeshifting and/or recording is active. In other words, TV Server does not have to be completely idle. Only the tuner has to be idle. For now I don't intend to support simultaneous grabbing with multiple tuners.

The proposed panel allows the user to enable both idle grabbing and timeshift grabbing. Presumably if timeshift grabbing is already active, idle grabbing won't start (even if there is an idle tuner).

How do you prevent the inbuilt epg grabber from overwriting xmltv entries? Because I can't tell it anymore: don't store the data for this channel

Easy: simply don't store data for channels that have an ExternalId value.

Windows 7 Media Center does a similar thing, but attempts to be a bit cleverer. I used Vista MC for five years, but W7 MC only for one week, so my knowledge of W7MC is not encyclopaedic. The question is: What should the EPG grabber do when the two EPGs disagree?

My experience with W7MC is that it uses the Microsoft downloaded EPG if the EPGs agree (good -- the MS EPG contains more information), but uses the broadcast EPG if the EPGs disagree. So what does "disagree" mean? It certainly means differences in programme start and end times. For example, the MS EPG often has a single entry for a programme where the broadcast EPG has that programme with a shorter duration and preceded or followed by an EPG entry for 3-4 minutes of news headlines. So one entry in the MS EPG becomes two entries in the broadcast EPG. "Disagree" might also mean differences in programme titles (not sure about that).

Also, the advantage of the broadcast EPG is that it can be updated by the broadcasters in real time, in response to last-minute schedule changes (e.g. live-sport running beyond its scheduled time, thereby affecting subsequent programmes). Even if using the MS (or other) EPG, I think that most users would probably want the EPG to be updated with last-minute schedule changes. Of course, it is easy for me to say that -- I am not writing the code!

-- from CyberSimian in the UK

 

[snowball]

[*]Removed "grab EPG only for channels on the same transponder" setting. No longer relevant due to the above change (remove TV and radio EPG grabber sections).

Please do not remove this option. Or make it default behavior. I introduced it in the past because we have in germany some broadcaster that send limited epg information on all theire channels on all transponders via sat. E.g. ARD or sky Means the first transponder with one of their channels will fill DB for all channels with limited info. As channel was grapped it will not grap again. Or did i understood your change wrongly, and you will always store what you find? In case wouldnt that increase db load due to entry matching prozessing? How will you identify a programm entry to be more accurate then an already injected one. Sky was sending programm on all transponders but desription only same transponder.

 

[mm1352000]

Or did i understood your change wrongly, and you will always store what you find?

Yes, this is what I will do.

In case wouldnt that increase db load due to entry matching prozessing?

Yes, it will increase CPU load a little bit... but not too much. Current CPU load is very low, so this change shouldn't be a problem.

(How will you identify a programm entry to be more accurate then an already injected one. Sky was sending programm on all transponders but desription only same transponder.

New data is assumed to be more accurate. So, if the start or end times for programs change then the new data will be used automatically.
If the only difference is the title or description then the data with the longest title/description will be used.

In short, no need to worry.