EPG grab improvement (2 Viewers)

[SciDoctor]

At the moment the broadcast EPG grab cycles through channels .

A better option would be to cycle through the broadcast/mux frequencies.

For example at the moment the grab starts at the lowest broadcast/mux frequencies and cycles through EVERY channel on this frequency before moving to the next MUX frequency

As a single grab can take between 5-10 minutes and every channel on a MUX will have the same reception criteria it could take over 100minutes to grab NOTHING .

In the Uk and some other countries a single frequency MUX could carry many channels.In the UK up to 15 channels on a MUX .

If the grab cycled MUX frequencies the EPG grab would be more succesful and population much quicker espeacialy when a particular MUX may have EPG grab timeouts which could last HOURS.

 

[chenks]

agreed

 

[Frodo]

Yes the tvserver cycles channels when grabbing EPG
But i dont think this is a problem since:
When it grabs epg for channel 1 on MUX a
it will also receive the epg any other channel on the same MUX.
This epg info is also stored in the database, meaning that
the tvserver will skip epg grabbing for those other channels.

 

[SciDoctor]

It is the efficiency of the grab, wasted time grabing from the same MUX and timing out.

And as the grab only happens when the whole TVserver is idle you could wait days or weeks for a succesful full grab.

In the UK the other benefit is that each channel carries ALL the EPG data for every channel on every MUX.So just hitting a grab on a MUX that isn't timing out would make EPG population much better and quicker.

Couldn't the grab cycle change to a MUX cycle with incremental channels as the MUX cycle repeats.

 

[Frodo]

You didnt read before you gave an answer or i completely misunderstand you
Let me try to give an example

Say we have 2 muxes
mux A : has channels 1-20
mux B : has channels 21-40

Now with this situation, the epg grabber will work like this:
1. it starts grabbing epg for channel 1 (on MUX A)
2. suppose receives the epg for channels 1-20
3. it updates the database for channels 1-20

Next, it will see that 1-20 are already updated, so it continues with channel 21
4. it starts grabbing epg for channel 21 (on MUX B)
5. suppose receives the epg for channels 21-40
6. it updates the database for channels 21-40
7. all epg has been received. epg grabber waits for 4 hours before starting again


Now in some countries things are different
A channel will contain epg for other channels , but not for all channels
In this situation the following happens:
1. it starts grabbing epg for channel 1 (on MUX A)
2. suppose receives the epg for channels 1-10 (not a complete mux!)
3. it updates the database for channels 1-10
Next, it will see that 1-10 are already updated, so it continues with channel 11
4. it starts grabbing epg for channel 11 (on MUX A)
5. suppose receives the epg for channels 11-20
6. it updates the database for channels 11-20

Next, it will see that 1-20 are already updated, so it continues with channel 21
7. it starts grabbing epg for channel 21 (on MUX B)
8. suppose receives the epg for channels 21-30
9. it updates the database for channels 21-30

Next, it will see that 1-30 are already updated, so it continues with channel 31
10. it starts grabbing epg for channel 31 (on MUX B)
11. suppose receives the epg for channels 31-40
12. it updates the database for channels 31-40
13. all epg has been received. epg grabber waits for 4 hours before starting again

After explaining this, i fail to see why your method:
1. is more efficient
2. takes in account that a channel might not contain the full epg for all channels in the mux

Frodo

 

[SciDoctor]

Say we have 2 muxes
mux A : has channels 1-20
mux B : has channels 21-40

(good so far)

Now with this situation, the epg grabber will work like this:
1. it starts grabbing epg for channel 1 (on MUX A)
2. suppose receives the epg for channels 1-20
3. it updates the database for channels 1-20

(not so good as if MUX A is timing out because of whatever error the grab will now continue to repeat on MUX A until all channels are tried 20x10 =200 minutes of wasted time grabing, whereas if after timing out on MUX A channel 1 the grab then moves to MUX B channel 1 and gets a succesful grab; this took one tenth of the time to achieve.
Now if you have six MUX's and 100plus total channels and MUX 6 is the only MUX that is capable of a succesful grab at this time then the total time taken is ridiculus when gabing from every channel on every MUX)

Next, it will see that 1-20 are already updated, so it continues with channel 21
4. it starts grabbing epg for channel 21 (on MUX B)
5. suppose receives the epg for channels 21-40
6. it updates the database for channels 21-40
7. all epg has been received. epg grabber waits for 4 hours before starting again

(Skipping data for channes that have grabbed is a bad move because most grabs are partial and not all data is collected succcesfully. To build up a complete EPG multiple grabs for the same channel will be required. Here redundency in grabbing is a good move)


Now in some countries things are different
A channel will contain epg for other channels , but not for all channels
In this situation the following happens:
1. it starts grabbing epg for channel 1 (on MUX A)
2. suppose receives the epg for channels 1-10 (not a complete mux!)
3. it updates the database for channels 1-10
Next, it will see that 1-10 are already updated, so it continues with channel 11
4. it starts grabbing epg for channel 11 (on MUX A)
5. suppose receives the epg for channels 11-20
6. it updates the database for channels 11-20

(same as above. there is no guarantee that the previous grab for channel x recieved ALL the data for that particular channel so redundency is yet again good to get a full EPG)

Next, it will see that 1-20 are already updated, so it continues with channel 21
7. it starts grabbing epg for channel 21 (on MUX B)
8. suppose receives the epg for channels 21-30
9. it updates the database for channels 21-30

Next, it will see that 1-30 are already updated, so it continues with channel 31
10. it starts grabbing epg for channel 31 (on MUX B)
11. suppose receives the epg for channels 31-40
12. it updates the database for channels 31-40
13. all epg has been received. epg grabber waits for 4 hours before starting again

After explaining this, i fail to see why your method:
1. is more efficient
2. takes in account that a channel might not contain the full epg for all channels in the mux

(1. not efficent, wastes time repeating a grab on a MUX until all channels are tried, also even if it had a succesfull grab the data may only be partial to populate the EPG for a channel but will exclude that channel for the next grab)


(2. overcomplicated as the grabber cannot tell if the data received is all the data available.)


(What we both want is a quick uncomplicated route to grab as much information as possible

So we have six MUX

Grab MUX 1 Ch 1 if succesful store what is grabed in database and move to next MUX. IF not succesful move to next MUX

Grab MUX 2 CH 1 as above

repeat

Grab MUX 6 CH 1

Grab MUX 1 CH 2

repat for MUX 2 - 6 CH 2

GRAB MUX 1 CH 3

ETC

Now the above method covers all the channels but prioritises the MUX first .

Simplify the grab don't exclude channel info just because we had a grab from it in the last grab. We want to get the EPG populated as quickly as possible and any changes or missed info as quickly as possible)

Your existing method works on a best case scenario that data is always availble and complete for the EPG grab, you need to work on the worst case scenario tha data isn't available and you need to try as many available options as possible

 

[Frodo]

for easy understanding i left out any fault/error handling in my example
But since that seems to be your point, here is how the epg grabber currently does it:

Again, the same example


Now with this situation, the epg grabber will work like this:
1. it starts grabbing epg for channel 1 (on MUX A)
2. timeout occurs, no epg found
3. it starts grabbing epg for channel 2 (on MUX A)
4. timeout occurs, no epg found
5. it starts grabbing epg for channel 3 (on MUX A)
6. suppose receives the epg for channels 3-10
7. it updates the database for channels 3-10

Next, it will see that
channels 3-10 are already updated and dont need grabbing
channels 1-2 need grabbing, but have just been tried. So it skips those
and will retry those after 2 hours again

So now it continues:
4. it starts grabbing epg for channel 21 (on MUX B)
5. suppose receives the epg for channels 21-40
6. it updates the database for channels 21-40
7. all epg has been received. epg grabber waits for 4 hours before starting again

Now you see that the epg grabber
- does not simply scans all channels in any mux (as you say it does).
- it skips channels for which epg has been received
- it skips channels for which epg has not been received or timed out
- meaning (in the uk situation) it moves to the next mux very quickly

Now about your solution. You assume that any channel on any MUX always
has the full EPG for all channels in that mux.
This maybe the case in the UK, but its surely not the case in many
other countries (netherlands/france/...)
In those countries a channel only contains its own epg
and perhaps the epg from a few other channels

The epg grabber should work with these muxes also.
You cannot say, hey 1 channel in this mux is done/failed, lets skip
all other channels of the mux


Futhermore you refer to the fact that the epg data may be incomplete.
Why would the epg data be incomplete? in all my tests the tvserver always
grabs all epg data available.

Frodo

 

[Broceliande]

I guess frodo didn't talked about performances ,wich you seem to be focused on .
There , where i live , your scenario won't grab EPG for most channels.
Cause there the EPG data isn't especially on the main mux , but can be on any channel from the same broadcaster.
Depending on the broacaster , there is EPG data for a channel only on the channel itself. A second channel , same broadcaster , will need to be tuned as well before getting EPG for it.
Some Broadcasters put all epg data for a same mux in a channel ( not always the same ).
So imo frodo's method is the right one for all ( a least the most ) users , that Mediaportal has to care of.
Yours won't work in many countries where actual one works : So question is , why should we change that if it make some users epg getting crap ?

 

[SciDoctor]

You are not taking into account worst case scenario which you should do with real time data grab .

The broadcsters don't guarantee that all data is present and error free.

- does not simply scans all channels in any mux (as you say it does).

It does if every channel times out WORST case scenario

- it skips channels for which epg has been received

In skipping a channel for data received there is NO guarantee that the EPG is now fully populated for that channel

- it skips channels for which epg has not been received or timed out
- meaning (in the uk situation) it moves to the next mux very quickly

Which is quicker (worst case) 200mins to grab from MUX A to get to MUX B or 10mins.

You could have grabbed 3 channels from 6 MUXs in the time it took to bash the living daylights out of MUX A and got no EPG data.

As the time is the same what ever the grab result the total time to grab from every channel (no missing cannles) is the same for both methods.

My proposed method cuts out wasted time on a MUX which may have no EPG data available because of errors and quickly covers the MUX's.

 

[SciDoctor]

I guess frodo didn't talked about performances ,wich you seem to be focused on .
There , where i live , your scenario won't grab EPG for most channels.
Cause there the EPG data isn't especially on the main mux , but can be on any channel from the same broadcaster.
Depending on the broacaster , there is EPG data for a channel only on the channel itself. A second channel , same broadcaster , will need to be tuned as well before getting EPG for it.
Some Broadcasters put all epg data for a same mux in a channel ( not always the same ).
So imo frodo's method is the right one for all ( a least the most ) users , that Mediaportal has to care of.
Yours won't work in many countries where actual one works : So question is , why should we change that if it make some users epg getting crap ?

Just because a channel is not being grabbed next doesn't mean it is not going to get grabbed.

OF COURSE IT WORKS, every channel is grabbed just the order of the grab is different to prioritise the least time to get to every MUX .

You obviously didn't read my post , I am not the one missing channels just because a grab occured .

If a MUX frequency is poor data wise for whatever reason EVERY channel on that MUX is affected .

If there are 100 channels and each channel has data just for itself then there is no avoiding that a channel will be garbbed the last whatever order you grab.

If there are 100 channels split over ten MUX each channel carries the data for every channel on that MUX the quickest way to get the most data is to try one channel form each MUX .

If you have a mix of the above two options then trying each MUX in turn is still the quickest way.