Scanning for new/updated channels results in a mess (2 Viewers)

doveman

Portal Pro
February 12, 2008
2,326
178
Home Country
United Kingdom United Kingdom
changing the unique ID from channel name to the 3 identifiers doesn't seem to make much sense, unless lots of other things have been changed or there are plans to do so.

Channel name is not ever unique. Same channel name can be received from multiple sources (-S, -T, -C, -IP) or broadcaster can even use the same channel name multiple times in the same broadcast type. Also broadcaster can change the channel name (rename it) and the channel content would be still the same.

I have seen all those to happen and name is not able to make unique channel identifier in those cases. DVB spec has already that issue sorted out, it just needs that both applications and broadcasters behave correctly. Now MP is complying the specs and if broadcaster isn't then we cannot unfortunately do anything about it.

I understand channel name is not unique, which is why it can't be used as the unique identifier and something else needs to be used. I think it's clear from my previous posts that I appreciate that. I can only assume that when channel name was used as the unique identifier however, that the actual tuning details (the three identifiers) must have been stored in a separate table and linked to the unique identifier. So if the unique identifier is now changed from channel name to the three identifiers, it would seem that there is some unneccessary duplication by having these details stored in two places, unless other changes have been made to do away with this separate table. If this separate table is still being used, it seems unneccessary to make the unique ID as complicated, when perhaps T-BBC1, T-ITV1 would suffice.

As for the DVB spec, that's a separate issue but if we know that broadcasters don't compy with it, it seems rather daft to ignore this and decide to make MP work on the assumption that broadcasters do follow it, as we know they don't and thus MP will encounter problems. I'm just trying to see if there's a better way to approach this that will make MP work better, even when broadcasters choose to ignore the spec. I think if the broadcaster changes the three identifiers but the channel name remains the same, MP could be made to be able to cope with this. It might not be possible for it to be able the handle broadcasters just changing the name, in which case perhaps the only solution is for the user to manually tell MP that channelname1 is now channelname2.
 

riksmith

Portal Pro
April 18, 2009
1,856
322
Home Country
Netherlands Netherlands
There is no such thing as separate table. We have two: One that stores all the channels. And one that stores all the tuningdetails (which has a foreign key to channel).

If providers are not sticking to the specs it is either:
User has to solve it on his own.
Or we try to add an exception to the normal flow. So: We are not going to base our normal flow on providers who define their own rules.

But first we would need an example of course of such a problem (and currently i do not have one).
 

doveman

Portal Pro
February 12, 2008
2,326
178
Home Country
United Kingdom United Kingdom
If you read my reply i never said i did not want to give answers. I already gave them to you: We follow the specs. That's it. The only reason you give is that it doesn't make sense to you. It works, and it makes sense to the dev team and the people who invented the DVB specs, so why should we bother arguing about it?

If you find something is WRONG with how we implemented it or if you run into a bug i am happy to discuss that with you or anybody else.

Your reply was very abrupt and didn't address any of the points I'd raised, so it seemed fair to assume you didn't want to answer them.

I think I've raised valid questions about whether blinding following the specs is the best way to make MP work as well as it could and suggested some alternative ways of working, not just said it doesn't make sense to me. I'm sorry if you think I'm arguing, rather than asking questions in an attempt to understand why things have been done a certain way and trying to suggest other possible options.

I'm not saying anything is necessarily wrong with the way things have been done, because there may well be very good reasons for it that I'm not aware of, but based on the information that's been provided it seems that there MIGHT be better alternatives. It may be that once someone's given me more information, I'll be able to appreciate that the best approach has been taken, but I think it's quite unfair and not in the spirit of Project MP to just say "it makes sense to us, so why should we have to explain or discuss it".
 

riksmith

Portal Pro
April 18, 2009
1,856
322
Home Country
Netherlands Netherlands
As for the DVB spec, that's a separate issue but if we know that broadcasters don't compy with it, it seems rather daft to ignore this and decide to make MP work on the assumption that broadcasters do follow it, as we know they don't and thus MP will encounter problems. I'm just trying to see if there's a better way to approach this that will make MP work better, even when broadcasters choose to ignore the spec. I think if the broadcaster changes the three identifiers but the channel name remains the same, MP could be made to be able to cope with this. It might not be possible for it to be able the handle broadcasters just changing the name, in which case perhaps the only solution is for the user to manually tell MP that channelname1 is now channelname2.

Even if the name stays the same we cannot use that. On your provider it works. But mine has 4 different channels with exactly the same name.
 

riksmith

Portal Pro
April 18, 2009
1,856
322
Home Country
Netherlands Netherlands
If you read my reply i never said i did not want to give answers. I already gave them to you: We follow the specs. That's it. The only reason you give is that it doesn't make sense to you. It works, and it makes sense to the dev team and the people who invented the DVB specs, so why should we bother arguing about it?

If you find something is WRONG with how we implemented it or if you run into a bug i am happy to discuss that with you or anybody else.

Your reply was very abrupt and didn't address any of the points I'd raised, so it seemed fair to assume you didn't want to answer them.

I think I've raised valid questions about whether blinding following the specs is the best way to make MP work as well as it could and suggested some alternative ways of working, not just said it doesn't make sense to me. I'm sorry if you think I'm arguing, rather than asking questions in an attempt to understand why things have been done a certain way and trying to suggest other possible options.

I'm not saying anything is necessarily wrong with the way things have been done, because there may well be very good reasons for it that I'm not aware of, but based on the information that's been provided it seems that there MIGHT be better alternatives. It may be that once someone's given me more information, I'll be able to appreciate that the best approach has been taken, but I think it's quite unfair and not in the spirit of Project MP to just say "it makes sense to us, so why should we have to explain or discuss it".

I already replied to your suggestion. It seems to involve the channel name (i am sorry if i missed a suggestion, please point me to it then) and as said in my previous post the name is not unique within a provider.
 

doveman

Portal Pro
February 12, 2008
2,326
178
Home Country
United Kingdom United Kingdom
There is no such thing as separate table. We have two: One that stores all the channels. And one that stores all the tuningdetails (which has a foreign key to channel).

If providers are not sticking to the specs it is either:
User has to solve it on his own.
Or we try to add an exception to the normal flow. So: We are not going to base our normal flow on providers who define their own rules.

But first we would need an example of course of such a problem (and currently i do not have one).

I don't understand how you can say there's no such thing as a separate table and then say there are two :confused:

Is the table that stores the channels the one which consists of what we've been calling the unique ID? Which used to be just channel name, but has now been changed to the three identifiers? And does the table that stores the tuningdetails not consist of (in part at least) the three identifiers? If so, it seems there is some unnecessary duplication happening, which is what I'm questioning.

I think jameson_uk gave a good example of the problem when he explained that:
Previously ITV2 + 1 had
ONID = 9018
TSID = 8198
SID = 8362

but the details are now
ONID = 9018
TSID = 12290
SID = 15952

so as far as MP is concerned a brand new channel.

So it seems that by relying on the three identifiers, MP will get confused, whereas if it looked at the channel name, it would be able to see that ITV2+1 was now using different identifers and any scheduled recordings for that channel would proceed without problem.

This sort of thing might not happen very often, but it seems that it wouldn't be that hard to make MP able to cope with it. It seems like sticking your head in the sand to say "We are not going to base our normal flow on providers who define their own rules." as ultimately, if broadcasters are going to choose to ignore the DVB specs and MP can be made to be able to accomodate what the broadcasters are doing, it seems only sensible to do so.

Having said that, if this sort of thing only happens very rarely, and it would require a lot of work to make MP be able to cope with it, I appreciate if no-one wants to spend their time doing so. Perhaps the time would be better spent implementing an easy way for users to manually tell MP when a channel has moved/changed name (which probably happens more often than a channel being assigned new identifiers), so that scheduled recordings work.

As for the DVB spec, that's a separate issue but if we know that broadcasters don't compy with it, it seems rather daft to ignore this and decide to make MP work on the assumption that broadcasters do follow it, as we know they don't and thus MP will encounter problems. I'm just trying to see if there's a better way to approach this that will make MP work better, even when broadcasters choose to ignore the spec. I think if the broadcaster changes the three identifiers but the channel name remains the same, MP could be made to be able to cope with this. It might not be possible for it to be able the handle broadcasters just changing the name, in which case perhaps the only solution is for the user to manually tell MP that channelname1 is now channelname2.

Even if the name stays the same we cannot use that. On your provider it works. But mine has 4 different channels with exactly the same name.

OK, that clearly complicates things then. What source is that (DVB-S?). And it has 4 channels with exactly the same name? Sorry, I just find it bizarre (stupid even) that a broadcaster would give the same name to 4 different channels! Can I ask what happened before, when MP was using the channel name as the unique ID, and you tried to record one of those channels? Did it just pick one at random?
 

riksmith

Portal Pro
April 18, 2009
1,856
322
Home Country
Netherlands Netherlands
We are not sticking our head in the sand because we are open to making exceptions to the normal flow. But it is a very bad practice (in general programming) to base your NORMAL code on exceptional situations: in this case providers who like to invent there own rules. So if you can come up with a solution that does not affect users who have behaving providers, we are happy to look at it. But so far i have not seen one.

The reason i said that there is no such table is because there have been no modifications to the database for this change and the three id's are in tuningdetails as they have always been.

The channel has a unique id just as every object would have in a normal database. This has nothing to do with what is found on the network and what we are currently discussing. Basically (there are more fields but they are less important) a channel is just : id, name.

My source: DVB-C. Before: I would endup with losing 2 of the channels. It scanned 4 but i ended up with 2. (And i don't mean they were grouped or something, they were literally gone). And about the naming: I don't see a reason why we should call that stupid.

I hope i answered everything now that is in the last posts.
 

doveman

Portal Pro
February 12, 2008
2,326
178
Home Country
United Kingdom United Kingdom
We are not sticking our head in the sand because we are open to making exceptions to the normal flow. But it is a very bad practice (in general programming) to base your NORMAL code on exceptional situations: in this case providers who like to invent there own rules. So if you can come up with a solution that does not affect users who have behaving providers, we are happy to look at it. But so far i have not seen one.

I imagine it's also bad practice to ignore very real and known possibilites (even if they are exceptional) that will cause the program to malfunction (in the sense that the user will experience problems). I'm not suggesting that the code should be written to cope with these situations in a way that would cause problems the rest of the time and perhaps there is no way to code it so that this doesn't happen. I'll see if I can suggest a workable way to handle this, once I further understand how things currently work.

The reason i said that there is no such table is because there have been no modifications to the database for this change and the three id's are in tuningdetails as they have always been.

But from what I understand, the three id's are now also used for the unique id as well, so that they're stored in two separate locations. Maybe table is the wrong word, in which case please tell me which word is more accurate so that we understand each other.

The channel has a unique id just as every object would have in a normal database. This has nothing to do with what is found on the network and what we are currently discussing. Basically (there are more fields but they are less important) a channel is just : id, name.

I don't see how the unique id can be "nothing to do with what is found on the network" if it consists of the three identifiers and the channel name, which are all pulled from the network. If I understand correctly, each channel is assigned a unique ID, consisting of the three identifiers (ie the tuning details) and the channel name, and this unique ID also links to a separate location which also stores the three ids (the tuning details). Is this correct?

My source: DVB-C. Before: I would endup with losing 2 of the channels. It scanned 4 but i ended up with 2. (And i don't mean they were grouped or something, they were literally gone). And about the naming: I don't see a reason why we should call that stupid.

It seems stupid for a broadcaster to give several channels the same name, simply because it looks strange and confusing to have several channels with the same name in the EPG. And imagine the conversation in the office "Did you see that programme on ChannelHistory last night", "Oh, which one, there's four" :D

Perhaps, when scanning, if MP picks up a channel with the same name but different tuningdetails as a channel already picked up and entered in the database, it could assign a suffix to the name to store as the unique ID. So that you'd have ChannelHistory-a, ChannelHistory-b, etc.

As for dealing with channel names changes, perhaps if the channel name is stored with the scheduled recordings data, then after a re-scan MP could look to see if there were any scheduled recordings on channel names that no longer existed. Then it could either scan the programme names in the EPG (once updated) to see if it could find the scheduled recordings, thus automatically identifying the new channel name, or it could just notify the user that the channel name no longer existed and ask them to manually locate the new channel name and assign it to those recordings (or tell MP that the channel has been taken off-air and to discard those scheduled recordings).
 

riksmith

Portal Pro
April 18, 2009
1,856
322
Home Country
Netherlands Netherlands
I do not know where you get from that the three id's are stored in two locations. They are not. They are only stored in the tuningdetails.

A channel has no three id's assigned from the network. The channel ID is a generated unique number (1, 2 , 3 etc).
I think your are confusing the concept of channels and tuningdetails here.

What you have on your network is a tuningdetail.
On the other hand you have channels. When you only have one source for your channels you will end up with one tuningdetail belonging to one channel.
So you would get:

Code:
Tuningdetail
ID: 1
ONID: 500
TSID: 6
SID: 5091
NAME: ITV 1+1
TYPE: DVB-T
CHANNEL: 1

Which results in a channel:
Code:
ID: 1
NAME: ITV 1+1

If you have multiple sources you would (not automatically) end up with the same channel but it will have an extra tuningdetail:

Code:
Tuningdetail
ID: 2
ONID: 6120
TSID: 4
SID: 124
NAME: ITV 1-1
TYPE: DVB-S
CHANNEL: 1

What used to happen when we scanned for channels was that we tried to locate an existing tuningdetail and channel in order to update it. That was name based. I am not entirely sure which two of the names it used, but i think the one of channel.

What we do now is try to find a tuningdetail with the same ONID +SID + Type. Note that we do not use the TSID here because this will allow us to detect when a channel moves between transponders. If we find such a tuningdetail we know which tuningdetail we need to update. The reason was already explained.

If we do not find such a tuningdetail we assume this is a new channel. So we make a new tuningdetail and a new channel.
This could mean (and we have the example from jameson_uk) that we update the wrong tuningdetail. In that case it currently is up to the user to solve this. We can not think of a fool proof automatic way to handle this (and you have also not found that). It is not for nothing that the DVB specs have put in that advice i quoted earlier. That is meant to prevent having the problems you are now facing.

On other places where we need to find the current tuningdetail that belongs to the info we have from tuning we use the ONID +TSID + SID + Type.

Why are you referring to the schedules? Is there some problem with that? Schedules are couple to the channel on its ID.

In our country we the concept of the LCN. So the channels with the same name have a different LCN. 401, 402, 404, 405 or something like that.

I hope i cleared up how the concept is working.
 

doveman

Portal Pro
February 12, 2008
2,326
178
Home Country
United Kingdom United Kingdom
I do not know where you get from that the three id's are stored in two locations. They are not. They are only stored in the tuningdetails.

A channel has no three id's assigned from the network. The channel ID is a generated unique number (1, 2 , 3 etc).
I think your are confusing the concept of channels and tuningdetails here.

Thanks for the detailed explanation. I'll study it properly later.

I think my confusion is quite understandable. At 0002128: Improve TVE3 channel identification - MediaPortal Bugtracker it says
"Currently we are using channel name as "unique" identifier. This is not enough as there is possibility for conflicting channels:
We should start to use more unique identifier for the channels (maybe channel and broadcaster IDs combined would be the correct way, needs some study..."


So from this I took that the unique ID was just channel name, therefore the tuningdetails must be stored elsewhere. I understood the second part of that statement to mean that consideration was being given to using the tuningdetails (or 3 identifiers) as the unique ID, instead of channel name.

Then at https://forum.team-mediaportal.com/...els-results-mess-88294/index2.html#post702041 you said: "So our unique identifier is: ONID + TSID + SID + channelType."

So putting those together I understood that the unique ID was previously just channel name, which meant that the tuningdetails must be stored somewhere else, and then the unique ID was changed to be the tuningdetails + channelType, which would have resulted in the tuningdetails being stored in two places.

Now you're talking about a channel ID, which is actually just a single number and not the tuningdetails + channelType. So is it that there's three values, stored separately but linked, the channel ID (1,2,3,etc), the channelname, and the tuningdetails, and the unique ID is generated from those? In fact, I assume the channelType must be part of the tuningdetails, in which case the unique ID would just be the tuningdetails, no?

Regarding the Channel ID, I can see these in TVServer config, eg if I select ITV2 and click Edit Channel it shows the ID number pulled off the network (in this case 8), but if I click Edit again, it shows the tuningdetails and at the top Channel: 6. Actually, it looks like I've got those the wrong way round, and the ID number shown on the first screen is the MP generated one, whilst the Channel number shown on the next screen is the one pulled off the network.

So is it the case that currently the Channel number pulled off the network is not used by MP at all. I mean clearly it's stored as part of the tuningdetails, but does any process actually reference this?

I've just done a re-scan to update after the ITV switcharound, so I might as well illustrate how complicated it is at the moment. Whether there's any way to make it simpler is a different matter :)

So firstly, I don't want to delete all the channels before scanning, as then I'd have to spend quite some time re-assigning the channels I actually want to see in the EPG to my Main group, getting them in the right order and mapping them to both my tuners. I suppose what some PVRs might do is to backup these details, clear the list, scan and then restore the list from the backup (except for any channels that were no longer present), adding any new channels at the bottom.

So I do a scan and it finds a new ITV1+1 and ITV2+1 (or it finds a channel with that name but new tuningdetails). So now I've got two ITV2+1 in the list and no idea which one is the correct one. First thing to do is delete the existing one from my Main group and add ITV1+1 to that group.

So now I've got to identify which ITV2+1 to keep and which to delete. This is actually easier to do from the All Channels tab rather than Channels, as it shows the channel numbers next to the channel names. This example isn't too hard to work out, because the information at Freeview - Home Resolutions About Channels Channel changes ITV1+1 to launch on 11 January 2011 tells me that the new channel ITV1+1 is on channel 33 and therefore I should be right in deleting the entry for ITV2+1 assigned to channel 33 under the All Channels tab.

It would be nice if there was a way to avoid all this having to look at webpages to get the information and manual editing though.
 

Users who are viewing this thread

Top Bottom