Any way to protect existing channels when scanning a new satellite? (1 Viewer)

[clarkebelt]

I am having a really strange issue that occurs whenever I attempt to scan the channels on a staellite. Just for reference, I have a 4 port DiSEqC switch, and one port of the DiSEqC switch has a 22 kHz tone switch hanging off of it. At present I have three C-band dishes and one Ku-band dish, each pointed at different satellites.

The problem is that each time I have added another dish to the mix, every time I try to scan the new satellite it invariably overwrites my good data in my existing channels with bad data from the satellite it is scanning. For example, today I added in a C-band dish on port 1 of the DiSEqc switch. When adding it to MediaPortal I only selected that single DiSEqC switch port, and selected C-Band. Yet when it scanned in the channels, it overwrote (or "updated") several existing channels. It did this regardless of the fact that:

• The channels it overwrote were on completely different satellites
• The channels it overwrote were on different frequencies
  
• That it overwrote good in-the-clear channels with data for scrambled channels
• That in one case, the channel was on a completely different band and frequency - it overwrote a Ku-band channel with data for a C-band channel!

I just cannot understand why this happens, but it has happened often enough that I'd like to find a solution or workaround. I think it's probably related to an earlier issue I had, where good data on a single transponder on a satellite was being overwritten by data from a different transponder on the same satellite.

Is there any way to tell MediaPortal that I am scanning a new satellite and that it should not be overwriting channels on any other satellite? I would think this should be how things work by default but apparently that's not the case.

Alternately, is there any way to back up my channels and then restore them later if things go amiss?

 

[mm1352000]

Hello again

I just cannot understand why this happens, but it has happened often enough that I'd like to find a solution or workaround. I think it's probably related to an earlier issue I had, where good data on a single transponder on a satellite was being overwritten by data from a different transponder on the same satellite.

Almost certainly the ONID+TSID+SID thing that I explained to you previously.

Is there any way to tell MediaPortal that I am scanning a new satellite and that it should not be overwriting channels on any other satellite?

In short: no.

Alternately, is there any way to back up my channels and then restore them later if things go amiss?

You could try the import/export section in TV Server configuration.

mm

 

[clarkebelt]

Hello again

I just cannot understand why this happens, but it has happened often enough that I'd like to find a solution or workaround. I think it's probably related to an earlier issue I had, where good data on a single transponder on a satellite was being overwritten by data from a different transponder on the same satellite.

Almost certainly the ONID+TSID+SID thing that I explained to you previously.

Is there any way to tell MediaPortal that I am scanning a new satellite and that it should not be overwriting channels on any other satellite?

In short: no.

Can I ask, WHY? I'm not asking why this happens, you have explained that to me. What I am trying to figure out is under what conditions this sort of operation would make any sense. Under what circumstances would it be valid for data from a channel on a completely different satellite, on a different frequency and in a different band, to overwrite an already scanned channel's data? I get it that this is the way the program works, but in my mind I cannot conceive of any logical reason it should work this way.

I understand that the way Free-To-Air satellite in Europe works is quite different from the way it works in North America, but even so, would this even be a valid way to do things over there? And are we in North America doomed to never have a version of MediaPortal that will scan a satellite without destroying previously stored channels? It's hard to fully describe the sinking feeling that occurs in the pit of my stomach because I finally got all my stored channels the way I wanted them, only to have them overwritten because I scanned another satellite.

Personally I wish there was some way to access whatever database holds the channel list directly (without using the TV Server Configuration program) because then at least I could take "snapshots" of the raw data, and maybe find a way to quickly restore overwritten channels after a scan.

 

[mm1352000]

Can I ask, WHY? I'm not asking why this happens, you have explained that to me. What I am trying to figure out is under what conditions this sort of operation would make any sense.

1. To avoid creating duplicate channels in rescans.
   
2. To update tuning details when channels move.

Under what circumstances would it be valid for data from a channel on a completely different satellite, on a different frequency and in a different band, to overwrite an already scanned channel's data?

When a channel moves (and yes channels can move from satellite to satellite).

...but even so, would this even be a valid way to do things over there?

Yes.

And are we in North America doomed to never have a version of MediaPortal that will scan a satellite without destroying previously stored channels?

As far as I am aware FTA satellite broadcast over America is like the wild west - proprietary, and anything goes. If that is true then the answer is possibly yes. I mean, put yourself in our position: how are we (based in Europe and other parts of the world) meant to develop support for proprietary or non-standard-compliant systems that we don't even have access to for testing? Having said that, I'm always keen to improve. If you can devise and explain a [better] way to identify channels that is compatible with DVB and/or that enables us to avoid creating duplicates on rescans, pick up channel movements, and also avoid incorrect channel overwriting/updating then I'm seriously all ears.

Personally I wish there was some way to access whatever database holds the channel list directly (without using the TV Server Configuration program)...

There is, using standard MySQL database manipulation tools such as MySQL Workbench.

...because then at least I could take "snapshots" of the raw data, and maybe find a way to quickly restore overwritten channels after a scan.

That is basically what the purpose of the import/export section is. You can export and import XML (which is just a formatted text file).

mm

 

[mm1352000]

For anybody that is interested, the DVB registration for network and original network IDs is here:
http://www.dvbservices.com/identifiers/network_id
http://www.dvbservices.com/identifiers/original_network_id

(Click "allocation table" under the heading.)

The country code list that goes with it is here:
http://www.dvbservices.com/country_codes/

As an example: in previous log files from clarkebelt, many of the services were using ONID/NID 1 or 2 which are registered globally by "Société Européenne des Satellites". By the looks of the network names they're intended for use only on Astra 19.2E or 28.2E. Why on Earth would providers on the other side of the world use such registered IDs? I can see from the logs that they're broadcasting DVB network information tables (!!!), so it isn't even like they could say they're using a different standard so the DVB rules don't apply.

 

[clarkebelt]

And are we in North America doomed to never have a version of MediaPortal that will scan a satellite without destroying previously stored channels?

As far as I am aware FTA satellite broadcast over America is like the wild west - proprietary, and anything goes. If that is true then the answer is possibly yes. I mean, put yourself in our position: how are we (based in Europe and other parts of the world) meant to develop support for proprietary or non-standard-compliant systems that we don't even have access to for testing? Having said that, I'm always keen to improve. If you can devise and explain a [better] way to identify channels that is compatible with DVB and/or that enables us to avoid creating duplicates on rescans, pick up channel movements, and also avoid incorrect channel overwriting/updating then I'm seriously all ears.

First, let me try to explain the situation here, with the caveat that I am not employed in the satellite industry so some of this is my observations. I've had a satellite dish for close to 20 years now and in that time I've picked up a little knowledge.

Your characterization of FTA satellite broadcast over North America as being like the wild west isn't that far off. The major difference is this: My understanding is that in Europe, broadcasters intend for people to receive FTA satellite broadcasts, so they do things to make FTA user-friendly, like adhering to certain standards.

In North America, however, there are few to no satellite signals that are broadcast with the intent that the end viewer will receive them directly. If there are a few, they are primarily religious channels that would like home viewers to tune in so they can watch and (they hope) respond to appeals for money. But the bulk of the television transmissions on satellite are there for one of two reasons. Either it is a television network (large or small) sending signals to their local affiliate stations for rebroadcast, or it is a cable channel sending their signals to Cable TV headends. In either case, all they care about is whether the intended recipients can receive the signals, and in such cases the uplinker often supplies the intended recipients with a proprietary receiver. Most such transmissions are in fact scrambled, but enough aren't to make it worthwhile to have a dish (or two or three) if you live far enough out in the country that the local busybodies won't hassle you about it.

The law in the United States is that as long as the signal is not scrambled you are free to receive and watch it, even if it's not intended for you. The presumption is that if the uplinker doesn't want just anyone with a satellite receiver to receive their signal, they should scramble it. It's very illegal to decode a scrambled signal if you are not paying for the service. So most of the major cable channels do scramble their signals most of the time, and there is NO option to purchase them if you are receiving them on C or Ku band (there used to be for some of the channels many years ago, but now they have migrated everyone that wants to pay for TV to the very small dish services on Ka band, and there is NO free-to-air on the Ka satellites - you may find a "barker" channel for a particular satellite provider in the clear on Ka, but nothing anyone would want to watch day in and day out).

So what you are left with on C and Ku bands is a royal mess. There is no real consistency in anything. On any given satellite, you can have scrambled channels next to data channels next to free-to-air (unscrambled) channels. Saying they are non-standard-compliant is putting it mildly - the technicians that set these up probably aren't even aware there ARE any standards, and whatever they may be, they aren't followed over here because the government regulatory agencies don't require it. I don't mean there are no rules whatsoever - obviously satellites must maintain their proper orbital location, and they cannot transmit outside of the normal frequency bands assigned for C or Ku band. But other than that, it appears that the satellite owners are pretty much free to assign frequencies on their satellites any way they like. And if you are an uplinker and you have paid for time on a satellite (whether that be only a few hours or 24/7 use of a particular channel), you are pretty much free to uplink using any format you like, as long as you stay within your assigned frequency range.

And there is no one C or Ku Band satellite that is assigned to a single purpose, such a television or data or radio. You can find all kinds of signals on all the satellites, but on any one satellite there will probably be relatively few free-to-air signals.

Another thing different from Europe is that virtually none of these channels transmit EPG data. Again, the signals aren't intended for direct reception by home viewers, and the equipment used to receive the signals at TV stations and cable headends wouldn't know what to do with guide data. I sometimes think that the only reason everything isn't scrambled is because of the expense, and the fact that many of the smaller stations and cable companies have no one on their engineering staff capable of tracking down and fixing an issue with the descrambling equipment. So if something goes wrong, that station or cable headend can be without signal for a week or more, plus the uplinker has to pay to send out new equipment to the station or headend site, even though the only real problem may be that a station technician or employee twisted the wrong knob, or entered an incorrect value.

In your other post, you wrote:

As an example: in previous log files from clarkebelt, many of the services were using ONID/NID 1 or 2 which are registered globally by "Société Européenne des Satellites". By the looks of the network names they're intended for use only on Astra 19.2E or 28.2E. Why on Earth would providers on the other side of the world use such registered IDs? I can see from the logs that they're broadcasting DVB network information tables (!!!), so it isn't even like they could say they're using a different standard so the DVB rules don't apply.

My guess is that they are working off some kind of example configuration that contains those numbers. Or, I can easily imagine that some of those uplinks were configured by guys who had no idea what that field was for, so they just tried "1" and it worked, and getting it working was all they cared about. Who cares about a global registration if nobody is going to enforce it? It's only a concern if they are sending a signal to a satellite that's far enough out over the Atlantic that it might actually be received in Europe. The DVB rules don't apply to them, probably for the simple reason that neither the government regulators nor the satellite owner cares about enforcing them.

So what is the solution? For North America it's simple: You simply assume that unless a channel is an EXACT duplicate - same satellite, same frequency, exact same information - then it is NOT a duplicate or moved channel. 99% of the time that would be a correct assumption in North America. Channels rarely move, except when an old satellite is failing and they are forced to move (and nowadays they often just launch a new satellite and put it in the same orbital position and hot swap it). What's a lot more common is for an unscrambled signal to suddenly become scrambled, though occasionaly the opposite happens - a previously scrambled signal becomes unscrambled.

My suggestion would be to have a "North America" checkbox somewhere in the channel scan configuration, and if that's checked then you assume that no matter how much two channels may look alike, if they are not on the EXACT same frequency and satellite, then they are different channels. That's the best idea I can come up with this late.

 

[clarkebelt]

Hello again mm1352000, thank you for the explanation. I am more awake now and am trying to understand what you said about the import/export. I think this might at least be a workaround to solving the problem I'm having here but I need to be clear about how it works.

...because then at least I could take "snapshots" of the raw data, and maybe find a way to quickly restore overwritten channels after a scan.

That is basically what the purpose of the import/export section is. You can export and import XML (which is just a formatted text file).

Just so I am clear on this (because it seems like every time I make an assumption about something it's wrong), let's say I did this:

• Exported just my channels and groups
• Performed a scan on a new satellite
• Imported my channels and groups back from the previous export

Would I be correct in assuming that would restore all my channels and groups back to the way they were before I performed the scan? And if there was non-conflicting data from the new scan, would that be preserved or erased when I re-imported the old data? In other words, does an import of channel and group data first erase any existing channel and group data, or does it attempt to merge the imported data into what's currently in the database?

 

[mm1352000]

Would I be correct in assuming that would restore all my channels and groups back to the way they were before I performed the scan?

Import can be a mixture of add and update, so when you say "restore... back to the way they were" I would probably use other terms.

Like the scan process, the import process also tries to avoid creating duplicate channels. This is to handle the scenario where TV Server has channels before import and there is some overlap between the channels in the XML file and the channels in the TV Server database. The crucial difference between scanning and import is that import uses the channel name (instead of ONID + TSID + SID) for matching. Therefore you have more control over what happens.

If all of your exported and "current" (ie. existing in TV Server at time of import) channels have unique names then there should be no merging/updating. The result of such an import would be A + B where A is TV Server's channels before import and B is the channels defined in the XML file. If some channels in the XML file have the same name as existing channels in TV Server at time of import then the result of the import will be less than A + B, because the channels with matching names will be merged.

And if there was non-conflicting data from the new scan, would that be preserved or erased when I re-imported the old data?

Data is never erased. In other words, the import process doesn't delete all existing channels before importing. However as explained above: if channels do not have unique names then you will experience similar (but not quite the same - merging as opposed to updating) issues as you're experiencing with the scan.

Merge (import) = add tuning detail.
Update (scan) = update existing tuning detail in-place.

You don't want either of these things to happen, so simply ensure all your channels have unique names.

In other words, does an import of channel and group data first erase any existing channel and group data...?

Categorically no.
Import does not delete.
Export does not delete.

...or does it attempt to merge the imported data into what's currently in the database?

Yes, using channel and group names.

One recommendation I would make to add to your process: delete all existing channels after export and before starting the next scan. Reduces the chance of getting channels mixed up.