[MP1.11 TV Server] Can't scan Fritz! DVB-C Wlan Repeater M3U (1 Viewer)

[Nervesagent]

Here's the logs and the story

1-Fresh start of tv-service
2-Scan Fritz.m3u
3-Channels are scanned, many channels are "updated" few are found
4-Manually canceled scan after lines in M3U
5-Result is uncorrelated new channel displaynames and tunernames (I don't really know what the db fields are called at this point, but just see the screenshots)
Also some tv channels are recognized as radiochannels?
6-Tried previewing some HD and SD channel, no pmt found
7-Tried altering one channel's tuningdetails, so it would have rtsp://192.168.1.65/?blabla, pmtid 4096 (I don't know why I chose this nr. but i think it was a suggestion elsewhere) other id's: 1 Result: No PMT found

I have no clue why these channels would work with ffmpeg -copy -mpeg2 in between, because this seems to change nothing to the input.
Where do I look?

 

[mm1352000]

Your results are as expected. You cannot successfully scan multi-channel/program/service transport streams using the DVB-IP tuner if PID filtering is active and allows the SDT (PID 17).

The reason is that when you scan each channel, the other channels from the same transport stream will be found and saved with an incorrect URL.

For example, scanning the first channel/URL:
[2015-12-03 09:42:53,233] [Log ] [23 ] [INFO ] - dvbip: TuneVBIP:tv: Nickelodeon HD Freq:0 ONID:-1 TSID:-1 SID:-1 PMT:0xFFFFFFFF FTA:False LCN:10000 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511

This will find all channels from the Nickelodeon HD transport stream:
[collapse]
[2015-12-03 09:43:48,578] [Log ] [23 ] [INFO ] - 0) 0x600 0x807 0x467A 0x1F6 Q-Music type:2
[2015-12-03 09:43:48,586] [Log ] [23 ] [INFO ] - Found: DVBIP:radio:Ziggo Q-Music Freq:0 ONID:1536 TSID:2055 SID:18042 PMT:0x1F6 FTA:True LCN:808 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,590] [Log ] [23 ] [INFO ] - 1) 0x600 0x807 0x467B 0x1F8 Sublime FM type:2
[2015-12-03 09:43:48,594] [Log ] [23 ] [INFO ] - Found: DVBIP:radio:Ziggo Sublime FM Freq:0 ONID:1536 TSID:2055 SID:18043 PMT:0x1F8 FTA:True LCN:812 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,598] [Log ] [23 ] [INFO ] - 2) 0x600 0x807 0x467C 0x1FA Arrow Classic Rock type:2
[2015-12-03 09:43:48,603] [Log ] [23 ] [INFO ] - Found: DVBIP:radio:Ziggo Arrow Classic Rock Freq:0 ONID:1536 TSID:2055 SID:18044 PMT:0x1FA FTA:True LCN:811 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,607] [Log ] [23 ] [INFO ] - 3) 0x600 0x807 0x467D 0x1FC BNR Nieuwsradio type:2
[2015-12-03 09:43:48,610] [Log ] [23 ] [INFO ] - Found: DVBIP:radio:Ziggo BNR Nieuwsradio Freq:0 ONID:1536 TSID:2055 SID:18045 PMT:0x1FC FTA:True LCN:815 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,612] [Log ] [23 ] [INFO ] - 4) 0x600 0x807 0x467E 0x1FE Classic FM type:2
[2015-12-03 09:43:48,615] [Log ] [23 ] [INFO ] - Found: DVBIP:radio:Ziggo Classic FM Freq:0 ONID:1536 TSID:2055 SID:18046 PMT:0x1FE FTA:True LCN:814 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,618] [Log ] [23 ] [INFO ] - 5) 0x600 0x807 0x467F 0x200 Radio Decibel type:2
[2015-12-03 09:43:48,621] [Log ] [23 ] [INFO ] - Found: DVBIP:radio:Ziggo Radio Decibel Freq:0 ONID:1536 TSID:2055 SID:18047 PMT:0x200 FTA:True LCN:818 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,624] [Log ] [23 ] [INFO ] - 6) 0x600 0x807 0x4680 0x202 SLAM! type:2
[2015-12-03 09:43:48,627] [Log ] [23 ] [INFO ] - Found: DVBIP:radio:Ziggo SLAM! Freq:0 ONID:1536 TSID:2055 SID:18048 PMT:0x202 FTA:True LCN:813 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,629] [Log ] [23 ] [INFO ] - 7) 0x600 0x807 0x4A61 0x125C NPO 1 type:1
[2015-12-03 09:43:48,632] [Log ] [23 ] [INFO ] - Found: DVBIP:tv:Ziggo NPO 1 Freq:0 ONID:1536 TSID:2055 SID:19041 PMT:0x125C FTA:True LCN:1 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,635] [Log ] [23 ] [INFO ] - 8) 0x600 0x807 0x4A62 0x1388 NPO 2 type:1
[2015-12-03 09:43:48,638] [Log ] [23 ] [INFO ] - Found: DVBIP:tv:Ziggo NPO 2 Freq:0 ONID:1536 TSID:2055 SID:19042 PMT:0x1388 FTA:True LCN:2 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,641] [Log ] [23 ] [INFO ] - 9) 0x600 0x807 0x4A63 0x14B4 NPO 3 type:1
[2015-12-03 09:43:48,644] [Log ] [23 ] [INFO ] - Found: DVBIP:tv:Ziggo NPO 3 Freq:0 ONID:1536 TSID:2055 SID:19043 PMT:0x14B4 FTA:True LCN:3 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,646] [Log ] [23 ] [INFO ] - 10) 0x600 0x807 0x4A64 0xB54 Ziggo Sport type:1
[2015-12-03 09:43:48,649] [Log ] [23 ] [INFO ] - Found: DVBIP:tv:Ziggo Ziggo Sport Freq:0 ONID:1536 TSID:2055 SID:19044 PMT:0xB54 FTA:True LCN:10000 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,652] [Log ] [23 ] [INFO ] - 11) 0x600 0x807 0x4A66 0x1838 Nickelodeon HD / Spike Channel type:19
[2015-12-03 09:43:48,655] [Log ] [23 ] [INFO ] - Found: DVBIP:tv:Ziggo Nickelodeon HD / Spike Channel Freq:0 ONID:1536 TSID:2055 SID:19046 PMT:0x1838 FTA:False LCN:937 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,658] [Log ] [23 ] [INFO ] - 12) 0x600 0x807 0x4A67 0xED8 2M type:1
[2015-12-03 09:43:48,661] [Log ] [23 ] [INFO ] - Found: DVBIP:tv:Ziggo 2M Freq:0 ONID:1536 TSID:2055 SID:19047 PMT:0xED8 FTA:True LCN:77 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
[2015-12-03 09:43:48,664] [Log ] [23 ] [INFO ] - Scan Got 13 from 13 channels[/collapse]

However, only one channel - the one that comes on PIDs 20,6200,3501,3511 - can work due to the PID filter configuration in the URL. The PMT, video, audio and subtitle PIDs for all the other channels are blocked, therefore you would get a "no PMT" error with them.

I have no clue why these channels would work with ffmpeg -copy -mpeg2 in between, because this seems to change nothing to the input.

Presumably that changes the transport stream from multi-program to single-program (ie. removes the information about all the other channels) so each scan only finds the one channel that is enabled by the PID filter.

The best way to enable successful scanning would be to disable the PID filter (ie. pids=all)... however we know that your tuner does not support that. So, I think the only way to use the tuner with TV Server currently would be to somehow import the channels directly without scanning.

 

[Nervesagent]

Your results are as expected... .

Exactly what I was going for...

Found: DVBIP:tv:Ziggo Nickelodeon HD / Spike Channel Freq:0 ONID:1536 TSID:2055 SID:19046 PMT:0x1838 FTA:False LCN:937 Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511

In your example for Nickelodeon HD this would mean manually creating a channel with stream:
Url:rtsp://192.168.1.65:554/?freq=346&bw=8&msys=dvbc&mtype=256qam&sr=6900&specinv=0&pids=0,16,17,18,20,6200,3501,3511
Network ID: 1536
Transport ID: 2055
Service ID: 19046
PMT: 6200 (HEX, just convert it to decimal)
Provider: whatever
Preview now gives me ChannelIsScrambled, could be because this one uses HEC264 or something because it's HD?

Tested another SD Stream
NPO1:

Network ID: 1536
Transport ID: 2055
Service ID: 19041
PMT: 4700 (HEX, just convert it to decimal)
Provider: whatever
First preview: No PmT found,

Copied over the URL again from my fritz.m3u
Second preview... SUCCES!!

Not sure exactly why it does now, but at least I'm sure I can get it to work properly if I take a little more time to dig in to it.

I'm at work right now so I won't be able to manually add them right now, but It seems to work like this.
Thanks for giving me the clues to where to look for the right values.

For the rest of the channels I'll make use of this website that should have the correct values:
http://www.dtvmonitor.com/nl/ziggo-zuid

Another way would be to just have MePo scan all channels and I'll go through the logs to sniff out the ID-values.

THANK YOU SOOO MUCH! MY WIFE WILL HATE YOU FOR HAVING ME MESS WITH MEPO FOR A FEW MORE HOURS!
I'll get back to you later once I've done all this.
Thanks again!

 

[mm1352000]

In your example for Nickelodeon HD this would mean manually creating a channel with stream:

Yes, correct.

Preview now gives me ChannelIsScrambled, could be because this one uses HEC264 or something because it's HD?

No. "ChannelIsScrambled" means exactly what it says. The video and audio are encrypted. You can't view or record the channel unless you can decrypt the channel, which requires a subscription to the cable provider.

Copied over the URL again from my fritz.m3u
Second preview... SUCCES!!

Exactly. You can't use the same URL for Nickelodeon HD and NPO1. They need different PID filter settings (the "pids" part).

 

[Nervesagent]

Hi I need help with one more thing...
Somehow DVB-EPG is not being scanned correctly for my Fritz (card id: 8) It seems to add epg info for my other tuner (N7), but I see a lot of
IdleEpgGrabber: no channel found for networkid:0x600 transportid:0x819 serviceid:0x4CBD
kind of errors...
Only 2 channels on the Fritz get EPG info: Ziggo Sport and Ziggo Sport HD
Where do I need to look? (see attached logs)

 

[mm1352000]

Have you created two separate versions of each channel... one version for each tuner?
...or have you created channels with two tuning details... one tuning detail for each tuner?

If you've taken the first approach (separate channels for each tuner) then you can expect that only one channel from each pair would get EPG. When two channels/tuning-details have the same ONID/TSID/SID (as your pairs would have), the DVB EPG grabber only updates the first one. That is how the DVB EPG grabber works. The only thing that you can do to work around that limitation is to get EPG from a different source (eg. EPG Collector + XMLTV plugin).

but I see a lot of
IdleEpgGrabber: no channel found for networkid:0x600 transportid:0x819 serviceid:0x4CBD
kind of errors...

That's not an "error". It just means that you don't have a channel with those identifiers. When you created your channels, you should have created them all with the correct [original] network ID, transport [stream] ID and service ID. If you didn't set the ID properties correctly then the DVB EPG grabber can't link the EPG to the channel - very simple!

 

[Nervesagent]

Have you created two separate versions of each channel... one version for each tuner?
...or have you created channels with two tuning details... one tuning detail for each tuner?

If you've taken the first approach (separate channels for each tuner) then you can expect that only one channel from each pair would get EPG. When two channels/tuning-details have the same ONID/TSID/SID (as your pairs would have), the DVB EPG grabber only updates the first one. That is how the DVB EPG grabber works. The only thing that you can do to work around that limitation is to get EPG from a different source (eg. EPG Collector + XMLTV plugin).

but I see a lot of
IdleEpgGrabber: no channel found for networkid:0x600 transportid:0x819 serviceid:0x4CBD
kind of errors...

That's not an "error". It just means that you don't have a channel with those identifiers. When you created your channels, you should have created them all with the correct [original] network ID, transport [stream] ID and service ID. If you didn't set the ID properties correctly then the DVB EPG grabber can't link the EPG to the channel - very simple!

1st approach.
but the combination ONID/TSID/SID is not used for EPG import, since for Tuner 2 (card 8) I've changed all ONID's to a different number.
The one channel that's not on Tuner 1 (Ziggo Sport) is still being updated, but the rest still not, so I'm expecting the combination of TSID/SID to be the primary key.

Would setting the channels to do EPG mapping only (in Tvserver config - Tv Channels - TV Mapping - Perform mapping for EPG grabbing only) fix this or would this only update the first tuner's channels with epg info found by the second?

Another option could be to run a daily query on the program table copying the EPG data over to a different tuning_id (after stopping Tv-service first of course).

 

[mm1352000]

1st approach.

Okay, so as I said in my previous reply: if you use TV Server's built-in DVB EPG grabber then I'm 99% sure that you will only be able to get EPG for one version of each channel. The version is pseudo-random (depends on the order of database query results) - it might be tuner 1's version for some channels and tuner 2's version for other channels.

but the combination ONID/TSID/SID is not used for EPG import

Yes it is!
As I explained in my previous reply:

When you created your channels, you should have created them all with the correct [original] network ID, transport [stream] ID and service ID. If you didn't set the ID properties correctly then the DVB EPG grabber can't link the EPG to the channel - very simple!

After the EPG is grabbed, TV Server has to store it in the database and link it to the correct channel. The "linking" process uses ONID + TSID + SID. It has to use them, because they are the only identifiers which are available.

...since for Tuner 2 (card 8) I've changed all ONID's to a different number.

Why on Earth did you do that?

Changing the ONID (or TSID) will break the EPG linking/storage process. That is probably why you see so many "...no channel found for..." messages. Certainly this is a second reason why EPG grabbing (well, actually - storage) is not working for card 8.

...so I'm expecting the combination of TSID/SID to be the primary key.

As above: the primary key TV Server uses is ONID + TSID + SID. If you want to use TSID + SID then you must customise the TV Server code.

Personally I don't understand why such a customisation would be desirable. It still wouldn't enable the EPG grabber to get EPG for both versions of each channel.

Would setting the channels to do EPG mapping only (in Tvserver config - Tv Channels - TV Mapping - Perform mapping for EPG grabbing only) fix this...?

No.

...or would this only update the first tuner's channels with epg info found by the second?

It would allow tuner 1 to tune tuner 2's channels... but only for EPG grabbing. That does not help in any way, because the problem is with linking/storage not with grabbing/tuning.

Another option could be to run a daily query on the program table copying the EPG data over to a different tuning_id (after stopping Tv-service first of course).

Yes, that would be one option.

 

[Nervesagent]

but the combination ONID/TSID/SID is not used for EPG import

Yes it is!
As I explained in my previous reply:

When you created your channels, you should have created them all with the correct [original] network ID, transport [stream] ID and service ID. If you didn't set the ID properties correctly then the DVB EPG grabber can't link the EPG to the channel - very simple!

After the EPG is grabbed, TV Server has to store it in the database and link it to the correct channel. The "linking" process uses ONID + TSID + SID. It has to use them, because they are the only identifiers which are available.

...since for Tuner 2 (card 8) I've changed all ONID's to a different number.

Why on Earth did you do that?

Changing the ONID (or TSID) will break the EPG linking/storage process. That is probably why you see so many "...no channel found for..." messages. Certainly this is a second reason why EPG grabbing (well, actually - storage) is not working for card 8.

...so I'm expecting the combination of TSID/SID to be the primary key.

As above: the primary key TV Server uses is ONID + TSID + SID. If you want to use TSID + SID then you must customise the TV Server code.

Ehm... no it isn't! It seems to not matter what I put in for ONID.. changing it doesn't break the current EPG collecting..so that's why I'm thinking the pkey is TSID+SID and not+ONID, since if it would be ONID+TSID+SID it would have been either broken, or that would have fixed my problem.

Situation:
Tuner 1: Anysee N7 network tuner scanned .m3u file to get channels. Properties of these channels show 1536 as ONID.and detected TSID+SID automatically
Tuner 2: Fritz dvb-c network tuner, could not scan .m3u file, so manually added all channels with 1536 as ONID and manually inputted all TSID+SID, since it was not detected automatically.
1 channel is not available on Anysee N7 (Ziggo Sport), it reads the EPG from Fritz with ONID 1536 and TSID+SID, but it also reads and tunes if I change the ONID to 1537.
So I thought it would be a smart "hack" to change all Fritz' channels' ONID to 1537, but it will still only update the Ziggo Sport channel that is not on Anysee N7.
Hence I come to the conclusion that the "linking" process you mention does not use ONID.

Maybe some dev can explain how the linking process works exactly. There could be a hack for me in there and I just got curious now?

In the meantime I'll write my script to update the table manually.

Thanks.

 

[mm1352000]

Ehm... no it isn't!

The code does not agree with you:
https://github.com/MediaPortal/Medi...Database/TvBusinessLayer/EpgDBUpdater.cs#L122
https://github.com/MediaPortal/Medi...Database/TvBusinessLayer/EpgDBUpdater.cs#L259
https://github.com/MediaPortal/Medi...atabase/TvBusinessLayer/BusinessLayer.cs#L468
https://github.com/MediaPortal/Medi...atabase/TvBusinessLayer/BusinessLayer.cs#L405

It seems to not matter what I put in for ONID.. changing it doesn't break the current EPG collecting.

With the greatest respect, I think you must be mistaken. According to the code linked above this is simply impossible.
If you clear all EPG for one channel, change the channel's ONID to a wrong value, then grab EPG again, I'm confident that EPG will not be stored/available for that channel.

...so that's why I'm thinking the pkey is TSID+SID and not+ONID, since if it would be ONID+TSID+SID it would have been either broken...

But it is broken!
You told me that you changed the ONIDs for your Fritz! channels... and EPG is not available for those channels. This is the definition of broken, is it not?

...or that would have fixed my problem.

Changing the channel's ONID to the wrong value would never fix the problem. The ONID + TSID + SID in the stream is fixed/unchanging, therefore changing the channel's ONID can only break the EPG linking/storage.

Situation:
Tuner 1: Anysee N7 network tuner scanned .m3u file to get channels. Properties of these channels show 1536 as ONID.and detected TSID+SID automatically
Tuner 2: Fritz dvb-c network tuner, could not scan .m3u file, so manually added all channels with 1536 as ONID and manually inputted all TSID+SID, since it was not detected automatically.

To be clear:

• you have channel "ABC" for the N7
• you have a separate version of channel "ABC" for the Fritz!
• ONID, TSID and SID should be the same for both versions of "ABC"; only the URL will be different

1 channel is not available on Anysee N7 (Ziggo Sport), it reads the EPG from Fritz with ONID 1536 and TSID+SID...

Yes, that is not unexpected. EPG storage/linking is not limited to the channels that the tuner can tune. In other words, EPG for N7 channels can be grabbed using the Fritz (or vice versa) as long as the ONID + TSID + SID is correct. This does not require any special mapping.

...but it also reads and tunes if I change the ONID to 1537.

I agree that it will tune successfully (ONID is not used for tuning), but I don't agree about the EPG part. Of course once the EPG is stored in the database then it will stay linked to the channel. So, it will appear like the EPG is okay for awhile... but it will never be updated, and so eventually the EPG will run out.

So I thought it would be a smart "hack" to change all Fritz' channels' ONID to 1537, but it will still only update the Ziggo Sport channel that is not on Anysee N7.
Hence I come to the conclusion that the "linking" process you mention does not use ONID.

I'm 99.999% sure you're mistaken. Most likely you simply haven't waited long enough for EPG grabbing to complete.

Maybe some dev can explain how the linking process works exactly. There could be a hack for me in there and I just got curious now?

I am a dev with more than 5 years experience working on TV Server. I've spent a lot of time trying to explain many things to you in this thread. In the last few posts I've already tried to explain in detail how EPG linking works and why the DVB EPG grabber cannot give you what you want. It seems you don't believe me. Therefore explaining again seems like a waste of my time.

Categorically: with your current configuration, there is no "hack" that can work unless you change the code.

If you don't want to change the code then you can use a script to copy EPG from one channel to another...
...or use an external program (eg. EPG Collector) to collect EPG, and import it using the XMLTV plugin [which can store/link the same EPG to more than one channel]...
...or consider combining the channels, so each TV Server channel has 2 tuning details.