[confirm] ATSC scanning problem (1 Viewer)

[x4mer]

MediaPortal Version: 1.2 Beta
MediaPortal Skin: Default
Windows Version: Windows Vista SP1
CPU Type: Intel Q8200
HDD: 120G + 2.25TB RAID
Memory: 4G
Motherboard: Gigabyte EP45-UD3R
Video Card: Radeon 3450
Video Card Driver: 8.12
Sound Card: Integrated
Sound Card AC3: No AC3
Sound Card Driver:
1. TV Card: Hauppauge HVR-2250
1. TV Card Type: dual-ATSC/Analogue
1. TV Card Driver: 7.9.1.28162
2. TV Card: Hauppauge HVR-2250
2. TV Card Type: dual-ATSC/Analogue
2. TV Card Driver: 7.9.1.28162
3. TV Card: Hauppauge HVR-2250
3. TV Card Type: dual-ATSC/Analogue
3. TV Card Driver: 7.9.1.28162
4. TV Card:
4. TV Card Type:
4. TV Card Driver:
MPEG2 Video Codec: MS
MPEG2 Audio Codec: MPA
h.264 Video Codec: FFDShow
Satelite/CableTV Provider: ATSC - OTA, QAM/Analogue - Rogers
HTPC Case: Antec Sonata III
Cooling: Stock
Power Supply: Antec 500W
Remote: MCE
TV: Samsung HL-T5687S
TV - HTPC Connection: HDMI

When 1.2b came out, I deleted all my cable QAM channels and tried to rescan cable. Due to a bug in 1.2b, I have had no digital cable stations since switching to 1.2b. All my ATSC channels via antenna, have been working fine under 1.2b.

Canada is currently in the final weeks of it's digital transition for over-the-air broadcasts, to be completed by the end of August. As such, a number of stations have changed recently, as they fire up their new digital transmitters to replace their analogue ones.

Once this happens, I have to delete the old channel & rescan for the new one. MP 1.2b is not picking up the stations properly. The attached logs, show Physical 7 finding 4 unique IDs, but only adding 1 channel. My TV adds all 4. It also shows finding nothing on Physical 11, while my TV picks up the channel no problem. If this continues, I'll have no channels left by the end of the month when the transition completes.

I see that mm has bumped the issue of the scan changes in 1.2b needing to be addressed, to V 1.3. This means if my stations continue to disappear, like I expect them to, I'll have to go all the way back to 1.1 on my server and all clients in my house, just to get any channels back. Could there please be a hotfix of some sort to address the scanning issue, if there's no time to get it fixed for 1.2RC?

 

[mm1352000]

Hi x4mer

I hope you are well.
Yes, I have bumped the scanning issue to a future version. I hope you didn't think we'd leave things as they are in 1.2.0b though - that would be unacceptable. You would have seen that I've made this comment on the mantis issue:

Changes causing the worst problems have been reverted in SVN 28118. This issue needs looking at again in a future version of TV Server as a part of a larger scanning rework.

What that means is that we have patched over the problems that you and other people with QAM and DVB-IP are having for 1.2.0rc and 1.2.0 final. We consider the "fix" unsatisfactory in the long term but we don't want to risk the integrity of 1.2.0rc/final with the larger changes that would be required to fix it properly. So in short you should expect 1.2.0rc to work like 1.1.x did.

Regarding the topic of hotfixes: I have been reluctant to provide a hotfix as I was unsure how the team would decide to deal with the problem. Now that things are more certain, I am happy to provide one - please find an updated TVLibrary.dll attached.

Installation instructions:

1. Stop TV Server.
2. Take a backup of the existing TVLibrary.dll in the TV Server install folder.
3. Download and extract the attached TVLibrary.dll into your TV Server install folder.
4. Start TV Server.

You should be able to rescan without problems.

Best regards,
mm

 

[x4mer]

Thanks mm.

This dll fixed the scanning problem on RF 7. It picks up all 4 channels now. It's not in the attached logs though, as the antenna is not pointed that way currently.

RF 11 still won't scan in though. MP keeps reporting "nothing found", instead of the usual "no signal". The station scans in fine using my TVs built in tuner. Using the signal monitor for my card, it shows a nice solid SNR, with an extrememly low error rate - much better than other stations that come in fine.

I've seen one other person, posting on a local forum regarding TV reception OTA, that's having the same problem also using MP.

 

[mm1352000]

Hi again x4mer

I have spent several hours analysing your logs and examining the code this evening. Things certainly don't go entirely smoothly in that scan... I think there may be two problems.

In the first case we have CHCH-DT not picked up from RF channel 11. I believe the issue here may be the use of an illegal PMT PID, although I'm not entirely certain as the logging isn't quite complete enough. I'm reserving my judgement on this for the moment...

The second issue is that 5 of the channels are stored sans important information like service/channel name which is present and logged in the VCT. This isn't the end of the world as the channels are picked up in the end, but in my book it is *highly* undesirable to have to enter such basic details as channel names, especially when MP has the required information - this is a clear bug.

Okay so where do we go from here?

The first thing I'm going to do is provide you with an additional patch that includes extra logging to help me figure out exactly what is going on. I will alert the rest of the team to this thread but unfortunately it may be too late to fix these problems in 1.2.0rc. Nevertheless, could you please try the patch at your earliest possible convenience. I would like to find a solution as fast as possible in case the team do agree to sneak it into 1.2.0rc. I note that my current understanding from my analysis of the code is that both of these issues were present in MP 1.1.x - are you able to confirm that?

Anyhow, instructions for this patch (please keep using the previous TVLibrary.dll patch):
1. Stop TV Server.
2. Take a backup of the TsWriter.ax in your TV Server install folder.
3. Download and extract the attached TsWriter.ax into your TV Server install folder.
4. Start TV Server.
5. Rescan for ATSC channels (no need to delete the existing ones).
6. Post the logs for the scan - the logs you posted today were perfect for me so please use the same procedure if possible.

If any of the people on the local forum that you mentioned have/had problems with MP and are still persisting and willing to help us track these problems down then please encourage them to drop by. I'd be happy to look at their logs too - the more examples of ATSC or QAM service information I see, the more certain I am about the special treatment that is required for ATSC (as opposed to DVB) streams.

Apologies for these ongoing problems,
mm

 

[x4mer]

Thank you very much for looking in to this mm.

The matter of the 5 channels you speak of, without names, is not an issue I believe. They may have been out of market signals, that just happened to be showing up during my last scan, but not strong enough to get a solid decode on the PSIP. That will happen on my TV's built in tuner as well, if the signal is too weak.

I haven't had an issue in previous versions of MP, of the antenna channels not getting names, unless it was a distant weak station, or weather was very bad during the scan.

Attached logs with new debug TSWriter. Everything got names during scan I believe, but CHCH shows as nothing found.

I have sent a PM to the other person, asking them to join us.

 

[Marbles_00]

Hi,

x4mer invited me to join this discussion to aid in helping MP to scan these illicit channels. I can't help out with the RF7 stuff, since I'm not lucky enough to get the signal, but I too had issues of scanning RF11.

When scanning it would respond with "nothing found", which I came to conclude that it's MP's way of saying: "There's a signal there, but not enough for a lock".

I also use TSReader, which is a great program for performing some basic signal analysis. TSReader was successful at locking in RF11.

What I did to get around this is to create a RF11 entry in the channel list. I used the Transport Stream ID and PMT PID number defined in the TSReader capture. There was one other setting (forgot right now what it is called), but once I added those three variables, RF11 came in nice and clear.

When I get home tonight, I can see what the TS ID and PMT PID values are, and I'll do a follow-up reply here.

 

[x4mer]

Thanks for joining Marbles.

TSReader supports saving out to a log file of some sort right?

Do you think you could post what TSReader parses from CHCH, so mm can compare that to what MP is obtaining?

 

[x4mer]

TSReader lite output

Don't know if this is right, as this is my first time using TSReader, but here's the output I saved from TSReader after getting it to tune to RF 11.

I hope you didn't go to a lot of trouble Marbles, if you did the TSReader export stuff before reading this.

 

[Marbles_00]

No problems, I hadn't gotten to it yet.

To get you the channel running right now, all you need to do:
1) is open TV Server Configuration
2) Click on TV Channels to get your channel list screen
3) click on the "Add" button under "Channels"
4) give the station a name (CHCH for example)
5) then in the "tuningdetails" window, right click and "Add"
6) select "ATSC" the "OK"
7) Then add the following info:

Channel:  11
Frequency:  -1
Major:  11
Minor:  1
Modulation:  Mod8Vsb
NetworkId:  0
TransportId:  17091
ServiceId:  1
PmtPid:  16
Provider: (whatever you want to add)

8) click on OK to add the channel
9) click on OK in the "Edit Channel" window
10) Test by clicking on the "Preview" button.

To let you know CKCO on RF13 came through for me with a normal scan. It was indicated to me that they are still testing that channel, so you may have success or not, but regardless, this bug didn't seem to affect RF13 from what I could tell.

 

[mm1352000]

Hi again x4mer, and thanks for dropping in Marbles_00

I wanted to reply earlier this morning when I saw that you were both online but unfortunately it has been a busy day.

Let's talk about the RF 11 issue first. It turns out I was right - the issue with picking up CHCH is the PMT PID, as clearly seen in both the TsWriter logs and the TSReader output:

PMT PID 16 (0x0010) - Program 1 CHCH-DT

So what is the problem with that? According to MPEG 2 TS specifications:

The PID is a 13-bit field, indicating the type of the data stored in the packet payload. PID value 0x0000 is reserved for the Program Association Table (see Table 2-25). PID value 0x0001 is reserved for the Conditional Access Table (see Table 2-27). PID values 0x0002 – 0x000F are reserved. PID value 0x1FFF is reserved for null packets (see Table 2-3).

No problem so far. However when we look at the DVB service information specifications (link, page 18) we read that PID 0x0010 is reserved for the network information table (NIT).

Even though ATSC doesn't have the same PID assignments as DVB, the ATSC standards body forbids ATSC broadcasters from using those PIDs (link, page 23):

6.9 PID Value Assignments
In order to avoid collisions with fixed PID values and ranges already established in this and other international standards, transport_packet() PID field values are restricted as follows:
• TS packets identified with PID values in the range 0x1FF0 – 0x1FFE shall only be used to transport data compliant with ATSC-recognized standards specifying fixed-value PID
assignments in that range. (Informative note: One such use is A/65, which requires the use of 0x1FFB to identify packets containing certain tables defined in that standard.)
• In order to avoid collisions with fixed PID values and ranges already established in thisand other international standards, PID values used to identify Transport Stream packets carrying TS_program_map_section() or program elements shall not be set below 0x0030. (Informative note: One such use is in ETS 300 468, which requires the use of 0x0011 to identify packets containing certain tables defined in that standard.)

So to summarise:
- MP (being software that supports DVB standards) expects that PID to be used for other things
- this broadcaster is breaking ATSC rules by using that PID

I don't know for certain that there would be any harm in bending the rules to allow this channel to be picked up, but we like to follow the standards wherever possible so I'm more than a little reluctant to do so. What I would like to do is provide you with a test patch to see if any of the other code would need to be fixed by removing this condition. I'll provide that when I get home.

Moving on to the issue about channel names...

The matter of the 5 channels you speak of, without names, is not an issue I believe. They may have been out of market signals, that just happened to be showing up during my last scan, but not strong enough to get a solid decode on the PSIP. That will happen on my TV's built in tuner as well, if the signal is too weak.

x4mer I'm not so sure about this one. In the case of 4 of the channels, they no longer show up in your latest logs. However your latest logs have more channels showing the same symptoms - VCT found and decoded correctly, but channel information returned without a name. It seems to be a "race condition" where if the VCT is received after the PMT then the channel name will be "found", but if it is the other way round then the channel name won't be found. Right now it seems like the VCT is only parsed once and I can't see any reason for this. Maybe the VCT is only broadcast infrequently... like once every 30 seconds or so. I'll try to look at this in more depth when I get home.

Best regards,
mm