[further infos missing] Unicode Error in UK EPG (1 Viewer)

[CyberSimian]

The attached screen shot shows a page from the EPG that is transmitted with the DVB-T/T2 broadcasts in the UK. If you look at the programme description at the bottom of the screen, you will notice that there is a spurious capital "A" with a circumflex accent preceding the pound symbol for the UK currency ("£"). This is with the default-wide skin.

I checked another skin, and that showed the same error. I then checked my Humax PVR, and that did not show the error. This suggests that the error is somewhere within MP, and not in the broadcast EPG itself.

Is there any other data that you need to debug this? A copy of the EPG data file, perhaps? If so, please specify its name and location. I am using MP 1.8.0 on Vista.

-- from CyberSimian in the UK

 

[Vasilich]

we need to check the transport stream to see what encoding markers it has:
http://wiki.team-mediaportal.com/1_MEDIAPORTAL_1/18_Contribute/4_Development/Debugging/TS_Dumps

 

[CyberSimian]

we need to check the transport stream to see what encoding markers it has.

I have uploaded a 1-minute TS file and a TXT file as requested. Filename is:

uk_freeview_498000_dvbt_1_fta

The TS file is approximately 187MBytes in size, and took 32 minutes to upload.

-- from CyberSimian in the UK

 

[Caesium]

Looks like Argus TV does exactly the same thing - I checked in the Argus scheduler and saw the same issue for next week's Escape to the Country

Interestingly, the EPG from BBC1 West Midlands (so SD, not HD) doesn't do it, that looks fine. I'll report it to dot-i as well..

 

[Owlsroost]

It's possibly something to do with the fact that the Freview HD (but not SD) EPG data is scrambled (Huffman encoded I think), so the decoding of it is basically reverse-engineered.

 

[CyberSimian]

Interestingly, the EPG from BBC1 West Midlands (so SD, not HD) doesn't do it.

When I first noticed this error, my recollection is that I checked both SD and HD versions of BBC1, and found the error present on both. I receive signals from the Rowridge (Isle of Wight) transmitter. But my recollection may be faulty...

I am currently completely revamping the software on my HTPC (60 Vista updates to install, followed by MP 1.9.0 pre), so won't be able to check this until later.

-- from CyberSimian in the UK

 

[CyberSimian]

Interestingly, the EPG from BBC1 West Midlands (so SD, not HD) doesn't do it, that looks fine.

I have finished updating Vista and installing MP 1.9.0 pre, and I have had a look at the EPG for "BBC1 SD" and "BBC1 HD"; they both show the error in the EPG. I have attached a screen shot of the "BBC1 SD" programme description.

I have also noticed an error in the programme description on both "Channel 4 SD" and "Channel 4 HD" (see second screen shot). The word in error is "protege", which is shown correctly on my Humax PVR, with acute accents over the two "e" letters.

It's possibly something to do with the fact that the Freview HD (but not SD) EPG data is scrambled (Huffman encoded I think), so the decoding of it is basically reverse-engineered.

This could be. I was under the impression (possibly incorrect!) that all MUXes transmit the EPG for all channels (except that the SD MUXes do not transmit the HD EPG). If this is correct, it would be possible for the EPG for the SD channels to be populated by the SD EPG received via the HD MUXes, and hence be subject to any errors that might exist with the Huffman decoding.

-- from CyberSimian in the UK

 

[Caesium]

I have also noticed an error in the programme description on both "Channel 4 SD" and "Channel 4 HD" (see second screen shot). The word in error is "protege", which is shown correctly on my Humax PVR, with acute accents over the two "e" letters.

Yep, can confirm this here as well, both SD and HD Channel4 exhibit this. I just double checked though and Escape to the Country next week, for example Wednesday afternoon (3pm I think it was), only shows the error on HD, not SD.

But I may be muddying the issue since I use Argus so I'll butt out for now and see what dot-i says about it

 

[Vasilich]

@mm1352000 what software do you use for checking EIT tables? the TSReader doesn't show me undecoded content of it (and i need to check what language bytes are used in EPG event descriptors)

 

[mm1352000]

what software do you use for checking EIT tables?

It depends what I'm checking.

the TSReader doesn't show me undecoded content of it (and i need to check what language bytes are used in EPG event descriptors)

You mean TsWriter?
I suspect you'd find that the EPG data is huffman encoded as Owlsroost has said, and so the "selection of character table" byte would be 0x1f and encoding_type_id would be 1, 2, 3 or 4 (BBC) as specified in the DVB registry:
http://www.dvbservices.com/identifiers/encoding_type_id

My understanding is that after Huffman decoding, the content is meant to be UTF-8 encoded... but all of that is specified in private Freesat and Freeview specifications. I could be wrong.

Unfortunately the raw dump is too big for me to download on my 10GB cap. I'd appreciate it if somebody could post a version where the video and audio PIDs have been filtered...