Teletext subtitles (4 Viewers)

[SweMart]

Any chance of this getting introduced into SVN?

 

[ziphnor]

The filter solution is a lot of coding and currently I'm still trying to compile the filters of TVE3 myself.
IMHO the fastest and easiest solution is to modify TsWriter/MPFileWriter and TsReader. First the TsWriter/MPFileWriter has to be modified to store the teletext stream in the ts files. When the teletext stream is available in the ts file, it should be possible to create a callback like the TsWriter already have for teletext.

As i believe i mentioned previously i am working on integrating (DVB based) teletext subtitle support into the DVBSubtitle2 filter.

The DVBSubtitle2 filter connects to TsReader (to which i added a teletext pin) and receives the subtitle and teletext streams, it decodes these and then calls back to MP (to Core/Player/SubtitleRenderer.cs ) with text and bitmap subtitles. All decoding is done on the filter side, and rendering is done efficiently on the MP side. Also in TsReader, the Service Information is parsed, identifying the language (not just the encoding like in the teletext page header), which should be sufficient to supply automatic display of the correct subtitle page without any user interaction except for setting language preferences (similar to DVD playback). As part of this work-in-progress, i yesterday added code to TsWriter to include the teletext PID when timeshifting and recording ( i havent had a chance to test it yet though, but it was only a very simple change).

I decided to not re-use the MediaPortal side C# caching/decoding because i only need support for the subset of functionality corresponding to teletext subtitle pages (and what was currently there seemed hard to maintain) as well as for performance reasons.

Anyway, i just want to make sure we avoid repeating each others work here, in fact it might be better if we tried to help each other

I was wondering if you might want to help out with debugging the small teletext decoder i have created in DVBSubtitle2. Then we can also get you up and running with compiling the TVE3 filters etc. When the subtitle part is up and running (which IMHO is the more important part) we could then take a look at the normal Teletext module together if it needs further improvement.

Oh btw, be sure to check out the blog entry on the topic:
https://www.team-mediaportal.com/blogs/team_blog/subtitles_on_dvb_broadcasts_in_mediaportal.html

 

[misterd]

Hi ziphnor,
I know that you're working on teletext subtitle filter. Since there is only a small number of shows with teletext subtitles in Germany, I'm more interested in a complete and good working teletext module, which include subtitles.

Your approach is really good, if you only want to display the subtitles of teletext pages. I think that a combined solution for teletext and teletext subtitles is the better approach, because the main part is the decoding and caching of the teletext page (avoid duplicate code). This part isn't also the reason for the performance problems in the current teletext module and the C# code should also be easier to maintain (I hate C++ ). A library for this could be the best, because the TVServer needs teletext decoding for extracting the name of analog channels. The extraction of subtitles or the rendering of the page is the smaller part and could be handled seperatly. The rendering is also the reason for the performance problems.

I saw your modifications for the teletext pids in the server, but I don't think that they will work for timeshifting. :sorry: For recording it should work, but the filter uses fake pids for timeshifting and there is no such pid for teletext. There are also changes in the filter required. But I must admit, that I haven't tested it so far.

I could test your modifications or have a look at it, but as mentioned above, I don't have much test material especially with teletext subtitles. BTW: My problems with compiling the filters are solved.

MisterD

 

[ziphnor]

I think that a combined solution for teletext and teletext subtitles is the better approach, because the main part is the decoding and caching of the teletext page (avoid duplicate code). This part isn't also the reason for the performance problems in the current teletext module and the C# code should also be easier to maintain (I hate C++ ). A library for this could be the best, because the TVServer needs teletext decoding for extracting the name of analog channels. The extraction of subtitles or the rendering of the page is the smaller part and could be handled seperatly. The rendering is also the reason for the performance problems.

I see your point but there are several issues here:

1. The teletext implementation in MediaPortal is a bit of a mess ( almost zero comments ).
2. I don't believe it detects the DVB Service information regarding teletext pages (which contains critical language information)
3. Its unsuitable for subtitle handling because it doesnt provide a way to extract the text , or at least i couldnt find it ( i dont want to render the subtitles in the normal ugly teletext manner).
4. Subtitle handling should go through one centralized place in order to automates selection and rendering of subtitles.
5. Handling(/maintaining code for) the full teletext specification is much more complicated than just the subtitle pages.

We could put the teletext TS + PES (C++) decoding into another separate directshow filter which could then callback to Mediaportal for each teletext packet it decodes(so no delays), where i by decode means it simply passes each chunk of teletext data to MP. It would then connect to TsReader on its teletext pin (not in SVN) instead of connecting the DVBSubtitle2 filter there. I can implement that without too much trouble ( its mostly copy-paste from DVBSubtitle2).

A new/modified teletext module could then receive and cache this information, but it would need to provide callbacks so it could deliver subtitles to the subtitle renderer (in text format along with meta-data on the subtitle stream). I'm not really interested in working on the teletext module as such (my interest is solely with subtitles which are a critical features in the Scandinavian countries), but i would be happy to provide the support needed if you were planning to work on it.

I saw your modifications for the teletext pids in the server, but I don't think that they will work for timeshifting. :sorry: For recording it should work, but the filter uses fake pids for timeshifting and there is no such pid for teletext. There are also changes in the filter required. But I must admit, that I haven't tested it so far.

I am not really familiar with how the server works, so you may very well be correct. I just noted that PidInfo already contained information about being teletext or and then looked at how the DVB subtitle PIDs where handled (and i believe they are recorded/timeshifted correctly?).

I could test your modifications or have a look at it, but as mentioned above, I don't have much test material especially with teletext subtitles. BTW: My problems with compiling the filters are solved.

That at least would be a good idea, no matter what we both need teletext data stored in the ts files You could maybe record (and time shift) using TV server and then playback in say DVB viewer (if you have it) or any program capable of showing teletext (sub or not) from .ts files.

 

[misterd]

Some additional comments to your points:
1. Since I have made several patches for the current implementation, I know how it works. I also added some comments to the code with my patches, but you are right, it is a mess. Unfortunately it can be easily reused, but it should be refactored and commented. BTW: I would prefer the code of TVE2 instead of TVE3, which is almost the same, because it fits more to that what I want to do.
2. Correct. It only handles the data stream which is transmitted through a callback from the TsWriter filter or the sample caputre filter for analog tv to the TVServer. I must admit that I don't know which informations are in the DVB Service information.
3. I think that this can be done very easily, if you know where (TeletextPageRenderer) Of course there are changes in the handling for that, but this shouldn't be the big problem.
4. Totally agree. My concept would also call your SubtitleRender, not in the first step but definitely in a later step.
5. I know that, but if the teletext module is a little bit more structured and documented, perhaps it could be extended to support other features of teletext. The current module only support the basic features of teletext.

Your idea about the additional filter is good and very similar to the approach I made in the post you quoted.
The problem of all these ideas about creating a new teletext module is that we have to include analog tv in our plan and this could be more complicated than dvb. Such a new teletext module (caching/decoding) should also be used in the whole system, because I want to remove the duplicate code (e.g. Hamming exists in TVServer and TVPlugin).

When I have finished the new version of my plugin, I wil test if the teletext is stored in .ts

MisterD

 

[ziphnor]

2. Correct. It only handles the data stream which is transmitted through a callback from the TsWriter filter or the sample caputre filter for analog tv to the TVServer. I must admit that I don't know which informations are in the DVB Service information.

It contains information linking subtitle numbers to actual languages. In the teletext header you only get encoding/character table information in the language field, so for example you have no way of telling Norwegian, Swedish and Danish apart (language is 2 (swedish) for all of them). The DVB SI will map (say) page 693 to 'dan' (danish) so that you can select the correct page automatically based on the users language preference instead of having him manually set up subtitle pages. This also enables you to easily make some fixes in the character encoding, such that for danish you for example swap ö for ø (or something like that).

Your idea about the additional filter is good and very similar to the approach I made in the post you quoted.

I was just trying to figure out how what i am doing right now, could be used to achieve what your suggesting, without having to make drastic changes. My point is that its probably best to have the callback in a filter separate filter from TsReader, as TsReader is supposed to be essentially a DirectShow demux'er for TS files, and hence it makes more sense that it just provides a teletext output pin (and also because i already created the code for it .

The problem of all these ideas about creating a new teletext module is that we have to include analog tv in our plan and this could be more complicated than dvb. Such a new teletext module (caching/decoding) should also be used in the whole system, because I want to remove the duplicate code (e.g. Hamming exists in TVServer and TVPlugin).

I hate analog TV But couldn't we just structure it so that the teletext module has no clue about whether the source is analogue or digital, after all the only difference is bit of the initial data of each packet (ie run-in is not present in DVB etc ). Once its unwrapped digital and analog should be the same, right? So the teletext module should just have a Receive() method that we can call from DirectShow for digital stuff and then you can create an analog teletext grabber that calls the same method (of course not at the same time ).

When I have finished the new version of my plugin, I wil test if the teletext is stored in .ts

A good place to start.

 

[misterd]

It contains information linking subtitle numbers to actual languages. In the teletext header you only get encoding/character table information in the language field, so for example you have no way of telling Norwegian, Swedish and Danish apart (language is 2 (swedish) for all of them). The DVB SI will map (say) page 693 to 'dan' (danish) so that you can select the correct page automatically based on the users language preference instead of having him manually set up subtitle pages. This also enables you to easily make some fixes in the character encoding, such that for danish you for example swap ö for ø (or something like that).
ea about the additional filter is good and very similar to the approach I made in the post you quoted.

This makes everything really easy for dvb

I was just trying to figure out how what i am doing right now, could be used to achieve what your suggesting, without having to make drastic changes. My point is that its probably best to have the callback in a filter separate filter from TsReader, as TsReader is supposed to be essentially a DirectShow demux'er for TS files, and hence it makes more sense that it just provides a teletext output pin (and also because i already created the code for it .

Unfortunately I think that there is no difference if you create a new filter or integrate the code into TsReader. But the new filter solution is of course the cleaner solution.

I hate analog TV But couldn't we just structure it so that the teletext module has no clue about whether the source is analogue or digital, after all the only difference is bit of the initial data of each packet (ie run-in is not present in DVB etc ). Once its unwrapped digital and analog should be the same, right? So the teletext module should just have a Receive() method that we can call from DirectShow for digital stuff and then you can create an analog teletext grabber that calls the same method (of course not at the same time ).

I don't have a analog signal even here, but I think that a complete concept is needed for analog and digital tv. I think the people won't be happy, if we remove the support for teletext of analog tv.
Since I haven't looked at analog tv yet, I can't evaluate how complicate it will be to integrate the teletext of analog tv in the .ts. The only difference than will be the missing DVB Service Informations. But first we should concentrate on the things we can "easily" test, which means digital tv.

MisterD

 

[ziphnor]

Unfortunately I think that there is no difference if you create a new filter or integrate the code into TsReader. But the new filter solution is of course the cleaner solution.

I suggested it exactly as to keep the code clean. TsReader is big enough as it is.

I don't have a analog signal even here, but I think that a complete concept is needed for analog and digital tv. I think the people won't be happy, if we remove the support for teletext of analog tv.

With the structure im suggesting, we wouldnt. Analog would but just be one supplier of data to the teletext module.

Since I haven't looked at analog tv yet, I can't evaluate how complicate it will be to integrate the teletext of analog tv in the .ts. The only difference than will be the missing DVB Service Informations. But first we should concentrate on the things we can "easily" test, which means digital tv.

There has never been any support for recording the teletext information of analog TV in MediaPortal (or any other digital recording device i think), so i dont think people will be upset if teletext is only available when watching live TV. The only problem i see is in regards to time shifted TV, i wonder how that is handled right now (meaning if watch TV 5 minutes time shifted is the teletext time shifted or not)?

Frankly grabbing and writing the analog teletext data into the .ts file would be a mess. Why would you want to record teletext anyway (except for the subtitles, which is almost never used for analog TV).

 

[misterd]

With the structure im suggesting, we wouldnt. Analog would but just be one supplier of data to the teletext module.

I just wanted to guarantee that we won't forget those people

There has never been any support for recording the teletext information of analog TV in MediaPortal (or any other digital recording device i think), so i dont think people will be upset if teletext is only available when watching live TV. The only problem i see is in regards to time shifted TV, i wonder how that is handled right now (meaning if watch TV 5 minutes time shifted is the teletext time shifted or not)?

Frankly grabbing and writing the analog teletext data into the .ts file would be a mess. Why would you want to record teletext anyway (except for the subtitles, which is almost never used for analog TV).

Currently you always get the current teletext informations which is received by the tvserver. The timeshift position is completely ignored.
I totally agree with you that recording analog teletext data into the .ts file is more complicated than for digital tv. Since such a change can't be done in one step, we could perhaps put this on the list for one of the last steps as a "nice to have" feature , because I think that recording the analog teletext would be the solution with the most consistency.

Today I had a look at TsWriter and I think that I have found where we have to modify it for teletext recording. I will try to implement it this weekend, but I'm not sure if I have identified all locations, since TsWriter is really BIG.

MisterD

 

[cage121]

Hi. A really dumb question:
How do I get teletext subtitles?
I see the subtitles, but it is in a black squre. I cant see the TV image behind it!
Sorry to bother you with this.