IPTV/DVB-IP in MP: Support for http, rtp and udp (3 Viewers)

[georgius]

Hi,

new version of filter with logging when GetDeliveryBuffer() method returns error. Also changed removing data from buffer. If buffer is filled more than half, the remaining data after remove are moved to beginnig of buffer (to free some space at end and new data can be written at end of buffer). Also in DecideBufferSize() method is changed number of buffers required from allocator (in old filter it was one buffer, now its eight).

Errors reported by velis could happen in UDP and HTTP streams, so changes written above are for both.

 

[Stepko]

AW: Re: AW: Re: IPTV/DVB-IP in MP: Support for http, rtp and udp

Yes, you're right actually is not working (all RTP code is commented). I'm planning to add support as soon as possible, but I don't know any RTP stream. I can simulate streams with VLC, but it's not as real stream.

I would really like to try the new filter with rtp
I can do some testing for you with a real stream on the weekends.

I just had an idea about the linear buffer. It would also be possible to use a single chained list of buffer objects. You could create a pool of unused buffer objects. When you get data you can put the objects one after another and put them back when the data is delivered in FillBuffer(). With pointers to the first and last objects it would be very quick. You could also extend the buffer dynamicly and don't need to move the data. Just an idea.

This time I noticed that you used the old version as the base of your version. That version implements the DoProcessingLoop function. I don't think this is done correctly in the old version - and thus in yours, too. I think you can delete the DoProcessingLoop function completly!! The default implementation of this method is almost the same as your code. It calls fillMedia where all fill logic should be done. Referring to msdn DoProcessingLoop can be overridden but I don't really see a situation where this really make sense. All the client logic (e.g. if (occupiedSpace > 0)) is already done somewhere else or can be done somewhere else. You just need to add a sleep(..) in FillBuffer if (lengthOfPostableData <= 0) to avoid 100% cpu usage.

msdn: CSourceStream Class

Many for your help to get a stable IPTV solution
Stepko

 

[georgius]

Re: AW: Re: AW: Re: IPTV/DVB-IP in MP: Support for http, rtp and udp

This time I noticed that you used the old version as the base of your version. That version implements the DoProcessingLoop function. I don't think this is done correctly in the old version - and thus in yours, too. I think you can delete the DoProcessingLoop function completly!! The default implementation of this method is almost the same as your code.

Yes, you're right, but I got implementation of DoProcessingLoop() method from TsWritter, so I think it's correct. Now I have this method only for testing purposes - it's whole implementation can be removed and necessary functionality can be moved to FillBuffer() method - except one thing. If GetDeliveryBuffer() fails the filter will not be notified and log message will not be written.

I just had an idea about the linear buffer. It would also be possible to use a single chained list of buffer objects. You could create a pool of unused buffer objects. When you get data you can put the objects one after another and put them back when the data is delivered in FillBuffer(). With pointers to the first and last objects it would be very quick. You could also extend the buffer dynamicly and don't need to move the data. Just an idea.

Idea is interresting, but problem with allocator cannot be solved by extending buffer or creating new one. I suppose that provider (HTTP or UDP) send its packets in the same speed as they are processed. If not than it's bad - because the filter in that case is receiving faster than packets are processed. In this situation doesn't help if you extend buffer or create new one. It only delays a problem.

In current situation (relatively stable IPTV filter without RTP support) its annoying for me that TvServcice and MP have to wait almost 20 seconds if there is no signal to return error (and this error is "Unable to start graph"). In DoProcessingLoop() I tried to return S_OK, S_FALSE, I tried to call DeliverEndOfStream() or notify m_pFilter->NotifyEvent(EC_ERRORABORT, hr, 0) but without success. Does anybody know how to notify TvService that there is no signal without waiting almost 20 seconds?

 

[velis]

Re: AW: Re: AW: Re: IPTV/DVB-IP in MP: Support for http, rtp and udp

I just had an idea about the linear buffer. It would also be possible to use a single chained list of buffer objects. You could create a pool of unused buffer objects. When you get data you can put the objects one after another and put them back when the data is delivered in FillBuffer(). With pointers to the first and last objects it would be very quick. You could also extend the buffer dynamicly and don't need to move the data. Just an idea.

This is exactly the way my version works. So far it's the only way of making sure there are no dropped packets, but the CPU usage is erratic as I described in my last post. And Georgius says he gets 100% CPU usage with it

Does anybody know how to notify TvService that there is no signal without waiting almost 20 seconds?

Hence my idea of spoofing a "snow" clip...

Georgius, I've been looking at the source code of this latest filter: I think it would be better to have three separate thread functions, each for its own transport. Currently the code is hardly readable because of all the branching. Also please see my latest solution for logging. It doesn't require #ifdef in code.
It's a shame we don't have a SVN for this. It's so hard for multiple people to work on the same file

Edit: I'm continuing work on my version of filter because so far it's the only one I know that produces stutter free grabbing. I must say that I can't see how Georgius's code makes any difference in regard to CPU usage stability. In fact, it should be worse since it performs memcpy every now and then. To be quite honest, I can't see where my code could *ever* use that much CPU, yet it happens. I will prepare the stutter free code tonight in case anyone would be interested in looking for what I am missing here (in regard to CPU usage problem, of course).

when a good, low CPU, stutter free code base is established, it can easily be reused for other methods. That's where I see usefulness of my ongoing efforts. Plus - I don't have to constantly verify any progress made by Georgius

 

[georgius]

Re: AW: Re: AW: Re: IPTV/DVB-IP in MP: Support for http, rtp and udp

This is exactly the way my version works. So far it's the only way of making sure there are no dropped packets, but the CPU usage is erratic as I described in my last post. And Georgius says he gets 100% CPU usage with it

It was between 50 - 55 % but result is same.

Georgius, I've been looking at the source code of this latest filter: I think it would be better to have three separate thread functions, each for its own transport. Currently the code is hardly readable because of all the branching.

I agree with hard readable code ... I separated each type of transport to its method. I don't think that separate thread for each transport protocol will be benefit.

Also please see my latest solution for logging. It doesn't require #ifdef in code.

I'll look on it

It's a shame we don't have a SVN for this. It's so hard for multiple people to work on the same file

Now we have two separate versions with different approaches. We must wait for one working version for all.

I'm continuing work on my version of filter because so far it's the only one I know that produces stutter free grabbing.

So am I for my solution. At least until you release working version for me Looking forward your new version to test.

In fact, it should be worse since it performs memcpy every now and then.

As I test until now, it's not worse or better ... without change, because memcpy is called when buffer is more than half full.

Anyway, added RTP support, new version in attachement

 

[velis]

Georgius, can you wait today so that I can rearrange the code of the filter?
I'd fix the logging and break up the transports (you didn't understand what I meant).

I promise not to insert any new code, just rearrange the existing code for readability
I'll post the altered code in the evening.

LP,
Jure

 

[georgius]

I'll wait ... anyway real testing of UDP protocol I can make only in night.

 

[tourettes]

Re: AW: Re: AW: Re: IPTV/DVB-IP in MP: Support for http, rtp and udp

I just had an idea about the linear buffer. It would also be possible to use a single chained list of buffer objects. You could create a pool of unused buffer objects. When you get data you can put the objects one after another and put them back when the data is delivered in FillBuffer(). With pointers to the first and last objects it would be very quick. You could also extend the buffer dynamicly and don't need to move the data. Just an idea.

You could possibly use the Allocator that DirectShow has. It has similar pool of IMediaSamples that could be used.

 

[georgius]

Re: AW: Re: AW: Re: IPTV/DVB-IP in MP: Support for http, rtp and udp

I think it's used. MPIptvSourceStream class derives from CSourceStream which derives from CBaseOutputPin which implements IMemAllocator interface. CBaseOutputPin implements method GetDeliveryBuffer() (which calls IMemAllocator.GetBuffer() method) which is called from MPIptvSourceStream to get free buffer.

 

[velis]

I'll wait ... anyway real testing of UDP protocol I can make only in night.

Same here Currently wife commands the TV

Edit: Here are the prettied sources with LogDebug defined such that it doesn't have to be #ifdeffed
Note that you have way too many files in your sources (baseclasses, kartina). Source compiles without those.
I have decided against modifying any threading code because the code there is already clean.
In fact, I was planning to change much more and now that I'm done, I'm kinda disappointed - the actual changes to the code are minor / cosmetic.

A few observations:
1. #define IPTV_SOCKET_BUFFER_SIZE 1024 * 1024
And other such defines:
I have determined this to have no effect whatsoever on losing packets. My latest filter variants don't modify this buffer precisely because it has no effect. 8KB (default) is too much, in fact - if you don't catch the packet as it comes in, you've lost it

2. I have found out why your filter is much more CPU stable than mine You're using normal priority threads... This means that each wait in the filter gives slices to other programs while in my filter it just returns straight back to the filter.
Will try my filter with normal priority to see how that behaves.