Advanced Stream Switching / Graph Rebuilding (1 Viewer)

[nevcairiel]

Hi,

as you may (or may not) know, i'm the developer of LAV Splitter.
I've recently been thinking about how to improve the whole mess that is stream switching and the possibility to let the player build a custom filter graph while doing so.

The situation right now:
- (On Audio only) LAV Splitter will try to replace the audio decoder with a new one, trying to honor the users preferences. This method does not work with custom graph building routines, as it relys on the default GraphBuilder interface to select the new decoder.
- Other splitters do not do this at all, the MPC-HC splitter will never replace the original decoder, Haali will only replace it if the old one does not support the new codec.

The idea that came to mind (actually, tourette (a MediaPortal developer) triggered it) is as follows:
Just let the player deal with it, if it wants to.

The general concept is simple. The player needs to provide a yet non-existant interface, dubbed "IGraphRebuilder". This interface would hold simply one function, possibly called "HRESULT RebuildPin(IFilterGraph *pGraph, IPin *pPin)", which the splitter would call on stream changes to tell the Player to re-render the pin with its own graph building logic.

From my point of view, this method of stream changes would be optimal, as only the player really knows how its filter graphs should be constructed. However, i need to rally the support of player developers for this to make sense.

I will myself implement this in MPC-HC, and i'm hoping for the developers of JRiver MC16, MediaPortal and ZoomPlayer to eventually join the boat (at least i approached them about it). Any other players are of course welcome to implement it as well.

On to the technical stuff:

I discussed the IGraphRebuilder interface above, and there really isn't much about it. On stream changes, the splitter would set the new media types on the output pin, and call that interface with a reference to that output pin. Its then 100% the players responsibility to deal with the change. How that looks is all your choice, it can be as simple as checking if the existing decoder supports the codec, and only if it does not change it, or you can implement a full blown codec priority system.
I'm not 100% decided if the splitter should do anything else after the call. I could simply send the new media type to the codec with the next data package, to ensure it really has the updated information. I think i would base that on the return value, S_OK means "do nothing, we took care of it", S_FALSE means "the existing graph is fine, send new media type".

The main question that bugged me for a while was how would the player get a reference to that interface to the splitter. I think i found a neat solution: IObjectWithSite. Through the IObjectWithSite interface, objects can be assigned a "site", which is basically their owning object. The player would simply query the IObjectWithSite interface from the splitter, and set its class that holds the IGraphRebuilder interface as the new site.

The Splitter would then query the IGraphRebuilder interface from the site reference, whenever its needed.

Anyhow, this is not a "finished" spec, its mostly a draft of how things would work, and i think it could work out quite nice, allowing the player to switch the decoder based on its preferences, and nothing else.

I would appreciate any support, feedback and comments from the different player developers!

Cross-posted to MediaPortal Forum, MC16 Beta Forum and ZoomPlayer forums.

 

[tourettes]

At least one thing that comes to my mind is that if the graph rebuilding requests are done on pin basis it might cause two rebuilds on a format boundary (blu-ray, live tv...) since the audio and video streams are processed separately. Is there anything we could do to try to minimize such? From user point of view it is better to lost 0.25 seconds worth of audio or video data than have two playback hickups in a short period.

 

[nevcairiel]

AW: Advanced Stream Switching / Graph Rebuilding

If the splitter decides to change a Blu-ray title, or TV channel, it would stop the graph, and rebuild all pins in one go. There shouldn't be a "gap" between those rebuilds. It would just call the API twice, once for video, once for audio, and when done, restart the graph. Should be rather smooth, i would think.

 

[tourettes]

Re: AW: Advanced Stream Switching / Graph Rebuilding

If the splitter decides to change a Blu-ray title, or TV channel, it would stop the graph, and rebuild all pins in one go. There shouldn't be a "gap" between those rebuilds. It would just call the API twice, once for video, once for audio, and when done, restart the graph. Should be rather smooth, i would think.

It could work if the splitter would play the clip to end and cache the rebuild needs so that the rebuild is only done once and after all data has been played back from the current clip.

Stopping should be one only if nescessary (read: rebuild for at least one of the pins is needed). For example TsReader allows channel changes to be done on the fly, without stopping the graph if no rebuilding is required.

 

[tourettes]

How about adding the media type to the call back? It would probably make the client / player side code less complex and should be mininal change to the splitter side.

HRESULT RebuildPin(IFilterGraph *pGraph, IPin *pPin, CMediaType *pPmt)

 

[nevcairiel]

AW: Advanced Stream Switching / Graph Rebuilding

It could work if the splitter would play the clip to end and cache the rebuild needs so that the rebuild is only done once and after all data has been played back from the current clip.

Stopping should be one only if nescessary (read: rebuild for at least one of the pins is needed). For example TsReader allows channel changes to be done on the fly, without stopping the graph if no rebuilding is required.

Thats all up the splitter, of course. The interface only allows to delegate the rebuilding to the application. The splitter decides when to rebuild, and if to always stop or pause the graph, or let the player handle that as well.

LAV Splitter for example will always stop the graph, because that automatically clears the buffers, and makes the switch "instant", and the playback glitch is very minor.
The player shoud not make any assumptions on the state of the graph when it gets the callback, and always ensure its stopped when doing graph changes.

One thing i'm undecided about however. If the player gets a callback for video, and the graph is still running, then the player should stop the graph. But should it start it as well again? This could potentially cause the issue you descriped. I would rather see the splitter taking care of that, returning the graph to the playing state when its done calling all rebuild callbacks.

I'll properly formulate that in the interface description.

How about adding the media type to the call back? It would probably make the client / player side code less complex and should be mininal change to the splitter side.

HRESULT RebuildPin(IFilterGraph *pGraph, IPin *pPin, CMediaType *pPmt)

One pin can have many media types, IMHO its easier to just let the application probe pPin->GetMediaType, thats the same logic it used initially when creating the graph.

I have been thinking about adding a enum value to the call though, which specifiys the type of the pin, "video", "audio", "subtitle", "unknown". That way the player can directly go into the proper routine to figure out what filters are relevant.

 

[LordMerlin]

All functions related to the compulsory assignment and reassignment of decoders graphs has long been realized. See here:

AV Splitter 1.1 (Audio Video Splitter for Windows) | AV Splitter

Russian support Forum
AV Splitter

 

[nevcairiel]

AW: Advanced Stream Switching / Graph Rebuilding

Thats *exactly* what i don't want. It should be managed by the player, not any random DirectShow component. That way it always works in the same way, no matter what filters you use. Doing it in the splitter is a terrible design, something i'm trying to actively prevent. Only the player knows how it wants the graph to be built, a splitter just cannot know how to do it properly, and anyone believing otherwise is just wrong.

That AV Splitter is doing the exact opposite of what i'm trying to do. As a user, why would i have to configure it in the player, AND in the splitter? Its alot easier if i just configure it in the player once, and it "just works".

PS:
BTW, I don't care what features AV Splitter has or does not have. I'll work on improving my software, no matter what anyone else does, and this specific problem has actually been discussed longer on the LAV thread then AV Splitter existed.

 

[LordMerlin]

Correct if I understand you correctly.
You offer to do a feedback between the splitter and the player that the player can issue commands through this splitter which decoders are forcibly inserted into the graph to the right content is played back, regardless of what kind of mess going on with the user in the system in terms of filters and decoders.

 

[Sebastiii]

Hey
Anynews
Thanks.