[fixed] Video and sound syncronisation (2 Viewers)

[Smeulf]

Just to make it clear for all :

Video leads the sound. I've been confused, and i'm sorry about it

Is it constant difference in the A/V sync or is there some drifting happening? Also how much the video is ahead if the diff is constant.

As soon as the video starts, there is a 50ms / 100ms delay (very hard to say as it's very short but perciptible) that seems to be constant.

It's not correct in the beginig and not becoming more and more unsynchronized, so no drifting. I did not ran a test on a long video to see if there is a drift at the end, I will.
And btw, if I skip IE 15min of video, the problem remains too.

Hope it will help.

Smeulf - what does the flickering/flashing look like ?

Tony

I'll check carefully this evening to determine exactly the kind of flickering.

Cheers.

 

[morpheus_xx]

AW: Video and sound syncronisation

Can you post a MediaInfo extract of your test video? I'd like to see details about video and audio streams.

Please also share a graphedit screenshot to see the used filters.

After this you should try different splitters and audio codecs too.

 

[Smeulf]

Re: AW: Video and sound syncronisation

Can you post a MediaInfo extract of your test video? I'd like to see details about video and audio streams.

Please also share a graphedit screenshot to see the used filters.

Please find as attachement a zip containing :

2 screen shots done from "MP1 and MP2 graphedit"
2 text files including all informations about the 2 graphs
a html file including information from the database about the test video

As always, works fine with MP1, not with MP2.

After this you should try different splitters and audio codecs too.

I can take time to try others codec / spliters, but it must help us to find out the real problem, cause as you will see with the screenshots or txt files, graph is exactly the same with MP1 and MP2, exept the presenter

I'm not sure it's related with codec/splitter, but I agree to try everything that would help to solve the issue.

Btw. I have the sound problem with all videos I have (AVI XVID, MKV XVID, TS, etc...) and with the SlimTV plugin too. It's not related with the number of audio streams, nor the subtitles.

I already tried to change the audio codec, but not the splitter. If you have some different configs in mind, please let me know in order to try.

Cheers.

Smeulf.

 

[Smeulf]

Smeulf - what does the flickering/flashing look like ?

Tony

Looks like old frames still presenting, but very not often.

Is it constant difference in the A/V sync or is there some drifting happening?.

No obvious drifting after 40 minutes

 

[Albert]

AW: Video and sound syncronisation

Morph, with that version of the EVR presenter, my problem still exists like before. About 1 wrong frame per 5 seconds (not regular, sometimes there are wrong frames directly one after each other, sometimes there are some seconds where no wrong frames are visible).

 

[Owlsroost]

The 'wrong frame' problem feels like a video sample queue management issue - possibly a buffer being freed before the sample has actually been rendered into the DWM(Aero) pipeline (allowing new data from the EVR mixer to overwrite the sample, so you see a frame from the future) ?

Or if it's 'old' frames coming around again, maybe the input side is occasionally taking a free buffer, marking it as used and putting it on the queue, but not actually updating the content ?

Tony

 

[Owlsroost]

Re: AW: Video and sound syncronisation

Can you please try this evr presenter?

I mainly did 3 changes:

• drop samples that are too late by 1/4 of the frame time (they were force to present)

Tony, shouldn't all samples that are late (even if 1 ns) be dropped to make sure there wont be drifting happening. A late frame will be scheduled always on the next v-sync on DirectX / GPU side so it will effectively move the A/V sync apart for one frame duration (20ms / 40ms on PAL material).

Actually in the current 'trunk' MP1 presenter the 'drop' limit = -displayFramePeriod/4, and the 'repeat/hold' limit = videoFramePeriod+(displayFramePeriod/4). This allows some latitude so that it doesn't get into a 'drop a frame then don't present the next in the queue because it's early' situation when the video & display rates are matched. So the effective A/V sync window/jitter is videoFramePeriod+(displayFramePeriod/2). Unless you use something like the MP Audio Renderer to provide feedback correction there is always at least one videoFramePeriod of jitter anyway.

Tony

 

[Albert]

AW: Video and sound syncronisation

Hi Tony!
Thanks for those explanations. Unfortunately, I'm not very used to frame scheduling. From what I saw in the code of our MP2 custom EVR presenter, I could not see any problem, I reviewed all the code. It's some time ago but as far as I remember, we use a queue implementation from one of the C++ standard libraries, so there should not be any +/- 1 problem in any counter which remembers the queue size.

One thing which could maybe cause our problem is the fact that we use the DX 9 Ex device from multiple threads, which should not be done afaik. I found code which uses a DeviceManager to synchronize multithread access to the DX 9 device. But that DeviceManager is not available from our C# side and as far as I understand, that class is part of some extended API (not standard DX 9 Ex), so it is not present in our SlimDX C# wrapper.

Maybe you can look into the MP2 code of our EVR presenter and check it yourself?

Thanks very much!

Albert

 

[tourettes]

Re: AW: Video and sound syncronisation

One thing which could maybe cause our problem is the fact that we use the DX 9 Ex device from multiple threads, which should not be done afaik. I found code which uses a DeviceManager to synchronize multithread access to the DX 9 device. But that DeviceManager is not available from our C# side and as far as I understand, that class is part of some extended API (not standard DX 9 Ex), so it is not present in our SlimDX C# wrapper.

Albert, do you still have my really old (over than year I would assume) DiretX 9 Ex patch for the MP2? It had a workaround for the threading - it was forcing the PresentEx() call to originate only from the EVR presenter's thread. It was used to "cure" the flashing frames issue.

I'm not that aware of MP2's today's codebase so I cannot say that if it is included or not. Anyway it should be fixed some more nice manner than the thread forcing that I was just using as a quick hack.

Also the EVR presenter's thread should be used to call the PresentEx() so that we would actually use the scheduling from the EVR presenter and not some "random" (from video's point of view) v-sync (and even frame) targets that the Skin Engine will generate.

 

[morpheus_xx]

AW: Video and sound syncronisation

today I spent some more hours in testing, here are some facts about current situtation:

1) Scheduling
The evr presenter schedules frames, present them if they are in due time (<1/4 of frame duration).
Here I found one issue: the "Kate Bush..." sample clip has a 29.97 frame rate, but on startup it's detected falsly as 25 fps. I have now fixed this inside scheduler by

hr = pSample->GetSampleDuration(&hnsSampleDuration);

to read the sample's duration. This divided by 4 is the new limit to callback.

2) Presenting
a) Renderloop is syncronized to evr callback and waits for a frame to present

b) A callback from evr presenter's scheduler is done to c# side, passing the surface. EVRCallback does draw this surface by:

          using (Surface surf = Surface.FromPointer(new IntPtr(dwSurface)))
          {
            GraphicsDevice.Device.StretchRectangle(surf, cropRect,
                _surface, new Rectangle(Point.Empty, _croppedVideoSize), TextureFilter.None);
          }

Then it set's the WaitEvent

c) Render loop continues, does:

        _device.Clear(ClearFlags.Target, Color.Black, 1.0f, 0);
        _device.BeginScene();
        _screenManager.Render();
          _device.EndScene();
          _device.PresentEx(Present.ForceImmediate);

This means, we use the ForceImmediate call that is not bound to vsync (see http://msdn.microsoft.com/en-us/library/bb172585(v=VS.85).aspx#D3DPRESENT_FORCEIMMEDIATE)

What I don't know here: does the _device.Clear() also clear the already handled surface GraphicsDevice.Device.StretchRectangle?

Anyway, for testing I split the render calls into two parts:

        _device.Clear(ClearFlags.Target, Color.Black, 1.0f, 0);
        _device.BeginScene();
        _screenManager.Render();

[B]// WAIT here for present callback[/B]

          _device.EndScene();
          _device.PresentEx(Present.ForceImmediate);

Not sure if it improves the situation, but my idea was to have to GUI already rendered and then we can immediatly PresentEx()...

3) Available statistics
The frame scheduling part can use collected statistics about PresentCount, PresentRefreshCount, SyncRefreshCount, SyncQPCTime, SyncGPUTime (IDirect3DSwapChain9Ex::GetPresentStatistics Method (Windows), D3DPRESENTSTATS Structure (Windows))

This can be used to improve the scheduling quality more.