Audio sync problems when screen set to 24 Hz (2 Viewers)

[Scythe42]

Quick update: rewind and fast forward showed some issues as I feared. The presenter cannot reliable detect this at the moment and starts to drop frames like hell on higher speeds. Everything else seems to be OK so far.

With the help of tourettes (thanks for the code!), MP itself can now send events to the presenter and take control over playback behavior from an application logic level. This makes things easier on the EVR presenter side as I don't have to do complex analysis of sample times, deltas of sample times and other stuff.

Currently rew/ff with transport stream files seems to be the last hurdle. Need to investigate this a bit more in detail.

 

[Seeco]

Sorry to nag, but could anybody try to answer my last question? I think it has some bearing on this subject, because there has been a lot of discussion about the correct setting of v-sync vs image flow.

 

[Scythe42]

I haven't read the ReClock docs, but here is what I assume it does from playing around with it. If I am wrong on something please correct me.

When using Win 7 with latest MP beta, latest SAF version and Reclock as renderer, should I or should I not use V-Sync in Reclock?

Short answer:

No, ReClock tries to assure that each frame is displayed at a multiple of the refresh rate currently selected for the GPU to avoid judder caused by this difference. That should be enough if fps/refresh rate differ. If you use 1:1 frame rate don't let Reclock touch anything (except for bitperfect audio playback and maybe channel adjustments).

I have never fully grasped what V-sync does (or the logic behind the adjustment of it in Reclock) but I think it might be related to what's being discussed in this thread, right?

Long answer:

Back in the days of CRTs there was a ray running around and lighting each pixel on the tube, because of phosphor decay the pixels need constantly be refreshed with energy to be lightened up. Once the ray came the the end of the screen (lower right corner) it had to trace to the beginning (upper left corner) in one straight motion. To avoid stuff like tearing, meaning a picture is changed when the ray is in the middle of the picture and already starts to display the next picture new picture information is provided during this vertical retrace period. That's the vertical synchronization: keep the display of new frames in line with vertical retrace of the cathode ray. The time it takes the ray to to get back to the start is often called the vertical blank.

On a PC it could happen that frames are send to late to the renderer, which means the renderer has to wait for the next vertical retrace before displaying the picture or else you would see juddering or tearing if a frame is displayed immediately or dropped by the renderer.

There is one important difference here when it comes to frame dropping. Frames need to be dropped when they are not in time, usually caused by codecs or CPU/GPU limitations. It simply means the frame was never early enough for being delivered to the renderer in the first place. Among other things this is what a presenter does: It takes data from the mixer, schedules frames for delivery to the renderer, which then sends it to the GPU. If something isn't on time it doesn't even bother sending it to the renderer. The renderer itself tries to render what it can. Here GPU limitations come in.

If you estimate from a software when the vsync happens you can be off. And even though you got the frame from the mixer in time you could send it too late to the renderer. That's the difference here. Frame was never received in time by a presenter vs. frame was not send at the correct time by the presenter. That's a huge difference. and ReClock's vsync correction deals with the second one.

The PC itself doesn't care about the display device. For him it's important that there is enough time to deliver the next frame to the GPU before the vertical retrace is completed. Of course this does not exist on non CRTs, but they once existed and technology was created to deal with that. That's why even the video processor in a modern LCD still thinks in this terms even if the LCs don't require any refresh (LCDs don't flicker). And for computer usage they agreed that LCDs accept frames at 60Hz for computer usage. What they do with it doesn't matter from this point on. Of course with video playback on computer and LCD TV's video processor started to handle more rates. In theory any rate up to the reaction time of a liquid crystal can be handled. And GPUs also need to support some rates. Usually it makes no sense to support rates on an LCD for which no material exist.

What does ReClock try to achieve with the Vsync Correction:
It tries to solve the problem by assuring that the vertical retrace doesn't happen at the same time as a new frame is delivered to the renderer. This can be a problem if the renderer doesn't accept new frames once the vertical retrace has begun. It only works if ReClock adjust's the frames to match the refresh rate. Again this should only be an issue if the vsync is estimated by the software and the software takes full control over when a frame is send to the renderer and the renderer displays it right away.

In case of MP, DirectShow handles this. Frame is send to the renderer sometime before the vsync and the Renderer waits for the vsync before sending it to the GPU. Driver options in regards to vsync might interfere here. Usually best practice is to let the application decide. If you run into problems, force it in the driver. If you still experiencing problems and can't solve them with various settings in a player, then ReClock needs to be used.

Depending on what options you enable in ReClock, it can show you when the vsync happens. From a PC point of view this is basically the time when the first pixel is displayed (the end of the vertical retrace and the new frame begins, what is usually called vsync these days). In Reclock you see various points on the left side. Every point is for one vsync in the last 100 frames. In addition there is a horizontal line. If the total heights of points is small < 10-15% of the video height the playback can be considered as very good. The position of this horizontal line is also important because they can show when frames are provided to the renderer. Movement of the horizontal line also tells that ReClock is trying to find the correct clock. If it constantly moves ReClock had a problem to get the correct clock.

Now don't get paranoid if you see the horizontal vsync line in the middle of the picture. Also not if the pixel on the left are spread apart very wide. Sometimes it's simply the codec that is not delivering frames at an accurate rate. Try to compare ffdshow with MPC for example and you see what I mean. Unless you see any juddering or tearing don't worry about it.

Usually ReClock shows the best effect with Overlay Mixer or VMR7. With VMR9/EVR you don't really have to care about it as there is a internal DirectShow function that takes control of the vsync estimation. Basically a pre-rendered frame is only send to the GPU during the vertical retrace depending on the selected refresh rate. That's why the EVR presents for example send frames early to the renderer. DirectShow's Vsync will take care of the rest.

For MP this shouldn't really be an issue, except for compensating fps vs. refresh rate. And when you use DVXA it shouldn't matter at all because the GPU should even take more responsibility for displaying frames on time. If you use software rendering the VSync correction might be an option for you.

Try it out. If you can't see a difference at once (don't turn on the vsync display, as it will fool your perception) don't think about it anymore.

Yes, DirectShow is also not perfect here, but it's as good as it gets. And Win 7 again has media enhancements over Vista. Not as much as Vista had over XP but everything is welcome.

 

[Seeco]

Wow, what a great reply! You must be the delight of whatever tech department you work on. Since I'm running Windows 7 (EVR) and since I aim to use as much DXVA as possible, I think I will opt not to fiddle with vsync at all. Again, thanks alot for that!

 

[Scythe42]

Wow, what a great reply! You must be the delight of whatever tech department you work on.

Thanks. Please tell this to my line manager. I could use a raise and a more challenging job

 

[onyx666]

how should reclock be setup ?
just install it and put it in MP as audio renderer ?
I am using 1.1 beta with win 7 and saf

 

[tourettes]

For MP this shouldn't really be an issue, except for compensating fps vs. refresh rate. And when you use DVXA it shouldn't matter at all because the GPU should even take more responsibility for displaying frames on time. If you use software rendering the VSync correction might be an option for you.

Actually there is a 2 nd use for ReClock as well. Even when display refresh rate matches the media fps there is almost on all PCs a issue with audio and video clock drifting as they both are handled with different HW crystals.

Audio renderer is used as the source for graph's reference clock and thus if the clocks are running on different speed (0.001 % is already enough) there will be dropped frame(s) or juddering eventually. ReClock combats this with resampling the audio stream a little bit so that the reference clock created by the audio renderer / HW is modified slightly on the fly in such amounts that the video renderer without slipping away from the presentetation interval that the physical display device (TV / monitor) is setting.

...this is getting a little bit off topic, so please open a new thread for the ReClock as it is it's own world completely. Slysoft's forum is a good place to gather some knowledge on how ReClock works (and what issues it has ).

 

[Scythe42]

Actually there is a 2 nd use for ReClock as well. Even when display refresh rate matches the media fps there is almost on all PCs a issue with audio and video clock drifting as they both are handled with different HW crystals.

Oh yes I forgot this one as I have Reclock running in the background for bit perfect playback for multi channel flacs and audio channel compensation. Even though it doesn't touch the video speed or does some vsync stuff I think it adjusts the clock a little bit now and then.

...this is getting a little bit off topic, so please open a new thread for the ReClock as it is it's own world completely.

Then back on topic: I sent you another update on the patch.

I think I'm really close now. For the first time rew/ff seems to work properly on my machine. Of course there are some codec limitations and a few oddities. But they are not related to any frame dropping or bad scheduling so far. It looks very promising. We're really getting close here to a public test. Maybe one additional round of fine tuning is needed to compensate for some non presenter related issues. Let's hope I didn't break anything else with the recent adjustments...

 

[johan_80]

Great to see progress in this area.

In regards to this comment:
"With the help of tourettes (thanks for the code!), MP itself can now send events to the presenter and take control over playback behavior from an application logic level. This makes things easier on the EVR presenter side as I don't have to do complex analysis of sample times, deltas of sample times and other stuff."

...I wonder if this ability to communicate with the presenter could solve this problem:
https://forum.team-mediaportal.com/505539-post101.html


The problem and "the dirty fix" is later confirmed in the same thread.

It seems like the presenter (renderer) is not updated with the new refresh rate (frame rate). The same problem and the same fix is also present when using Vista Media Center/Windows 7 Media Center. The author of Media Control (plugin that can take care of refresh rate swithing in Media Center) found out that Media Center needed to be minimized then maximized to have the presenter refreshed.

 

[Seeco]

johan_80: I'm seeing that problem too, although I hadn't realized that it had something to do with RR changing. For me, when I start a video it sometimes plays normally, then when I fast forward it goes into this extremely stuttery mode (more of a slide show than a video).