Audio sync problems when screen set to 24 Hz (3 Viewers)

[Scythe42]

I'm still wondering why it seems to plague only 24fps files.

My conclusion so far with the current code: because people run them at their native refresh rate and there are no additional vsyncs that can be used to display late frames, so that playback can catch up without dropping frames. Also MAX_WAIT is used as a default when no timestamp is available, which puts additional lag on frames. In addition the used thread scheduling is not accurate enough. All in all it's a combination of these things that have impact on playing a file with an 1:1 refresh rate.

I can reproduce the same issues with 60Hz as well by providing a HD 60fps file. Try it with transcoding a 23.976fps file to 60fps and feed it to the player. The problem here is file fps = refresh rate.

So far I could reduce the dropped frames by allowing a drift in late frames up the maximum of 3/4 of the display time of a single frame based on a file's fps. It it's already late, so it can be late up to the next frame, it wouldn't make a difference. Some lagging builds up but you can't really see it until a frame needs to be dropped to catch up again. I'm very sensitive to lip-sync issues but couldn't detect this one unless a few frames got dropped. The current code does this with 15ms, but with 24Hz we have more room to breath here.

Still room for enhancements. But one thing after another. First an acceptable frame dropping for material played at the same frame rate, then on to a better thread scheduling for minimizing late frames and finally go for an own vsync clocking to avoid tearing. Just taking it evolutionary step by step here. I have an updated patch for testing available soon.

PS: I think I found out some stuff why Win 7 works better. Looks like the thread scheduling has been updated and is more accurate in general than under Vista.

 

[tourettes]

My conclusion so far with the current code: because people run them at their native refresh rate and there are no additional vsyncs that can be used to display late frames, so that playback can catch up without dropping frames. Also MAX_WAIT is used as a default when no timestamp is available, which puts additional lag on frames. In addition the used thread scheduling is not accurate enough. All in all it's a combination of these things that have impact on playing a file with an 1:1 refresh rate.

I can reproduce the same issues with 60Hz as well by providing a HD 60fps file. Try it with transcoding a 23.976fps file to 60fps and feed it to the player. The problem here is file fps = refresh rate.

Ah, now I finally see why only the 24 fps has been plagued with the bad scheduler code.

So far I could reduce the dropped frames by allowing a drift in late frames up the maximum of 3/4 of the display time of a single frame based on a file's fps. It it's already late, so it can be late up to the next frame, it wouldn't make a difference. Some lagging builds up but you can't really see it until a frame needs to be dropped to catch up again. I'm very sensitive to lip-sync issues but couldn't detect this one unless a few frames got dropped. The current code does this with 15ms, but with 24Hz we have more room to breath here.

Could you check with PowerStrip what is the exact refresh rate of your 24Hz mode? If it is not 24.000Hz then we have already some drifting (and if audio HW is also something else than 100% exact in its timing the effect is same).

This is the are I have been working already, so I think we should combine our effort so we don't do duplicate work. On my dev PC I have 75.025Hz video clock (so it is not possible to playback PAL material without stuttering caused by "random" duplicate frames) but with the improvements I have been working on (based on some other open source code) it is possible to play without stuttering.

Still room for enhancements. But one thing after another. First an acceptable frame dropping for material played at the same frame rate, then on to a better thread scheduling for minimizing late frames and finally go for an own vsync clocking to avoid tearing. Just taking it evolutionary step by step here. I have an updated patch for testing available soon.

Actually we don't need to implement own syncing to cure the tearing (as there won't be any when using AERO + DirectX. Those both are handling the v-sync correctly).

PS: I think I found out some stuff why Win 7 works better. Looks like the thread scheduling has been updated and is more accurate in general than under Vista.

Could be, or Windows 7 has less buggy / CPU hogging drivers on your HW. As badly behaving drivers are usually the cause for non-stabile timers on Windows (bad driver can block code execution for 10 ms easialy, or for seconds of there is some bug )

 

[Scythe42]

Ah, now I finally see why only the 24 fps has been plagued with the bad scheduler code.

With 1:1 Frame Rate the issue is reproducible every time because of improper frame dropping. I continued experimenting with the code.

So far I made the following adjustments to include proper frame dropping in my current tests:
- Drop frames if another frame is already due (similar to the existing code)
- When no time stamp is available set delay to 0, instead of MAX_WAIT, so that there is no artificial introduced lag
- Delay is never set to min of scheduled time and MAX_WAIT, instead use the correct delay, but schedule everything up to 5ms early to compensate for scheduling inaccuracy (current test case) for now.

When playing back 23.976fps material at 24Hz, a frame needs to be dropped in theory about every 41,66 seconds to avoid any drifting as with 1:1 frame rate there is no chance to catch up on additional vsyncs. I deliberately choose this setup, so that I need to drop frames or get lip-sync issues over time.

From the logs I see that I have a dropped frames every 34 seconds. No "late scheduled frames" log messages anymore unless something else is causing it (filter, driver, clock etc). That's a big improvement. Need to get it a little bit closer though. Still not good enough, but at least constant, which should make it easier to find what I did wrong.

Skipping is very accurate, fast forward works smooth (need to find out how to detect it properly to avoid "dropping frame" log messages). Rewind is currently broken because I don't detect fw/rew at the moment and rewind is always detected as frames being behind. Also need to test with DVDs.

I'd say I found the root cause, now it's just to include the required frame dropping properly without breaking anything else and testing it at various fps vs. refresh rate combination with various filters. From there one we should combine our efforts for further enhancements you and others are already working on. But I need to play around a bit more with the code to fully understand how an EVR presenter works. Not there yet...

Could you check with PowerStrip what is the exact refresh rate of your 24Hz mode? If it is not 24.000Hz then we have already some drifting (and if audio HW is also something else than 100% exact in its timing the effect is same).

Installed Powerstrip. Completly screwed up my video driver after installation. I now can only select 30Hz and nothing more *grmpf*. Need to fix this.

I can choose between 23Hz and 24Hz on my NVidia graphics cards. 23Hz reports 23.971Hz and 24Hz reports 24.000Hz. Audio clock seems to not perfectly stable (ReClock show this). I really don't trust these reported 23.971Hz. It seems to be strangely off only a bit. I need to check for some other tools that report me this stuff in more detail. Powerstrip is a no-go for me.

I'll continue playing with the code and doing some more testing later this week: one adjustment, watch a movie. If it's worse, go back a step. If it looks good watch the whole movie and check the logs afterwards. Guess I need a few fresh Blu-Rays for this. Amazon, here I come

 

[Seeco]

I just wanted to say that this is by far the most exciting developer thread I've read in a long time! I used to have really severe sync drift, now I'm running Windows 7 and things seem a lot better. As I understand it though, what you guys are doing won't only make sync close to perfect, but will also help perfect one of the holy grails of the HTPC world (as opposed to dedicated dvd players etc.) - highest possible video smoothness.

A big thanks to Scythe42 and the others contributing to this thread!

 

[tourettes]

Ah, now I finally see why only the 24 fps has been plagued with the bad scheduler code.

With 1:1 Frame Rate the issue is reproducible every time because of improper frame dropping. I continued experimenting with the code.

So far I made the following adjustments to include proper frame dropping in my current tests:
- Drop frames if another frame is already due (similar to the existing code)
- When no time stamp is available set delay to 0, instead of MAX_WAIT, so that there is no artificial introduced lag
- Delay is never set to min of scheduled time and MAX_WAIT, instead use the correct delay, but schedule everything up to 5ms early to compensate for scheduling inaccuracy (current test case) for now.

Sounds like you have made good progress on improving the scheduling algorithm. About that 5 ms, that propably needs to be calculated based on the refresh rate? btw. do we know what would be the optimal time for painting the frame? How much early to v-sync we should be? (We could calculate some average based on the time it takes to render the frame on D3D and adjust the 5 ms sync spot with it). Of course this will be the fine tuning phase, and doesn't have to be done for 1.1.0 release (but well see how the testing & implementing goes.)

When playing back 23.976fps material at 24Hz, a frame needs to be dropped in theory about every 41,66 seconds to avoid any drifting as with 1:1 frame rate there is no chance to catch up on additional vsyncs. I deliberately choose this setup, so that I need to drop frames or get lip-sync issues over time.

That is the problem domain I'm currently working on (playing video stutter free on non-matching Hz/fps or GPU & audio clocks drifting away from each other) It won't allow 24 fps to be played on 25 / 50 Hz like ReClock does, but those small inaccuraties like display or GPU not capable for 23.976. So, at least you don't currently have to worry about that juddering / stuttering you are seeing with 41.66 s intervals.

From the logs I see that I have a dropped frames every 34 seconds. No "late scheduled frames" log messages anymore unless something else is causing it (filter, driver, clock etc). That's a big improvement. Need to get it a little bit closer though. Still not good enough, but at least constant, which should make it easier to find what I did wrong.

Theoretical 41.66 second interval turning into 34 second interval could be maybe explained with for example that we cannot present the frame on exactly the same time as v-sync happens. I'm not sure how much safe zone there is (time to try to Google it, but I guess it will be hard to find any good answers...). Mabe there is also similar safe zone after the page has been flipped? I'm really newbie when it comes to the DirectX

But sill, I'm quite sure that the juddering on 34 second interval you are seeing is definitely caused by the 23.976 vs. 24 difference.

Skipping is very accurate, fast forward works smooth (need to find out how to detect it properly to avoid "dropping frame" log messages). Rewind is currently broken because I don't detect fw/rew at the moment and rewind is always detected as frames being behind. Also need to test with DVDs.

Normaly there would be the SetRate( or was it something else, don't have the source codes at the moment) call to the presenter, but unfortunately there is no audio renderer that supports > 2.0x speeds so MP has to fake the REW/FWD with seek + play loopping. Currently I have no idea how you could detect the REW/FWD (I'll let you know if I figure something out).

I'd say I found the root cause, now it's just to include the required frame dropping properly without breaking anything else and testing it at various fps vs. refresh rate combination with various filters. From there one we should combine our efforts for further enhancements you and others are already working on. But I need to play around a bit more with the code to fully understand how an EVR presenter works. Not there yet...

Yep, sounds like the root cause is pin pointed and fixing is already progressing nicely. Next step would be to provide a dshowhelper.dll for our user base to be tested. If you provide the patch for the source codes I'll arrange a new thread for testing purposes so that we can get more test coverage (not just the people who have issues with 24 Hz playback.)

Installed Powerstrip. Completly screwed up my video driver after installation. I now can only select 30Hz and nothing more *grmpf*. Need to fix this.

I can choose between 23Hz and 24Hz on my NVidia graphics cards. 23Hz reports 23.971Hz and 24Hz reports 24.000Hz. Audio clock seems to not perfectly stable (ReClock show this). I really don't trust these reported 23.971Hz. It seems to be strangely off only a bit. I need to check for some other tools that report me this stuff in more detail. Powerstrip is a no-go for me.

Too bad that you are having issues with PowerStrip, as it is pretty accurate (more accurate than DirectX itself) when it comes to reporting refresh rates. By any change do you have NVidia 8xxx/9xxx/2xx series GPU? Those aren't fully supported by PowerStrip as Nvidia hasn't released needed documentation of their chips.

I'll continue playing with the code and doing some more testing later this week: one adjustment, watch a movie. If it's worse, go back a step. If it looks good watch the whole movie and check the logs afterwards. Guess I need a few fresh Blu-Rays for this. Amazon, here I come

Testing sync issues is a bit boring stuff, unless you happen to buy a really good movie

 

[Scythe42]

About that 5 ms, that propably needs to be calculated based on the refresh rate? btw. do we know what would be the optimal time for painting the frame?

I just chose a random small offset here to see if it had any impact. Since my first tries with the scheduler I just took the actual delay and subtracted 5 ms to be sure the thread will never be late. I haven't touched this area since then. Now it's time to revisit this area before finalizing the patch.

Current working assumption: because the delay is an integer (if I'm not mistaken) derived out of delta/10000 threads woke up late, because the value is rounded. This introduced and additional lag that build up over time and wasn't compensated properly. Need to check what a trace has to say about this. So far it's just a hunch and can be total BS.

Theoretical 41.66 second interval turning into 34 second interval could be maybe explained with for example that we cannot present the frame on exactly the same time as v-sync happens. [..]But sill, I'm quite sure that the juddering on 34 second interval you are seeing is definitely caused by the 23.976 vs. 24 difference.

Makes sense. It was constant and that's a good thing. Probably some additional latency is accumulating so that after 34 seconds a frame is finally more than 41.66ms late and needs to be dropped to keep audio/video in sync. I need to take a look at the timestamps if there's really a constant drift of roughly 0.2ms in this scenario. 19 frames dropped too much per hour of played video is not acceptable . Microseconds, hours of playback. That's progress.

Normaly there would be the SetRate( or was it something else, don't have the source codes at the moment) call to the presenter, but unfortunately there is no audio renderer that supports > 2.0x speeds so MP has to fake the REW/FWD with seek + play loopping.

I see. I tested with the original DLL. Rewind seems to be broken there as well. No frames displayed, only when you change the rewind speed. So I won't investigate this any further at the moment.

Next step would be to provide a dshowhelper.dll for our user base to be tested. If you provide the patch for the source codes I'll arrange a new thread for testing purposes so that we can get more test coverage (not just the people who have issues with 24 Hz playback.)

I provide it as soon as I finished my testing scenarios. I probably won't touch the code anymore unless I find a newly introduced issue.

From my latest test with the same file as usual:

13-09-2009 14:02:48.173 [c28]OnClockStart
13-09-2009 14:02:48.173 [c28]Flushing: size=0
13-09-2009 14:08:38.112 [12b4]OnClockPause
13-09-2009 14:08:38.118 [12b4]OnClockStop

Roughly 6 minutes at the 23.971Hz my GPU reports (I'm still think the driver reports inaccurate or it drifts a bit). No dropped frames. Usually there are some when playback starts, but not always. No clue why (probably frames are already provided but the presenter is not fully ready). But I don't really care at the moment if the first view frames need to be dropped to get in sync. No drift, no late frames. Under the original DLL playback would have drifted already for a few frames by now. Audio was still spot on.

By any change do you have NVidia 8xxx/9xxx/2xx series GPU? Those aren't fully supported by PowerStrip as Nvidia hasn't released needed documentation of their chips.

It's an onboard 9300.

Testing sync issues is a bit boring stuff, unless you happen to buy a really good movie

WALL-E, Dark Knight and Eagle Eye are in the mail. Haven't seen them in fully 1080p glory yet. Also it's time for my yearly Godfather I+II double feature. So I'm set for now...

 

[tourettes]

Current working assumption: because the delay is an integer (if I'm not mistaken) derived out of delta/10000 threads woke up late, because the value is rounded. This introduced and additional lag that build up over time and wasn't compensated properly. Need to check what a trace has to say about this. So far it's just a hunch and can be total BS.

Yesterday evening I did notice that the current GetCurrentTimestamp() method is not accurate! (Same applies to the MPC-HC as they are sharing the same code ). It will lose some precision. It should be enough precise to be in ms range, but maybe it could as well lead into some minor drifting during hours? (I noticed this as I wasn't able to time code execution time properly. multiple GetCurrentTimestamp() calls are providing the same timestamp when QueryPerformanceCounter() is not ever going to be producing same tick count as the method itself has a slight overhead... so the GetCurrenttimestamp() needs to be fixed. I'll try to do it this evening).

I see. I tested with the original DLL. Rewind seems to be broken there as well. No frames displayed, only when you change the rewind speed. So I won't investigate this any further at the moment.

Ok, then it is most likely some other bug (some users have reported it) but on my HTPC & dev PCs seeking is working fine.

I provide it as soon as I finished my testing scenarios. I probably won't touch the code anymore unless I find a newly introduced issue.

I'm already waiting for the patch

By any change do you have NVidia 8xxx/9xxx/2xx series GPU? Those aren't fully supported by PowerStrip as Nvidia hasn't released needed documentation of their chips.

It's an onboard 9300.[/QUOTE]

ok, that might be as well not fully supported. At least it is produced after 8xxx series (I guess it is based on the 9xxx GPUs).

 

[Seeco]

Just a quick question: Will the changes you describe be a part of the next svn release?

 

[tourettes]

Just a quick question: Will the changes you describe be a part of the next svn release?

If you were asking about the changes I have been talking about then the answer is "Highly unlikely that those are available in 1.1.0 release. 0.1% )

And if about Scythe42's changes then answer is "definitely not in next SVN, but after public testing has been done..."

We are talking about modifying the core component. Changing something that could have some nasty effects on all Vista / Windows 7 (or XP with EVR) PCs needs to be tested properly, before adding it into SVN (even when SVN is not stable by a definition).

 

[Seeco]

OK, thanks for the info. I'll be happy to test whatever you guys come up with, this all sounds really exciting to me.