This was one of the problems that got me started on this whole development thread a long time ago.....
What happens is that Windows will sometimes block the rendering until the next vsync (which shows up as the long render time), then sometime later it seems like a pipeline gets unblocked and it returns to normal.
I've never managed to work out why (and I've tried all sorts of ideas to fix/work around it). It got much less frequent when the vsync correction was added, but it still happens sometimes - and it sometimes seems worse with 24Hz video...
My theory for that render time jump and then one dropped frame when it returns is quite simple. Reference clock is running a bit different speed than the GPU's HW clock and therefore there is drifting between those two clocks. At some point it is not possible to present on the targeted v-sync and then it will cause all the paint times to jump up approx the amount of one display refresh cycle.
I would suggest to try the MediaPortal Audio Renderer since it was build to combat that issue. I haven't ever seen that issue to appear during almost two years that I have been developing / using MPAR At least it is worth to try to see if the issue goes away. If it goes then there are only two possible solutions: 1) to use MPAR or 2) to find some HW combination that matches 1:1 on the video and audio clocks.