MediaPortal Audio renderer - better video playback quality (1 Viewer)

Henkie Flits

Portal Pro
November 1, 2008
210
44
Home Country
Netherlands Netherlands
Tourettes, regarding feedback: most of my file play perfectly, neatly synced. I'm using V12 of the renderer on my system.
The only problem I have is that while playing HD video and audio the searching doesn't work properly: often the screen hangs on the current frame while I can hear the audio already of the scene which I skipped to. I will try to reproduce this issue today so I can produce some logs.
 

tourettes

Retired Team Member
  • Premium Supporter
  • January 7, 2005
    17,301
    4,800
    Tourettes, regarding feedback: most of my file play perfectly, neatly synced. I'm using V12 of the renderer on my system.
    The only problem I have is that while playing HD video and audio the searching doesn't work properly: often the screen hangs on the current frame while I can hear the audio already of the scene which I skipped to. I will try to reproduce this issue today so I can produce some logs.

    Please try with directshow audio renderer as well. Sounds more like a video codec or source splitter issue (I have seen such with ds renderer as well on some divx files).
     

    tourettes

    Retired Team Member
  • Premium Supporter
  • January 7, 2005
    17,301
    4,800
    I need help with math!

    I'm trying to implement audio HW based reference clock to stop A/V sync drifting when audio clock drifts away from the system clock. Normally this drifting shouldn't matter / be visible, but on some "broken" HW it could be visible in longer run.

    As the basic information: About how to implement IReferenceClock interface?

    What we have are three different values:

    • System time - performance counter that is able to provide monotonically increasing timestamps, much like the coccoo clock on the wall at your parent's house :)
    • Audio playback buffer position - this increases monotonically as long as the stream is playing
    • System time during the audio playback buffer query - this is the system time value when audio buffer position was queried

    How the audio buffer position behaves / is visible to us (to make it easier, we can consider this as a timestamp that is in same format and "size")

    • Buffer position can stay still when no stream is playing - this should be visible as 0 in the queried timestamp value
    • Advances in bursts - buffer position could for example advance in 2 ms steps, but we vould like to extrapolate the values between those steps with the system time that we have (it is much more accurate than the 2 ms)
    • Returns negative values at the beginning - most likely the buffer "flushing"
    • Skips backwards in the beginning...

    So, how to generate a stream of continuous, monotonically increasing timestamps that are going forward in the speed of the audio buffer?

    Sample timesamps:

    qpcNow == system time from performance counter
    TS == audio buffer position
    qpc == time when audio buffer position was read

    Code:
    25-09-2010 22:11:54.594 [ 172c] TS: 0 qpcNow: 0 qpc: -9536744406748
    25-09-2010 22:11:54.595 [ 172c] TS: 0 qpcNow: 664 qpc: -9536744406748
    25-09-2010 22:11:54.595 [ 172c] Pause
    25-09-2010 22:11:54.598 [ 172c] TS: 0 qpcNow: 31026 qpc: -9536744406748
    25-09-2010 22:11:54.612 [ 1464] TS: 0 qpcNow: 171524 qpc: -9536744406748
    25-09-2010 22:11:54.798 [  ea8] TS: 0 qpcNow: 2031054 qpc: -9536744406748
    25-09-2010 22:11:54.798 [  ea8] TS: 0 qpcNow: 2039466 qpc: -9536744406748
    25-09-2010 22:11:54.809 [  ea8] TS: 0 qpcNow: 2146746 qpc: -9536744406748
    25-09-2010 22:11:54.810 [  ea8] TS: 0 qpcNow: 2154340 qpc: -9536744406748
    25-09-2010 22:11:54.813 [ 15e8] TS: 0 qpcNow: 2184296 qpc: -9536744406748
    25-09-2010 22:11:54.813 [ 15e8] Run
    25-09-2010 22:11:54.897 [ 15e8] TS: 0 qpcNow: 3025716 qpc: 3025848
    25-09-2010 22:11:54.897 [ 15e8] TS: 0 qpcNow: 3027036 qpc: 3027135
    25-09-2010 22:11:54.897 [ 15e8] TS: 0 qpcNow: 3027482 qpc: 3027557
    25-09-2010 22:11:54.898 [ 1464] TS: 0 qpcNow: 3036512 qpc: 3036767
    25-09-2010 22:11:54.898 [ 1464] TS: 0 qpcNow: 3037562 qpc: 3037673
    25-09-2010 22:11:54.899 [ 1464] TS: 0 qpcNow: 3043546 qpc: 3043756
    25-09-2010 22:11:54.899 [ 1464] TS: 0 qpcNow: 3044308 qpc: 3044411
    25-09-2010 22:11:54.899 [ 1464] TS: 0 qpcNow: 3047500 qpc: 3047615
    25-09-2010 22:11:54.899 [ 1464] TS: 0 qpcNow: 3048012 qpc: 3048104
    25-09-2010 22:11:54.900 [ 1464] TS: 0 qpcNow: 3051342 qpc: 3051456
    25-09-2010 22:11:54.900 [ 1464] TS: 0 qpcNow: 3051851 qpc: 3051944
    25-09-2010 22:11:54.900 [ 122c] TS: 0 qpcNow: 3052559 qpc: 3052675
    25-09-2010 22:11:54.918 [ 122c] TS: 500000 qpcNow: 3230592 qpc: 3231092
    25-09-2010 22:11:54.959 [ 1464] TS: 112083 qpcNow: 3640895 qpc: 3641262
    25-09-2010 22:11:54.959 [  9ac] TS: 113125 qpcNow: 3642298 qpc: 3642422
    25-09-2010 22:11:54.959 [  9ac] TS: 113958 qpcNow: 3643036 qpc: 3643141
    25-09-2010 22:11:54.961 [ 1464] TS: 137916 qpcNow: 3650620 qpc: 3667110
    25-09-2010 22:11:54.962 [ 1464] TS: 142500 qpcNow: 3671613 qpc: 3671783
    25-09-2010 22:11:54.962 [ 1464] TS: 143125 qpcNow: 3672329 qpc: 3672441
    25-09-2010 22:11:54.962 [ 1464] TS: 146666 qpcNow: 3675802 qpc: 3675926
    25-09-2010 22:11:54.962 [ 1464] TS: 147291 qpcNow: 3676371 qpc: 3676462
    25-09-2010 22:11:54.962 [ 1464] TS: 150208 qpcNow: 3679359 qpc: 3679472
    25-09-2010 22:11:54.962 [ 1464] TS: 150833 qpcNow: 3679940 qpc: 3680032
    25-09-2010 22:11:54.963 [ 1464] TS: 156875 qpcNow: 3685818 qpc: 3686061
    25-09-2010 22:11:54.964 [ 1464] TS: 162083 qpcNow: 3691197 qpc: 3691448
    25-09-2010 22:11:54.974 [ 1464] TS: 172083 qpcNow: 3700998 qpc: 3701454
    25-09-2010 22:11:54.974 [  d58] TS: 269583 qpcNow: 3798419 qpc: 3798838
    25-09-2010 22:11:54.975 [  9ac] TS: 271875 qpcNow: 3800928 qpc: 3801048
    25-09-2010 22:11:54.975 [  9ac] TS: 272500 qpcNow: 3801575 qpc: 3801667
    25-09-2010 22:11:54.975 [ 1464] TS: 274166 qpcNow: 3803236 qpc: 3803366
    25-09-2010 22:11:54.975 [ 1464] TS: 278541 qpcNow: 3807707 qpc: 3807860
    25-09-2010 22:11:54.975 [ 1464] TS: 279166 qpcNow: 3808412 qpc: 3808512
    25-09-2010 22:11:54.976 [  9ac] TS: 281458 qpcNow: 3810602 qpc: 3810708
    25-09-2010 22:11:54.977 [  9ac] TS: 298750 qpcNow: 3827593 qpc: 3828076
    25-09-2010 22:11:54.977 [  9ac] TS: 300000 qpcNow: 3829097 qpc: 3829195
    25-09-2010 22:11:54.977 [  9ac] TS: 300833 qpcNow: 3830057 qpc: 3830145
    25-09-2010 22:11:54.978 [  59c] TS: 302916 qpcNow: 3831970 qpc: 3832088
    25-09-2010 22:11:54.978 [ 1464] TS: 305000 qpcNow: 3834107 qpc: 3834268
    25-09-2010 22:11:54.978 [ 1464] TS: 305833 qpcNow: 3834906 qpc: 3835023
    25-09-2010 22:11:54.981 [  9ac] TS: 337083 qpcNow: 3866139 qpc: 3866442
    25-09-2010 22:11:54.981 [  d58] TS: 337916 qpcNow: 3867048 qpc: 3867148
    25-09-2010 22:11:54.981 [  9ac] TS: 338333 qpcNow: 3867602 qpc: 3867694
    25-09-2010 22:11:54.981 [  9ac] TS: 339166 qpcNow: 3868402 qpc: 3868493
    25-09-2010 22:11:54.981 [  59c] TS: 339791 qpcNow: 3869007 qpc: 3869100
    25-09-2010 22:11:54.981 [  d58] TS: 340416 qpcNow: 3869649 qpc: 3869742
    25-09-2010 22:11:54.982 [ 1464] TS: 344375 qpcNow: 3873601 qpc: 3873730
    25-09-2010 22:11:54.982 [ 1464] TS: 345000 qpcNow: 3874273 qpc: 3874369
    25-09-2010 22:11:54.982 [ 1464] TS: 347916 qpcNow: 3877077 qpc: 3877187
    25-09-2010 22:11:54.982 [ 1464] TS: 348333 qpcNow: 3877600 qpc: 3877693
    25-09-2010 22:11:54.983 [ 1464] TS: 351458 qpcNow: 3880364 qpc: 3880635
    25-09-2010 22:11:54.983 [ 1464] TS: 352083 qpcNow: 3881237 qpc: 3881332
    25-09-2010 22:11:54.983 [ 1464] TS: 355208 qpcNow: 3884276 qpc: 3884386
    25-09-2010 22:11:54.984 [  ea8] TS: 369166 qpcNow: 3898233 qpc: 3898457
    25-09-2010 22:11:54.985 [  ea8] TS: 373333 qpcNow: 3902471 qpc: 3902614
    25-09-2010 22:11:54.985 [  ea8] TS: 374166 qpcNow: 3903242 qpc: 3903341
    25-09-2010 22:11:54.985 [  9ac] TS: 375208 qpcNow: 3904262 qpc: 3904367
    25-09-2010 22:11:54.986 [  9ac] TS: 380833 qpcNow: 3909881 qpc: 3910166
    25-09-2010 22:11:54.986 [  9ac] TS: 381875 qpcNow: 3911050 qpc: 3911152
    25-09-2010 22:11:54.986 [  59c] TS: 382708 qpcNow: 3911842 qpc: 3911944
    25-09-2010 22:11:54.987 [  9ac] TS: 391666 qpcNow: 3920744 qpc: 3921027
    25-09-2010 22:11:54.992 [  59c] TS: 441875 qpcNow: 3970778 qpc: 3971160
    25-09-2010 22:11:54.996 [  59c] TS: 488541 qpcNow: 4017502 qpc: 4017843
    25-09-2010 22:11:54.996 [  59c] TS: 490208 qpcNow: 4019351 qpc: 4019468
     

    davidf

    Retired Team Member
  • Premium Supporter
  • April 3, 2006
    796
    348
    Scotland
    Home Country
    Scotland Scotland
    Maths Help...

    I knew my Maths, Statistics and Computing degree would come in useful at some point.

    The biggest question I have is how accurate do you want the clock to be 1 ms, 0.1 ms or 0.01 ms, you can't get any more accurate than the last under windows (at least you can't guarantee to be more accurate than that).

    I assume that what you really want is to use the Audio clock as the real reference and adjust it using the system time to interpolate. The stream timestamp isn't really useful as you an't tell how much is buffered (50ms by default in your Audio Renderer) so there is an x amount of error in that where x<50 ms.

    so you want a clock like:

    Code:
    0
    664
    31026
    171524
    2031054
    2039466
    2146746
    2154340
    2184296
    3025584
    3026871
    3027293
    3036503
    3037409
    3043492
    3044147
    3047351
    3047840
    3051192
    3051680
    3052411
    3230828
    3640998
    3642158
    3642877
    3666846
    3671519
    3672177
    3675662
    3676198
    3679208
    3679768
    3685797
    3691184
    3701190
    3798574
    3800784
    3801403
    3803102
    3807596
    3808248
    3810444
    3827812
    3828931
    3829881
    3831824
    3834004
    3834759
    3866178
    3866884
    3867430
    3868229
    3868836
    3869478
    3873466
    3874105
    3876923
    3877429
    3880371
    3881068
    3884122
    3898193
    3902350
    3903077
    3904103
    3909902
    3910888
    3911680
    3920763
    3970896
    4017579
    4019204

    I assume.

    Ah I see what you mean now. It's possible for clock to go backwards (by a few nano seconds) when you interpolate if you manage to query at just the wrong time.

    If you use the audio clock as the correct clock then from your example:

    use the system clock when paused or -ve, when audio clock becomes active then add to the current time the difference in the clocks i.e. current reference clock time is 2 * system time - Audio clock time and use increments in the audio clock to maintain this time. To get an interpolated time add the difference between the last 2 system clocks then take it away when adding the audio clock increase again. I'll knock up a speadsheet and show you what I mean.

    tourettes

    if you give me a longer stream with a few pauses it'll make it easier to show you want I mean.

    Here's a simply piece of pseudo code of what I mean (I've assumed I'm right about the stream time)

    Code:
    SetTimestamps(LONGLONG systemTime,LONGLONG rendererTime, bool rendererInUse)
    {
    
    if (m_lastRendererInUse && rendererInUse) m_referenceClock+=rendererTime-m_lastrendererTime; //only use audio renderer time if in use throughout
    else m_referenceClock+=systemTime-m_lastsystemTime;
    m_lastRendererInUse=rendererInUse;
    m_lastrendererTime=rendererTime;
    m_lastsystemTime=systemTime;
    } 
    
    LONGLONG GetCurrentTime()
    {
    return m_referenceClock+GetCurrentSystemTime-m_lastsystemTime; //you could correct the systemtime to audio time here if desired and m_lastRendererInUse was true.  Drift doesn't matter if paused??
    }

    I find it unlikely it's this easy but at least you can point out the bit I haven't understood properly.
     

    red5goahead

    MP Donator
  • Premium Supporter
  • November 24, 2007
    695
    144
    Italy, North West
    Home Country
    Italy Italy
    Been testing this most of the week, and have to say i'm impressed, obviously have to keep WASAPIUseEventMode at 0, but video has been nice and smooth so far, and audio has fine, with no pops and crackles, and hasn't totally cut out on me like it did when using ReClock.
    I did have to correct minor sync issues with my receiver when using ffdshow as the audio decoder, but it looks like a ffdshow thing, as sync is fine when using MPC MPA Decoder.

    I tried MPC MPA Decoder but it's not very different from ffdshow audio decoding about A/V synch.
    I've an Htpc with a Passmark cpu mark of about 1350 point (PassMark - CPU Benchmarks - List of Benchmarked CPUs, 420^ position :( ), CoreAvc 2.0 as h.264 video decoding. I have to set -15 Ms in ffdshow audio output to get a perfect a/v synch. With h264 from ffdshow video decoder the value must be different . So to get a good A/V is very important know and consider the complete filter chain because CoreAvc should be faster than ffdshow to decode video . (-15 ms mean the audio must be anticipated probably due video is decoded faster using CoreAvc)
     

    Redemption80

    MP Donator
  • Premium Supporter
  • September 21, 2010
    16
    1
    Glasgow
    Home Country
    United Kingdom United Kingdom
    Been testing this most of the week, and have to say i'm impressed, obviously have to keep WASAPIUseEventMode at 0, but video has been nice and smooth so far, and audio has fine, with no pops and crackles, and hasn't totally cut out on me like it did when using ReClock.
    I did have to correct minor sync issues with my receiver when using ffdshow as the audio decoder, but it looks like a ffdshow thing, as sync is fine when using MPC MPA Decoder.

    I tried MPC MPA Decoder but it's not very different from ffdshow audio decoding about A/V synch.
    I've an Htpc with a Passmark cpu mark of about 1350 point (PassMark - CPU Benchmarks - List of Benchmarked CPUs, 420^ position :( ), CoreAvc 2.0 as h.264 video decoding. I have to set -15 Ms in ffdshow audio output to get a perfect a/v synch. With h264 from ffdshow video decoder the value must be different . So to get a good A/V is very important know and consider the complete filter chain because CoreAvc should be faster than ffdshow to decode video . (-15 ms mean the audio must be anticipated probably due video is decoded faster using CoreAvc)

    Yes, your correct, MPC MPA decoder doesn't differ that much, it seemed to just be better on one TV show for some reason.

    I hadn't thought about different decoding speeds depending on the h264 codec used, but it makes sense, have you tryed with DXVA variations such as PDVD10 or ffdshow DXVA?

    No idea what my passmark CPU score is, will check when i get home from work, but i imagine a QX9650 at 3.6ghz should have no issues even with software decoding.

    Are you 100% happy when you have sorted the sync, i've tryed adjusting sync with ffdshow, also MediaPortal itself, as well as using my actual receiver, and i can't seem to get it perfect, as it seems to be fine one moment, then drifts out a few minutes later, then sorts itself again.
     

    red5goahead

    MP Donator
  • Premium Supporter
  • November 24, 2007
    695
    144
    Italy, North West
    Home Country
    Italy Italy
    Been testing this most of the week, and have to say i'm impressed, obviously have to keep WASAPIUseEventMode at 0, but video has been nice and smooth so far, and audio has fine, with no pops and crackles, and hasn't totally cut out on me like it did when using ReClock.
    I did have to correct minor sync issues with my receiver when using ffdshow as the audio decoder, but it looks like a ffdshow thing, as sync is fine when using MPC MPA Decoder.

    I tried MPC MPA Decoder but it's not very different from ffdshow audio decoding about A/V synch.
    I've an Htpc with a Passmark cpu mark of about 1350 point (PassMark - CPU Benchmarks - List of Benchmarked CPUs, 420^ position :( ), CoreAvc 2.0 as h.264 video decoding. I have to set -15 Ms in ffdshow audio output to get a perfect a/v synch. With h264 from ffdshow video decoder the value must be different . So to get a good A/V is very important know and consider the complete filter chain because CoreAvc should be faster than ffdshow to decode video . (-15 ms mean the audio must be anticipated probably due video is decoded faster using CoreAvc)

    Yes, your correct, MPC MPA decoder doesn't differ that much, it seemed to just be better on one TV show for some reason.

    I hadn't thought about different decoding speeds depending on the h264 codec used, but it makes sense, have you tryed with DXVA variations such as PDVD10 or ffdshow DXVA?

    No idea what my passmark CPU score is, will check when i get home from work, but i imagine a QX9650 at 3.6ghz should have no issues even with software decoding.

    Are you 100% happy when you have sorted the sync, i've tryed adjusting sync with ffdshow, also MediaPortal itself, as well as using my actual receiver, and i can't seem to get it perfect, as it seems to be fine one moment, then drifts out a few minutes later, then sorts itself again.

    Your cpu overclocked from 3.0 to 3.6 have a 5000 score probably . So I think have no issue. about software decoding.
    Confirm the a/v synch it seems to be perfect one moment and then drift a little bit later but it'is acceptable at all . and this kind of issue is similar to Reclock audio renderer. Probably Tourettes choose event driver mode that we can't use due Xonar issue because with that approach there are less problems.
    note: CoreAvc support up to 16 core. Ffdshow I don't know.
     

    tourettes

    Retired Team Member
  • Premium Supporter
  • January 7, 2005
    17,301
    4,800
    I find it unlikely it's this easy but at least you can point out the bit I haven't understood properly.

    Thanks!

    I think your proposal doesn't take into account the drifting of audio and system clocks (only works around the discontinuity points). I have taken a bit different approach - calculating the clock drifting based on the timestamps when those are available.

    here's the current implementation:

    Code:
      DWORD dwTime = timeGetTime();
    
      UINT64 hwClock(0);
      UINT64 hwQpc(0);
      HRESULT hr = m_pAudioRenderer->AudioClock(hwClock, hwQpc);
    
      if (hr == S_OK)
      {
        if (m_dStartQpcHW == 0)
          m_dStartQpcHW = hwQpc;
    
        if (m_dStartTimeHW == 0)
          m_dStartTimeHW = hwClock;
    
        if (m_dPrevTimeHW > hwClock)
        {
          m_dStartTimeHW = m_dPrevTimeHW = hwClock;
          m_dStartQpcHW = m_dPrevQpcHW = hwQpc;
        }
        else
        {
          double clockDiff = hwClock - m_dStartTimeHW;
          double qpcDiff = hwQpc - m_dStartQpcHW;
    
          if (clockDiff > 0 && qpcDiff > 0)
            m_dSystemClockMultiplier =  clockDiff / qpcDiff;
    
          // TODO id this needed?
          if (m_dSystemClockMultiplier < 0.95 || m_dSystemClockMultiplier > 1.05)
            m_dSystemClockMultiplier = 1.0;
    
          m_dPrevTimeHW = hwClock;
          m_dPrevQpcHW = hwQpc;
        }
      }
      else
      {
        //Log("AudioClock() returned error (0x%08x)");
      }
    
      REFERENCE_TIME delta = REFERENCE_TIME(dwTime) - REFERENCE_TIME(m_dwPrevSystemTime);
      if(dwTime < m_dwPrevSystemTime)
      {
        delta +=  REFERENCE_TIME(UINT_MAX) + 1;
      }
    
      m_dwPrevSystemTime = dwTime;
      delta = (REFERENCE_TIME)(delta * (UNITS / MILLISECONDS) * m_dAdjustment * m_dBias);
    
      if (m_bHWBasedRefClock)
        delta *= m_dSystemClockMultiplier;
    
      //Log("mul: %.10f delta: %I64d - hwClock: %I64d hwQpc: %I64d qpc: %I64d clock diff: %I64d qpc diff: %I64d", m_dSystemClockMultiplier, delta, hwClock, hwQpc, qpcNow, hwClock - m_dStartTimeHW, hwQpc - m_dStartQpcHW);
    
      m_rtPrivateTime = m_rtPrivateTime + delta;
    
      return m_rtPrivateTime;
     

    tourettes

    Retired Team Member
  • Premium Supporter
  • January 7, 2005
    17,301
    4,800
    Probably Tourettes choose event driver mode that we can't use due Xonar issue because with that approach there are less problems.

    A/V sync is not affected the event / pollig based mode. Only the glitch resistance is better with the event driven mode since the buffer filling event requests are coming from drivers itself.
     

    tourettes

    Retired Team Member
  • Premium Supporter
  • January 7, 2005
    17,301
    4,800
    Been testing this most of the week, and have to say i'm impressed, obviously have to keep WASAPIUseEventMode at 0, but video has been nice and smooth so far, and audio has fine, with no pops and crackles, and hasn't totally cut out on me like it did when using ReClock.
    I did have to correct minor sync issues with my receiver when using ffdshow as the audio decoder, but it looks like a ffdshow thing, as sync is fine when using MPC MPA Decoder.

    I tried MPC MPA Decoder but it's not very different from ffdshow audio decoding about A/V synch.
    I've an Htpc with a Passmark cpu mark of about 1350 point (PassMark - CPU Benchmarks - List of Benchmarked CPUs, 420^ position :( ), CoreAvc 2.0 as h.264 video decoding. I have to set -15 Ms in ffdshow audio output to get a perfect a/v synch. With h264 from ffdshow video decoder the value must be different . So to get a good A/V is very important know and consider the complete filter chain because CoreAvc should be faster than ffdshow to decode video . (-15 ms mean the audio must be anticipated probably due video is decoded faster using CoreAvc)

    Decoder performance is not affecting the A/V sync. There are always few decoded video frames in the EVR side queue. If the decoder is too slow it will be shown as a dropped frame(s) but not as different A/V sync.
     

    Users who are viewing this thread

    Top Bottom