Upgraded my TV server now have live tv stuttering/pixelation (1 Viewer)

[jonathantwynham]

I had my SSD installed just for timeshifting only, no OS loaded etc. I figured this was the better way of doing it as the system is always on and never sleeps - perhaps not?

Ok, here are the results from the latency tests (I also ran whilst doing CPU/Memory/HDD testing):
As requested, I have unmapped BBC1 East Midlands (non HD) and BBC1 HD from all my cards then mapped it back to one card at a time and ran the test - hope thats what you were after.

Idle Latency: 90-112uS

BBC 1 East Midlands SD
Aver DVBT: 90-120uS
BG DVBT(1): 77-660 (Bounces up and down)
BG DVBT(2): 75-880uS (Bounces up and down - mainly at the 75-100uS but jumps to 800uS+)
BG DVBS(1): 440-450uS then 100-445uS (bounces about)
BG DVBS(2): 435-440uS then 100-435uS (bounces about)

BBC 1 HD
Aver DVBT: Non HD
BG DVBT(1): 255-922uS - No Video/Audio in preview but "playing"!?
BG DVBT(2): 200uS - No Video/Audio in preview but "playing"!?
BG DVBS(1): Starts at 440uS then 100-460uS (bounces about)
BG DVBS(2): Starts at 450uS then 100-445uS (bounces about)

Running Network Stress Test (10s max)
512KB Packets: 62567uS
1024KB Packets: 60120uS
5120kb Packets: 96208uS
10240kb Packets: 214448uS

Running HeavyLoad CPU Test
100% CPU Usage 103uS

Running HeavyLoad Disk Test (1000mb/s maximum speed)
50mb/s apx transfer: 110uS

Running HeavyLoad Memory Allocation Test
60mb/s apx: 109uS


Thanks.

 

[resa]

I was unable to find out, if you ever have changed the PSU itself on your server since the beginning of your problems? But since you've got some sort of warning from the PSU already i would remember to take special care about power stability in your system.

So if PSU was not changed or you have used two PSUs with the same wattage (unfortunately Dell isn't concerning much about users that upgrading/extending their systems heavily), try to attach another PSU to that system (if i.e. standard ATX PSU dosent fit's into the dell, try to positioning it beside the casing during the test) and look if errors remains with new/stronger PSU...

Maybe the 12V rail inside PSU which is supporting for installed drives etc. is working slightly outside the parameters already (i.e. due to aged capacitors) that is accepted by any of attched HDD or other consumers in your system connected to FDD/HDD/SATA-Power outlets or even taking power from the board only... that can i.e. causes an HDD to do short fail-safe spindowns or causes extension-cards to throttle down for a moment... those incidents can be too short for Windows beeing "informed" or noticed about and even too short for resulting into other problems like read/write errors etc., but maybe already long enough i.e. to halt the systems data-throughput from the slots towards other attached or on board components resulting into those discontinuities finally...

Also take care, that you have attached all extension-cards that own's an external FDD/HDD or GFX power input socket to the corresponding PSU outlets properly. Especially DVB-S cards needs extra power to serve the LNB, which can be obtained from PCI-Slots only in most cases too, but resulting into overload the PCI powerlanes in some cases i.e. where multiple cards are used. Take care that sometimes there are parts of an FDD power socket, made out of white plastics, is missed and you will only find four pins bended to the side on some cards, what is not to be identified as power socket such easily like if the white plastics can be seen in place...

Unfortunately it is one of my hard learned lessons, that only changing operating system and even only changing software or drivers can reveal upcoming power problems where the PSU is already sliding on the theresholds... macabre it sounds... but it can depend only on some new services that increases the load on idle and then pulls the load over thereshold on activity ... or there are changes in the handling for the cases where an device is not ready (i.e. dropping the data instead waiting for device ready etc.) ... In Addition... what can be inside accepted theresholds for the board or any other component (or is catched out by stability circuits some components, i.e. CPU power inputs, are equipped with) can interferer already on some other components... especially HDDs are much more sensitive, since they need enough of remaining spindle speed to perform pull back into parking position for the heads, so they often do precautionousely "micro" spin-downs on unstable power inputs but do not dismissing them selfes from the system that early, if power levels returns to normal shortly enough...

In adition you can try to change the engergy saving options within windows, not only those you find under energy in system settings, but also those you find in device-manager on tabs called energy or advanced for several devices too, i.e. for the NIC and sometimes for the slots and/or the cards in there too... maybe some devices are sended into engery saving mode but do not returning properly on activity...

Good luck anyway, i will trace this thread with interrest, to see what is the final fix to these issues

best regards.

 

[mm1352000]

Running Network Stress Test (10s max)

To be clear: when you say (for example) "512KB Packets: 62567uS", do you mean that the DPC latency is 62567 us?
If yes, that may be indicative of a driver problem.

 

[jonathantwynham]

Hey.

Yes, if sending a 512kb packet through the NIC to itself then the DPC latency is 62567us....

Im running the latest driver.. but it doesnt seem to help - thinking of putting in a PCI NIC to try and see if its a broadcom issue, but it worked fine with XP so i cant understand why it screwed up with 7....

 

[mm1352000]

Yes, if sending a 512kb packet through the NIC to itself then the DPC latency is 62567us....

Okay, thanks for the clarification.
Can you also confirm similar DPC results are seen when the network test involves another PC on your network?

Im running the latest driver.. but it doesnt seem to help - thinking of putting in a PCI NIC to try and see if its a broadcom issue

To be very clear: my understanding is that the test result doesn't necessarily indicate that there's a problem with the NIC driver. It's likely... but not a guarantee. In theory any component that uses DPC handling could be causing what you see.

...but it worked fine with XP so i cant understand why it screwed up with 7....

Disregard the fact that it worked with XP. W7 is a new environment. Lots of things changed under the hood between XP and 7. Any one of those changes may affect driver compatibility/performance such that it becomes "screwed up".


Moving forward...
From my perspective the important thing that the test has confirmed/showed is the cause of the problem when a network test is running. That cause is: "something" is starving the system of DPC handling time. Your tuner (and/or GPU) need regular/frequent DPC handling time allocations in order to function correctly. If/when they're not given that time, they're not going to be able to function correctly.

Normally the next step in the process of identifying the source of a DPC latency issue like this would be to use process of elimination to figure out which driver(s) (it could be more than one!) are causing the problem. You'd do that by removing or disabling all non-essential hardware (including your tuners!), then repeating the DPC latency test as you re-add/re-enable each device. When the DPC latency test goes red, you've found a cause of the problem. If the DPC latency for the "essential" configuration still exhibits the problem then unfortunately you may have to accept that your system isn't up to the task of real-time TV streaming.

One last thing: at this point I'm not sure if the network test is simulating/accentuating the problem that you have with HD channels when the network test is not running. It might be... but I don't think there's any way to know. It would be excellent if you could think of a way to confirm the linkage (eg. a large network file transfer while previewing TV???).

 

[resa]

As i already mentioned too, maybe because Win7 has much more functionallities regarding energy saving.

Go to device-manager and open properties of the nic, disable anything on the Energy Saving tab. Then Check Advanced tab, scroll the list if there any options that may is regarding to energy things. There can be i.e. functions that reduce energy depending on the cable length, where the mesurement can fail or similar settings.

Also try to deactivate jumbo frame related settings and fix the bandwidth to 1GB Full-Duplex instead Auto.

And if PSU was not changed, you may remember for it if you ran out of options

Good luck and best regards.

 

[jonathantwynham]

MM:

If I run the network test between the TV server machine and my laptop (which is wifi) for instance I am not seeing such an issue - however trying to send a 512kb packet to my laptop it slows the transfer speed down massively and i only get a 1600uS average latency.

If i use my file server for the network test then things spike over 16000uS.. these are sat next to each other right by my gbit switch with 1m patch cables..

so it seems if the network is slowed down then things can cope, if its doing something over the ethernet then it really hits it hard.

if i do a physical file transfer between tv server and fileserver (a 20gb folder) then the latency spikes to 3000uS.

i've done this whilst watching tv on an actual client (it does exactly the same as if previewing on the tv server) - and again if its on a BBC HD channel then the tv server skips a little, if its on an ITV HD channel it basically stops until the network settles down again.

Resa:

Thanks for your suggestions..
All power management is set to maximum performance.
The PSU as far as I know is fine.. I used to use this machine as a CAD and 3DSMax rendering workstation. It is physically powerful enough as it is a 750watt. I dont feel that it is a psu issue as I have now had this issue on two machines running windows 7 sp1 64bit and both of these machines were dell's with the broadcom netxtreme...

The NIC is a broadcom netxtreme gigabit onboard job - however in advanced settings for speed/duplex i can only force it to 10mb/100mb in full/half duplex - there is no option to force to 1gbit/full duplex for some reason - so i have left it as auto..




I appreciate both of you trying to help me - its so annoying that since trying to upgrade to the more modern versions of mepo and windows ive run into what is effectively a game-stopper!

I have a spare network card that I am going to install into the tv server later this evening and see if there is any difference..

Failing that I am going to jump to windows 10 and hope thats different!

 

[mm1352000]

If I run the network test between the TV server machine and my laptop (which is wifi) for instance I am not seeing such an issue...

Hmmm, I expected that might be the case. In some ways I feel like the network test you were previously running is a bit unusual - I guess like stress testing the NIC - and it may not be representative of "real world" situations.

...however trying to send a 512kb packet to my laptop it slows the transfer speed down massively...

You mean the network transfer speed?
How much of a slow down are we talking about here?

...and i only get a 1600uS average latency.

1600 is still quite high and would still give me cause for concern.

If i use my file server for the network test then things spike over 16000uS.. these are sat next to each other right by my gbit switch with 1m patch cables..

This is not unexpected. Proximity and connectivity (eg. Ethernet vs. powerline vs. Wifi) shouldn't make any difference. Rather, it is the connection bandwidth (eg. 1 Gb/s) that is the key. The more data the NIC has to handle, the more demand it will place on deferred procedure calls (DPCs). I would assume you see less of an impact (and slower transfer) with the laptop because its Wifi connection is slower (802.11n 150 or 300 Mb/s???).

so it seems if the network is slowed down then things can cope, if its doing something over the ethernet then it really hits it hard.

Yeah, as above - the key is the bandwidth/bitrate that the NIC has to handle.

if i do a physical file transfer between tv server and fileserver (a 20gb folder) then the latency spikes to 3000uS.

Here I assume the transfer bandwidth is limited (compared to the full 1 Gb/s theoretical bandwidth) by the write speed of the file server's storage subsystem. Lower transfer bandwidth means lower DPC demand, and that's why you "only" see 3000us latency instead of the 16000us you saw with the network test between the same machines.

i've done this whilst watching tv on an actual client (it does exactly the same as if previewing on the tv server) - and again if its on a BBC HD channel then the tv server skips a little, if its on an ITV HD channel it basically stops until the network settles down again.

Yep, that all makes sense in the context of the proposed explanation.

I appreciate both of you trying to help me - its so annoying that since trying to upgrade to the more modern versions of mepo and windows ive run into what is effectively a game-stopper!

I'm 99.999% certain you'd have run into the same problem even if you hadn't updated MediaPortal. This is a system level problem, and all applications including MediaPortal will be affected by it... though obviously the effect on MediaPortal appears more significant/severe due to the fact that real time streaming is time/latency-sensitive.

I have a spare network card that I am going to install into the tv server later this evening and see if there is any difference..

Please keep in mind what I said in my previous post. In particular: though it appears like the NIC driver is responsible for the problems, it is still possible that it's actually another driver or drivers. That's why I recommended the process of elimination.

Another idea: assuming you prove (or are already convinced) that the NIC driver is at fault, you can also try older versions of the driver. In some cases newer is not better.

Failing that I am going to jump to windows 10 and hope thats different!

Ummm... well, I can't say that I hold much hope of that. In most cases (GPU being one big exception) the guts of most W10-compatible drivers will be the same as those for W7 (and even Vista). If there are no fundamental changes in the driver then it's unlikely that the problem would go away. Simple logic really.

 

[jonathantwynham]

My next step will be to try disabling things and see what effect it has.

I am also wondering if forcing the NIC into 10 or 100mbit might prevent it from doing this.. it also has flow control disabled and again im wondering if that might make a difference.

The NIC im going to fit in is a full server class card (intel pro/1000 pt) which according to the literature that came with it has 'interupt moderation' which apparently reduces cpu usage and can spread its usage across multiple cpus.

I cant downgrade the current broadcom drivers as they dont seem to have any older ones on their website and im wary of drivers off 3rd party websites.. I have however just noticed on the dell website there are a set of specific drivers just for windows 7 64bit.. will give them a go and see what it does.

I have just installed latencymon and according to it the highest user of the DPC interupt is the NDIS driver followed by my BGBTCap driver (which I guess is for the tuners) - im presuming I should see that fairly high on the list as it is doing stuff when tuning?

I wont bother trying to upgrade to Windows 10 then either!

Dont get me wrong, im not blaming MePo for this issue.. im squarely blaming it on windows 7 - there seem to be a lot of people who have latency issues with windows 7 and the vast majority seem to be running 64bit!

As a last resort if I cant fix this issue then im going to go back to XP and will run a previous MePo1.5 on it like I did before!

 

[mm1352000]

I cant downgrade the current broadcom drivers as they dont seem to have any older ones on their website and im wary of drivers off 3rd party websites.. I have however just noticed on the dell website there are a set of specific drivers just for windows 7 64bit.. will give them a go and see what it does.

Ahhh, that's definitely worth a try. Most of the time it's okay to get drivers directly from the chip vendor (eg. Broadcom), however sometimes - due to implementation, customisations etc. - it's necessary to get drivers from the system vendor (eg. Dell).

I have just installed latencymon and according to it the highest user of the DPC interupt is the NDIS driver followed by my BGBTCap driver (which I guess is for the tuners) - im presuming I should see that fairly high on the list as it is doing stuff when tuning?

Yep, NDIS is related to the NIC... so I guess that's another indication that the network driver is likely to be the issue.
...and yes, BGTCap.sys is the driver for the Blackgold card. I'd only expect it to use significant numbers of DPCs when it is tuned. When idle - not time-shifting, recording or EPG grabbing - it shouldn't use many DPCs.

As a last resort if I cant fix this issue then im going to go back to XP and will run a previous MePo1.5 on it like I did before!

XP is certainly the "tried and true" option for me. Still running it here.