Potential Memory Leak when adding a large collection of movies (1 Viewer)

[tourettes]

The spike is very obvious and it's the only application I know of that causes such behavior.

When ever the memory hogging happens on a separate PC where the process itself is running the issue is definitely not caused by the importing code. It is something on the server side that has bug.

Are you using some software RAID or anything similar?

 

[jasonwc]

I'm using FlexRAID on the file server. However, I observed the same behavior on the HTPC when the movies were stored locally with no RAID of any kind in use.

 

[fforde]

What you saw locally may have been attributable to the memory issues that was fixed in the latest version. It's probably this was an unrelated problem that looked similar.

 

[jasonwc]

I restarted the file server and I'm running an import with mediainfo scanning DISABLED. See if that helps.

EDIT: I'm seeing the same behavior with MediaInfo scanning disabled. Scan is still running but I'll get you the logs when it completes.

Would it be helpful to install Mediaportal on the WHS server and run the import with the movies stored locally?

 

[fforde]

Sorry jasonwc, I think tourettes is right, this sounds like a problem with your file server. With MediaInfo disabled all Moving Pictures is doing is browsing the file system. If this causes memory usage on the file server to spike then there is a problem with the file server.

 

[jasonwc]

Any idea what the issue might be? I've been able to replicate the issue on every version of x64 Windows I've tried with a diverse set of hardware - different Intel socket, different processor, motherboard, storage controller etc. The only consistent variable is that the issue occurs with Windows x64 but not the x86 version. Perhaps it's an issue with the OS itself.

Perhaps poolmon can show what's using the memory.

 

[fforde]

Are you using flexraid in all scenarios as well? There has to be some commonality.

 

[jasonwc]

No, I only use it on my file server. I had this issue before I used FlexRAID. However, it should be easy to test if FlexRAID is the problem. I can just add each directory that contains movies directly rather than accessing it through the FlexRAID storage pool.

EDIT: Same behavior when I point directly to the individual folders (not using the FlexRAID storage pool). It also appears that memory usage is tied to the number of movies added. When I added a folder with 50 movies, it only used about 1.5 GB of kernel paged memory as opposed to the 6 GB it used with my entire collection.

 

[RoChess]

No, I only use it on my file server. I had this issue before I used FlexRAID. However, it should be easy to test if FlexRAID is the problem. I can just add each directory that contains movies directly rather than accessing it through the FlexRAID storage pool.

Do you have any monitor software running in the background on the client or server? Microsoft Security Essential or other Anti-Virus or any monitor app in general that could compound the issue?

FlexRAID has some memory issues when a lot of data is copied, such as: FlexRAID Heavy RAM Usage

This is why I suggested to look into MediaInfo, because it has to 'read' (which means copy over network) the header file. However if this triggers your monitor software it might result in much more data being copied then just the small header. Also come older versions of MediaInfo had a bug that could result in entire mediafile being copied over LAN to 'read' the small header.

SysInternals indeed offers many tools that allow you to debug this issue to the source, but going through some basic steps of debugging via elimination should also work.

 

[jasonwc]

I don't have any Antivirus software installed on the file server because it doesn't execute any 3rd party software other than Hamachi that provides VPN access. Is Windows Security Essentials downloaded by Windows Update automatically? I have recommended updates including security patches installed automatically.

Basically, it's a standard Windows Home Server 2011 (based upon Windows Server 2008 R2) install with FlexRAID for Snapshot RAID + Storage pooling. However, I don't think FlexRAID is the cause since I get the same behavior even when I access movies from their individual drives, bypassing the storage pool. I could disable the FlexRAID service to be certain.

I should also say that I routinely transfer tens or hundreds of GBs of files over the network, download via uTorrent to the file server, unrar data to the file server, stream movies/tv using Mediaportal, and run RDP sessions tunneled through the Remote Desktop Gateway on the server. None of these activities results in excessive memory usage.

I can move one or two of the drives housing movies to my desktop tomorrow. That way I can test importing movies from a local disk on a different system with completely different hardware.