Suggestion on fanart folder organization. (1 Viewer)

[JSchuricht]

Just a suggestion but perhaps there could be some better organization on the C:\ProgramData\Team MediaPortal\MP2-Server\FanArt folder.

The reason I bring this up is I just upgraded to 2.2.3 on two client only boxes and one server only box. The clients were quick, the server wasn't horrible but took long enough for me to look around and see it disk bound with a lot of I/O the the fanart directory with 3GB read and 6GB written during the upgrade. The fanart directory has 63,611 folders, performance tends to suffer with that high of number.

 

[henso]

Just a suggestion but perhaps there could be some better organization

Please elaborate.

performance tends to suffer with that high of number

What performance? Windows file explorer?

lot of I/O the the fanart directory with 3GB read and 6GB written during the upgrade

Logs please.

 

[morpheus_xx]

@henso from another log file I found some issue when accessing a non-existant fanart folder:

[2019-12-23 11:18:38,521] [618092 ] [AMQ 'FanArtLibraryManager'] [DEBUG] - FanArtManagement: Scheduling fanart cleanup.
[2019-12-23 11:18:38,528] [618099 ] [64       ] [WARN ] - FanArtCache: Error reading fanart directory 'C:\ProgramData\Team MediaPortal\MP2-Server\FanArt\'
System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\ProgramData\Team MediaPortal\MP2-Server\FanArt'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileSystemEnumerableIterator`1.CommonInit()
   at System.IO.FileSystemEnumerableIterator`1..ctor(String path, String originalUserPath, String searchPattern, SearchOption searchOption, SearchResultHandler`1 resultHandler, Boolean checkHost)
   at System.IO.DirectoryInfo.EnumerateDirectories()
   at MediaPortal.Common.FanArt.FanArtCache.GetAllFanArtIds()
[2019-12-23 11:18:38,533] [618104 ] [64       ] [DEBUG] - FanArtManagement: No orphaned fanart found.

First we should make sure the folder exists.

Also this method could be slow when there is such a huge number of folders (like the 63k in screenshot above)?

 

[henso]

I found some issue when accessing a non-existant fanart folder:

AFAIK this code will only be triggered when you delete a media item or a share. During import the folder is created I think, but it might be that if you don't copy to the cache and you don't have online fanart retrieval enabled, that the folder is not created. Or did you delete it manually?

First we should make sure the folder exists.

I added code to check for its existence.

Also this method could be slow when there is such a huge number of folders (like the 63k in screenshot above)?

My guess would be it should take around a few seconds unless fragmentation, anti-virus etc. is interfering. I added parallelization, so now it should be even faster. I guess the kind of activity seen should only happen if there is a lot of orphaned fanart (triggered by deleting a share?), a fanart import is resumed or if something is interfering with fanart folder access.

 

[JSchuricht]

@henso, I just mean in general it seems like an excessive amount of folders which contain over 160,000 files. I only have 87 series and 8,242 movies, antivirus has the MP2 folders excluded but it's still a lot of small files and folders for a system to deal with. Maybe it wouldn't make much difference cutting stuff down, I am just throwing an idea out there.