Is there any way to speed this import up? (1 Viewer)

[Lyfesaver74]

My import time is excessively long. As @ltfearme knows, my collection is massive, so I expect it to be slow compared to others. It is so long though that I am suspecting something is wrong.

I have attached my log and in it you can see the biggest bulk of time is after it parses the files and before it adds anything new. as seen between the time stamps:

2016-03-12 13:32:01.775 [INFO][07]: Finished Local Filename Parsing
2016-03-12 13:34:11.655 [INFO][07]: Adding 0 new file(s) to Database

That's 2 mins and 10 seconds between those two.

Is there anything I can do (other than mark ended series as "Ignore on scan" as I have done that) to make this quicker? Any advice would be greatly appreciated.

 

[ltfearme]

Maybe but probably not easily. Thats just how long it took your system to work out all the files on your system import paths and find what might be unknown (0 files in this case).

The only thing I can suggest is ensure that the import folders are isolated to only have files used by tvseries. This is probably something you already do but just in case there might be some optimisations. What we could do is add an option to completely ignore an import folder so you can potentially put all the ended shows there.

I would say hardware wise that a CPU / HDD will be the limitations, so faster the better, im sure @RoChess will have some suggestions on that

 

[Lyfesaver74]

Hey, D Yeah I do already separate my media by drives, not only by folders. As for your suggestion that would be a really cool feature!!! This is something I also thought would be good for MovPic and mentioned it to someone who will go nameless and was told it was a stupid idea. To have folders would hold media for library, but are full so no longer needed to search for importing.

Is this something you think you could add to MPTVS in the future?

 

[ltfearme]

Yes, it's something I could add in a future version.

 

[RoChess]

I already compared my import speed and even through my collection is only 5% smaller, it does the same scan to find 0 new files about 2-3x faster. Would trace logging reveal what is happening in that section of code, or do you have to add extra logging @ltfearme?

 

[RoChess]

Ohh the only thing I just realized is that I use a master subfolder usually along the lines of "TV Shows" even if the entire drive is used. If @Lyfesaver74 is sharing the X:\ root folder then those hidden Windows system folders might be causing a delay to ignore/skip the content for recycling, system volume, etc.

 

[Lyfesaver74]

@ltfearme is there any way to see what is going on during that 2 min 10 seconds?

 

[ltfearme]

Its doubtful extra logging can be added but will double check when I have time. There could be many reasons why Rochess has a faster scan e.g. Local vs Network Drive, Better CPU/ HDD etc.

Rochess does have a point regarding the root drive, it would be recommended to have a master sub-folder to filter out any hidden folders.

 

[RoChess]

@ltfearme, here are the comparison log results between @Lyfesaver74 database of 64MB and the one on my box of 35MB:

Mine:

2016-03-14 17:19:47.278 [INFO][03]: ***************  Gathering Local Information  ***************
2016-03-14 17:19:47.282 [INFO][03]: Starting Local Filename Parsing, processing 19324 files
2016-03-14 17:19:56.659 [INFO][03]: Finished Local Filename Parsing
2016-03-14 17:20:10.508 [INFO][03]: Adding 0 new file(s) to Database
2016-03-14 17:20:19.698 [DEBG][03]: Begin Parsing action: MediaInfo

His (only 9.7% more files):

2016-03-14 16:36:27.234 [INFO][03]: ***************  Gathering Local Information  ***************
2016-03-14 16:36:27.234 [INFO][03]: Starting Local Filename Parsing, processing 21198 files
2016-03-14 16:36:38.714 [INFO][03]: Finished Local Filename Parsing
2016-03-14 16:38:54.994 [INFO][03]: Adding 0 new file(s) to Database
2016-03-14 16:39:01.194 [DEBG][03]: Begin Parsing action: MediaInfo

The massive delay is not in the parsing (9s versus 11s), but in the determining if new files were added since last time. Taking roughly 14 seconds for me, but 2m16s on his box.

From what I understand from the code @ https://github.com/damienhaynes/mpt...ne Parsing Classes/OnlineParse.cs#L3043-L3059 that makes no sense to me

Adding some extra log lines might clarify if there is one specific item causing the delay, such as for example the RemoveFilesInDB() function, your guess is as good as mine.

PS: I had him do a vacuum and reindex on the database after cleaning up some shows no longer on disk, and it shrunk database to 56MB (did the same on mine and it shrunk to 31MB), but no idea why it is almost 100% more in size in comparison for only 10% additional episodes.

PPS: System spec wise we are pretty identical, mine is a crappy AMD A4-5300 dual-core APU with 8GB of memory, and an old 40GB Mushkin SSD, and pretty sure his was roughly the same (@Lyfesaver74 correct me if I am wrong).

 

[Lyfesaver74]

A6-5400K with 4GB RAM and a Samsung 840 Pro SSD