SQLiteDatabase Plugin for MP2 (3 Viewers)

[breese]

I wrote a little method which determines the amount of RAM and based on this returns the recommended default cache size. Currently

Does this take into consideration some of the Ram being assigned as a RamDrive?
I have 16G of ram, 6G is assigned to R: RamDrive
Just something to think about...

 

[breese]

Does anyone know what the process name for this SQLite is?
I am trying to locate it in my task manager Running Process's as a scan is happening
Thanks

 

[chefkoch]

It doesn't have any. It's just a dll that is used by MP2. So the process is MP2-Server.exe.

 

[breese]

Thanks.. I am monitoring that process.
Is there a way I can monitor / watch the wal and DB file as the scan is happening.
From what I see as the scan goes thru my collection, there are times it seems to pickup speed and the slow back down.

Also, where do I find what is actually being done durring a music scan?
Looking to see if it is more than just the file name and folder jpg

 

[morpheus_xx]

I think we should add some more (optional) logging into MediaLibrary importer in next release. This would help for diagnose.

If the database plugin is compiled as DEBUG, you can at least watch the SQLDebug.log file to see what is happening on DB.

 

[MJGraf]

I thouhgt about additional logging as well - in my MPAnywhere project and here in the SQLite project. For SQLite e.g. we can hook into the internal logging mechanism of SQLite to see exactly what's happening inside the database code. Same applies to Owin/Katana and ASP.Net Webservice. The only fear I have is that we end up with extremely verbose logging (my SQLDebug.log e.g. after an import is about 250MB). We can do this as we already do with SQLDebug.log - i.e. only log "IF #DEBUG", but then we really switch on every verbose logging we have for the whole project just by making a debug build. Maybe we should think about kind of a standard setting for various parts of MP2 such as "bool UseVerboseLogging" (separately for MediaLibrary, UPnP Server, etc. and every plugin) or something like that to be able to switch verbose logging on for the particular part of MP2 which we currently want to debug.
Just a thought...

 

[breese]

While watching the running process, I see a lot of file access and a number of open wal access
Is it possible there are to many threads going on there-by slowing down the scan's / DB access?
Directory access from the Client logs seem to very from less than a second to 13+ seconds.

I have added an excel spreadsheet that breaks down a few o fthe larger Artists with multiple albums and how long it took to scan thru the directories.

The Resource_01 and _02 files are screenshots of my Resource Monitor showing the large number of wal and other files open.
Hope this helps.

 

[MJGraf]

Thanks breese!
The pictures are really interesting - maybe I should start using more tools like the performance monitor to get a feeling what's going on (probably I'm still living in a Windows XP world with its simple task manager and haven't realized that Win7 has such nice things inbuilt...). Anyway, what we see there is write access to the WAL file and also to the database file itself. The latter one must be a checkpoint operation...

Is it possible there are to many threads going on there-by slowing down the scan's / DB access?

No - from my current version here locally I can see that there is actually only one thread working on the import. At some point I would even go the route to paralellize this with multiple threads as I think this could give us some further performance improvement. But this is not wihin the scope of the database plugin...

Directory access

This is by far not only directory access. What is happening there is that MP2 Server opens every music file in these directories and reads the included MP3 tags. So the important question would be: How many music files are in those directories for which it takes more than 10 seconds to import them? What tags are included in those files - only text or also embedded coverart?

And then I still don't understand one thing in your logs. You said that you have a single seat system (i.e. MP2 Server and MP2 Client on the same computer). Is your music collection on the same computer? Your logs show that MP2 Server scans for music on \\BUGSBUNNY\music\... If your music collection is on a local drive to your MP2 Server the it should read something like E:\music\... Can you still remember whether you chose the "Local File System Resource Provider" or the "Network Neighborhood Resource Provider" when you added the Share?

 

[breese]

start using more tools like the performance monitor

It has been a long time sense I did any of this type of testing. Eventually as i learn what are MP2's process, I will start creating logs from the Performance Monitor to get a better look at things...
If anyone else is doing that and can export the template, I could sure use it.

locally I can see that there is actually only one thread working on the import

I can tell you that MP2-Server has 42 threads and MP@-Client has 97 current threads on my server as I write this. Seems to be the standard numbers from the tests I have run.

single seat system

Currently durring these tests it is single seat. I verified the Standard install to C: and local shares worked. C: and Global shares worked. Currently using the Custom install to F:
btw: You were right and changing just the single path in Paths.xml was all I needed to install it correctly.
So, by installing this time with Global shares, I can (if and when) install MP2 Client on my other Client and see what happens when I tell them to work together.... Just trying to set myself up properly.

I will look at the albums listed and get the info together on # for songs and tags. Should be an interesting file....

 

[MJGraf]

Thanks and please do not forget my last question: Do you have your music collection on a local harddrive on the same computer as your MP2 Server is installed on?