Aibility to play Media without adding them to DB (1 Viewer)

[Lehmden]

Hi.

what makes your system slow when you import a lot of data in your database.

I'm pretty sure it's the handling of the huge DB file itself. I can hear the HDD working a long time when accessing DB. A SSD could (should) be speed up things a lot. As I'm unemployed since a long time (health issues) I did not have a lot of money. SSD will come but this will take a while. IMHO MP2 should be usable on slower systems (like mine) also. So the attempt to grab only needed data will help a lot, I bet.
MP2 really is lightning fast with little amount of media items. But it feels significant slower with some 20.000 items I have in my "working DB". (15.000 Series Episodes, 1.500 Movies and some test collection of Audio and Pictures). When try to add all audio (150.000 Songs) and especially pictures (aprox 500.000) then MP2 is getting slower and slower, nearly unusable.

BTW, I really did not need my pictures in DB. I'm not using the category system as I've started sorting them by filesystem long before I've used some HTPC software for the first time. I can find anything I want in no time due to my storing and naming system... As for thumbs, nearly all folders are containing a thumbs.db file generated by windows itself. I don't know if they can be used inside MP2 like in explorer...

 

[MJGraf]

Can you tell me how big your database file is (1) for your "working DB", (2) when adding your 150k songs and (3) when trying to add your pictures? Rough values are ok, I'm just trying to get a feeling on what amounts of data we are talking about. When I import my whole music collection, my SQLite db file is about 1.7GB in size and this is still very fast...
Thanks, Michael

 

[Lehmden]

Hi.
My "Working DB" is about 500 MB. With all my music it's a bit over 2 GB and with 20% of my pictures it's more than 3 GB. So I suspect a size of about 6-10 GB if I have "all in". Seems to be logical as my MP1 thumbs dir is around 11 GB in total....

With the 500 MB the connection to server (on MP2 startup until Series and Movies are shown on Home) is about 1 second. Entering the series section (300 series in total) takes about 20 seconds the first time after a refresh of Share (then all covers are slowly shown one after another and the server is heavily working on HDD) and 5 seconds normally (then all covers are there at once). Entering a Series with lots of Episodes (Doctor Who, Navy CIS, Bones, Smallville,...) takes about 5-10 seconds. This could be reduced a lot if finally we have a season view as default entering a series...

With all my music in DB the startup connection takes about 10 seconds and entering the series is about 30 seconds instead of 5. With the 3 GB DB file this times did increase a lot

When I started testing MP2 with only a few sample media items non of those times could be measured at all. All was there instantly. So this is not a core MP2 issue. It's due the amount of media items to handle by DB...

 

[MJGraf]

Thanks a lot Lehmden, that's very interesting!

First of all, a DB size of 10GB must IMO not be a problem for MP2. In MS Outlook 2003 and 2007 e.g. the standard size limit for pst files was 20GB since 2010 it's 50GB. And Outlook is perfectly fast with a 20GB pst file - tested this myself. This has to be the goal for MP2.

As to your observations: Do you still have the 2Gb or better the 3Gb version of your DB?. If so, could you please

First: restart your MP2 Server with your working DB, do exactly what you described above and then post your SQLDebug.log here.
Second: restart your MP2 Server with your 2GB or 3GB version of the database and do exactly the same as before. Then post the SQLDebug.log generated in this test.

This would be very helpful in improving the speed, as in this log we can see exactly every single SQL command as well as the exact time it took the database to execute this query.

Thanks for all your help!
Michael

 

[Lehmden]

Hi.
I only have the 500 MB DB left.
I can do those tests with that. To do more (adding music and /or pictures) I would like to wait some days. I'm expecting to get DSL finally here. Should be this or next week so then I can grab data from online a lot better. It's ordered and I'm waiting for the connection and the FritzBox.

 

[MJGraf]

Ok, then let's wait with all the tests until you have DSL. Crossing my fingers that all works well...

A bit of background to explain what I need:

First I need a test with your working DB having all your series (and maybe some additional media) imported. Something like
Starting MP2 client takes x seconds,
Entering series takes y seconds
Entering "bones" takes z seconds
And then I need the SQLDebug.log to find out how much of the time for the three steps is actually caused by database queries.

The second test should be identical, just with another database. This database should have exactly the same amount of series, but A LOT of additional media such as your 150k music files imported. Then I can compare the results with the first test. Theoretically, if the amount of series in the second DB is identical and the difference is only a lot more music files, there should only be a minimal difference in DB performance when accessing the series. If the difference is big, there is something wrong which we should improve...

Michael