MP2 / SQLite Performance Testing (1 Viewer)

breese

Retired Team Member
  • Premium Supporter
  • July 11, 2011
    3,902
    770
    65
    Arlington Heights, Illinois
    Home Country
    United States of America United States of America
    I have been doing some system analysis testing to find out the impact of the server and then server / client and server /client / media scan.
    Background: Windows 7 Ultimate. 16G of Ram with a 6G Preconfigured RamDrive. All drivers for hardware updated. All windows service packs installed. Verified C: (SSD) speed and f: (Raid5 SATA) speed.
    Raid5 is a 6MB SATA Bus. Burst speed 3190.3 MB/s. Average read 215.9 MB/s. Sequential Read Average 240 MB/s
    I did a fresh install of MP2 and added the v7 of SQLite drivers. Started everything, configured, and did a scan of my media (music). The music collection is 119,000+ files at 470G.
    Watched the Resource Monitor for memory and disk activity. Verified access to the Media on F:, temp files to R:\temp (RamDrive), and other activity.
    Stopped the client and server. Killed off all background processes.
    While monitoring Resource Manager and creating a Performance Monitor configuration set to run for 2 minute intervals.
    The included spreadsheet (MP2 System Usage) has the results of testing and separated tests with Yellow Highlights.
    First test was of the Server Process being started.
    Second test was of the Client being started
    Third test was a Rescan of my media (seems I might have also found a bug with this test.. More later)
    As you look at each of the tests, you can see what the impact is with each of the Process’s, Handles, and Threads. It is interesting to note the Commit KB (how much is being ask for vers Working (actual) KB). It is also interesting that a Rescan of media makes the Client Handles jump from a Minimum 384 to 1,223.

    Rescan bug?:
    Each and every time I did scan media test I could see the files being accessed in Resource Monitor. Seems the scan would touch / request information on hundreds of files. It also opens numerous temp files (So does Server but not so much). To this point I am including Server Startup Temp.jpg and Client Startup Temp.jpg. Within these you can see the Temp files being written to my RamDrive and the directory. The Server places them in R:\Windows and the Client in R:\Users.
    I am also including jpg’s Resource_01 and 02. It help to understand the files being accessed during a first time media scan. As you can see, there are a lot.
    As for the Bug. While the Rescan was going on for Test #3, the Resource Manager never showed any access to the media files on F:\music. Yet a look at the Server.log file showed all the directories being accessed. What is real interesting here is the number of resources the server log is reporting during this scan.
    During a first time scan I would see 44 or so resources pending. During the Rescan those number jumped up over 100 resources pending yet, there still was no activity happening on my R: drive.
    Because I ran this Rescan test 3 times (no reboot of anything) and got the same results, I have to question what the code is actually doing.
    I have been following and responding to JGraf’s SQLiteDatabase Plugin for MP2 and understand where the question of Database import and access read times are different.
    Is it possible that the Rescan code is actually doing a compare against what is already in the DB File?
    This might explain the Auto-Scan at 02:00:08 and stopped at 02:36:06.
    I did a rescan starting at 05:28:45 and it ended at 05:46:32 and never did complete the entire scan of media. At the same time there was no F: access seen in the Resource Monitor.
    Now if I can figure out how to get the resource manager or performance monitor to breakout the SQL dll’s and what they are doing, I might get a better handle on things.

    Edit:
    I turned on Procmon (analize process) and filtered it to watch only MP2-Server.exe
    Located the begining (or I beleive the begining) of a Rescan analyis
    Back to Broadway is a cutout from the samed file... Might help to see what all is happening.
     

    Attachments

    • Client Startup Temp.jpg
      Client Startup Temp.jpg
      268.8 KB
    • Server Startup Temp.jpg
      Server Startup Temp.jpg
      275.6 KB
    • Many Wal.jpg
      Many Wal.jpg
      543.4 KB
    • Resource_01.jpg
      Resource_01.jpg
      443.6 KB
    • Resource_02.jpg
      Resource_02.jpg
      441.6 KB
    • MP2 System Usage.zip
      6.8 KB
    • Back to Broadway.txt
      74.8 KB
    Last edited:

    Users who are viewing this thread

    Top Bottom