FASTER music/movie browsing and DB access (1 Viewer)

htpcNoob

Portal Pro
January 23, 2005
52
0
Here's a trick that should speed up MP music/video browsing and database access by using a ramdrive. For Music/video browsing, MP checks to see if there is an image in its own THUMB directory for the item. if its not there it will check the directory where the item is located for a file called folder.jpg. If the images are in MP's thumbs directory, then you will notice a speed increase in browsing. In general, database access will be faster.

Tools:
-------
Ramdisk: Freeware. If you're not familiar with ramdrives they basically emulate a local hard drive using your systems memory and thus gives you a MUCH MUCH MUCH faster "drive". The only problem is that upon reboot, all data on the ramdrive is erased (ram is cleared). There are some Ramdisk programs that'll automatically restore/backup everything at startup/shutdown but I had probelms with them. We will restore and backup the ramdrive contents during system start and shutdown. (batch scripts below).
link: http://www.codeguru.com/Cpp/W-P/system/devicedriverdevelopment/article.php/c5789/

Sysinternal Junction: Freeware. This software creates a sort of symbolic link directory. If you are familiar with linux, then you know what I'm talking about.
link: http://www.sysinternals.com/Utilities/Junction.html

Considerations
-----------------
You can install MP entirely on a ramdisk. The problem is that MP has a large footprint because of the skins and such. Even deleting unused skins is not worth installing MP on a ramdrive yet. In my setup, I have stored the database and some of the thumbs directories on the ramdrive. Everything else is in the default MP install dir (C:\Program Files\MediaPortal).

Also, I wouldnt recommend putting the ENTIRE thumbs directory on the ramdrive. If you create thumbs for say "My Pictures" and you have a lot of pictures, then the ramdrive will fill up the ramdrive in no time. So its best to be selective about which thumb directories go in the ramdrive. For me I've put the Thumbs\Music\Albums and Thumbs\Videos\Title under the ramdrive.
The size of the ramdrive should be set to at least 32mb. This will support a lot of the thumbs that are downloaded by from IMDB and allmusic. The images that MP download, usually range from 1KB to 11KB with a couple of 30KB's as well. At 11KB each, you can store about 6000 thumbs on a 64MB ramdisk.

Currenlty MP doesnt allow you to designate where specific folders (such as database and thumbs) can reside. They are created in the MP install directory. This is where Sysinternals junction.exe comes in.

Here we go...

Steps:
-------
1) Install the ramdisk software and set it up. (You can set it up by going to your System Properties->Hardware->Device Manager and accessing properties of "Ram Disk" item after it is installed. Set it to the 64MB, specify the drive letter. I have mine set to Z. You may need to format the drive first by going into explorer, right-clicking the drive and choosing format.

2) Install junction.exe anywhere. I have mine installed under C:\Utils.

3) Create a folder that will copy the contents of the ramdrive to the hardrive during shutdown and restor them during startup. I have mine set to C:\RAMCACHE.

4) Its decision time! What would like to keep on the ramdrive? I put the entire database, Thumbs\Music\Albums, and Thumbs\Videos\Title on the ramdrive. So under C:\RAMCACHE, i created the following folders: database, Music-Albums, Videos-Title.

5) Go into the MP install directory (mine is C:\Program Files\MediaPortal) and look under the database folder. **MOVE ALL FILES and FOLDERS (if any)** from C:\Program Files\MediaPortal\database to C:\RAMCACHE\database. This will make the MediaPortal database directory empty. NOTE: Junctions can only be created on empty directories. If the folders have files, or other directories in them, then junctioning will fail. This is why we MOVE the contents.

6) Do the same thing with the Thumbs\Music\Albums. Go into C:\Program Files\MediaPortal\Thumbs\Music\Albums and MOVE EVERYTHING to C:\RAMCACHE\Music-Albums.

7) Again, go into C:\Program Files\MediaPortal\Thumbs\Videos\Title and move everything to C:\RAMCACHE\Videos-Title.

8) Next comes the script, startup.bat, which will be startup when system is booted up. This script will 1) copy the contents of C:\RAMCACHE to ramdrive Z:\ and 2) create the junctions from the mediaportal database, Thumbs\Music\Albums and Thumbs\Videos\Title directories. If you have set up the directories like me,
Code:
MP install directory = C:\Program Files\MediaPortal
ramcache directory = C:\RAMCACHE
ramdrive = Z:\
junction.exe = C:\UTILS\junction.exe
then you dont have to edit the startup.bat file. Otherwise, you need to make changes to it and make sure the directories are correct.

Code for startup.bat:
Code:
@echo off
mkdir Z:\Database
C:\utils\junction.exe "C:\Program Files\MediaPortal\database" Z:\Database

mkdir Z:\Music-Albums
C:\utils\junction.exe "C:\Program Files\MediaPortal\thumbs\music\albums" Z:\Music-Albums

mkdir Z:\Videos-Title
C:\utils\junction.exe "C:\Program Files\MediaPortal\thumbs\videos\title" Z:\Videos-Title

xcopy "C:\RAMCache\*" /e /c /q /h /r /y "Z:\"

9) Now comes the script shutdown.bat. Upon system shutdown, it will backup the ramdrive contents for Z:\ to C:\RAMCACHE. Again you may need to change the paths if they were different from mine.

Code for shutdown.bat:
Code:
@echo off
cd /d C:\
del /s /q /f "C:\RAMCache_BACKUP"
rmdir /q /s "C:\RAMCache_BACKUP"
mkdir "C:\RAMCache_TEMP"
xcopy "Z:\*" /e /c /q /h /r /y "C:\RAMCache_TEMP\"
rename "C:\RAMCache" "RAMCache_BACKUP"
rename "C:\RAMCache_TEMP" "RAMCache"

In this script you will notice that I create two OTHER directories C:\RAMCACHE_BACKUP and C:\RAMCACHE_TEMP. They are actually temporary directories used by the script to make sure copying is properly done. Call me paranoid if u want :-D :-D :-D but better safe than sorry I say.

10) After saving the two batch files, it is time to make sure that they are properly executed during startup and shutdown. Go to START->Run on the taskbar and type gpedit.msc. This will launch the group policy editor. Go to Computer Configuration->Windows Settings->Scripts (Startup/Shutdown).
On the right panel double-click Startup and add the startup.bat file to the list. I have mine as C:\startup.bat.

11) Now select Shutdown and add the shutdown.bat. I have mine as C:\shutdown.bat.

12) Double check to make sure the following directories are empty: C:\Program Files\MediaPortal\database, C:\Program Files\MediaPortal\Thumbs\Music\Albums, and C:\Program Files\MediaPortal\Thumbs\Videos\Title.
As I mentioned junctions can only be created on empty directories.

You are now set. You can test the scripts w/o rebooting. Just run the startup.bat file. If it was done properly, then everything in c:\ramcache should now exist the ramdrive z:\. Now run mediaportal and check your music/videoes see if see the albums/movie images. Check the tv to see if you have all the channels. This will confirm, that the database was properly moved and junctioned as well. If you do, then you're set and MP should run faster now.
when browing music and movies thumbs.

You may be thinking of putting the startup.bat in the startup folder of your programs or in the regisrty but that can lead to problems. If you do his and have mediaportal load at startup as well, then what could happen is mediaportal may start loading while the script is running and this could cause problems. The sections is group prolicy and scripts that are ran even earlier and guarantees that the copying takes place before users are logged on and other programs are started.

Word of caution: If your system BSOD's, hangs, or you use the reset button on the pc and windows is not properly shutdown, you will lose the thumbnails/database info that was added FOR THAT SESSION only. previous saved thumbnails and db changes will still be retained. Scripts can be written to call the shutdown.bat at specific intervals in the background or manually from "my programs" so that the ramdrive contents are saved often.

Happy htpc-ing and lemme know if you notice the speedup.
 

rtv

Retired Team Member
  • Premium Supporter
  • April 7, 2005
    3,622
    301
    Osnabruck
    Home Country
    Germany Germany
    wow! this is just what i had in mind - not for thumbs but tv-database. I realized that by adding ALOT of EPG information via xmltv the channel switch gets slower and slower :-/

    Maybe it'd be easy for you in your running setup to include this as well? I'd like to hear if it's worth the work :p
     

    htpcNoob

    Portal Pro
    January 23, 2005
    52
    0
    It should definately speed up EPG loading/browsing as the file will reside on the ramdrive. I can now browse all my albums quickly as scrolling thru them one page at a time takes < one second. before it took about 2 seconds to scroll one page at a time.
     

    htpcNoob

    Portal Pro
    January 23, 2005
    52
    0
    An improved caching system for images would definately help. But this trick has to do with loading the images from the disk. The bottleneck here was the time it took to load the image into memory. The rendering of the images didnt seem to slow it down much.

    I myself did other tweaks to the code for additional boost. All my mp3's are stored in folders as follows: <Genre>\<Artist - Album>\<Track> - <Title>.mp3. I took all my folder.jpg images and renamed to "<Artist - Album>.jpg" and put these album images on my ram drive. I changed the code in MP's GetCoverArt function so that it first looks in my ramdrive directory for "<Artist - Album>.jpg". If that is found, then just display it. The trick here was not to look for a folder.jpg on the directories corresponding to the mp3 file, but rather look for "<Artist - Album>.jpg" on one specific directory (this being a directory on my ramdrive).
     

    rtv

    Retired Team Member
  • Premium Supporter
  • April 7, 2005
    3,622
    301
    Osnabruck
    Home Country
    Germany Germany
    Wow, you're not intending CVS-updates with this pile of work :shock:
    But your changes would be interesting because I organize myself identically ;) Would be fine if MP would have an option to choose any pic in the folder if it isn't named folder.jpg ..
     

    htpcNoob

    Portal Pro
    January 23, 2005
    52
    0
    rtv said:
    Wow, you're not intending CVS-updates with this pile of work

    Actually. I CVS update MP on a daily basis. Its all scripted.
    1) Download CVS
    2) Use PowerGREP to automatically add my code to the proper places
    3) Compile the project
    4) Send it to my HTPC machine via windows sharing (samba)
    5) Run a script on HTPC machine using PSEXEC to close MP, update the files and restart MP.

    All with just one click :D :D :D
     

    rtv

    Retired Team Member
  • Premium Supporter
  • April 7, 2005
    3,622
    301
    Osnabruck
    Home Country
    Germany Germany
    You're a born linux user ;)

    Everybody complains tricky setup/fine-tune but it's also quite impressive everytime it works :lol:

    To stupid i've no other windows machine beneath my htpc.. I really don't like the idea to have an IDE with all sdks laying around on my htpc usable for everyone.
     

    SpaceProtector

    Portal Member
    November 30, 2004
    23
    0
    I now use the RAMdisk it worked fine just as explained above. I also put the tvguide.xml file in the RAM. Thx for this neat trick.
     

    rtv

    Retired Team Member
  • Premium Supporter
  • April 7, 2005
    3,622
    301
    Osnabruck
    Home Country
    Germany Germany
    It works great for me too!

    well, do you have the complete xmltv directory on ramdisk? Since I know from previous versions that there were issues with the file Tvguide.xml being stored in different places..
     

    Users who are viewing this thread

    Similar threads

    Come on - not interested in another challenge?! ;)
    Come on - not interested in another challenge?! ;)
    The backslash is in the temp path now and the new album thumbnail resolution setting seems to be working. Thank you! (y) Sorry to...
    Replies
    106
    Views
    4K
    MP1 MP2 MQTT Plugin DE
    @Jean-Marc i add play command, but i dont test it ... No time, sorry ...
    @Jean-Marc i add play command, but i dont test it ... No time, sorry ...
    https://ci.appveyor.com/api/projects/status/yj74bbhstsucwojj/branch/master?svg=true MQTT Plugin for MediaPortal - Publish some...
    Replies
    17
    Views
    2K
    We are proud to present MediaPortal 2.3.1 MediaPortal 2.3.1 is a full-blown media center software that addresses the most common user requirements out of the box. Aside from its support for Live TV and Radio, it offers a complete media server for music, movies, and TV shows. Info is automatically downloaded from various sources to...
    We are proud to present MediaPortal 2.3.1 MediaPortal 2.3.1 is a full-blown media center software that addresses the most common...
    We are proud to present MediaPortal 2.3.1 MediaPortal 2.3.1 is a full-blown media center software that addresses the most common...
    Replies
    0
    Views
    978
    Ok I could solve my problem by myself. The logs said that the delimiter of that song is not in text form. Although it was already in text form and I deleted the music database, the title information was still not available. I ripped the song again and now it works. I think that something went wrong the first time.
    Ok I could solve my problem by myself. The logs said that the delimiter of that song is not in text form. Although it was already...
    Hi, After the update from 1.25 to 1.26 the replay gain functionality appears to be broken. I had no sound on all audio files (TV...
    Replies
    12
    Views
    1K
    To anyone else who encounters the issue that the MediaPortal playback hangs and the I've found the rout cause for the "[ERROR] - BASS: GCHandle of Musicstream not found in Dictionary" is logged: On some of my files the Xing header was included in the audio frame count: Most players like Windows Media Player can handle this issue but...
    To anyone else who encounters the issue that the MediaPortal playback hangs and the I've found the rout cause for the "[ERROR] -...
    Hi, when playing music from a playlist sometimes suddenly the playback hangs in the middle of the track. When i press next, the...
    Replies
    58
    Views
    6K
    Top Bottom