How best to keep Moving Pictures databases in sync across multiple clients? (1 Viewer)

mpsam

Portal Pro
June 16, 2009
179
75
Just a thought. In lieu of multiseat support, would it be possible to allow the Moving Pictures config program to connect different databases.

The current lack of multiseat support leads to, I think, to two main problems. One is duplication of data and the second is duplication of effort.

For now I can live with the duplication of data, and to be honest, a lot of the duplication of effort is reduced by 'proper naming' of files and the auto scanning, importing and downloading of metadata. However there are some tasks that are mouse and keyboard tasks (config) and others that are remote (GUI). With the addition of categories, there are now more mouse and keyboard tasks. It would be nice to use the config program from my office machine, where I rip the discs, to add movies to categories on the HTPC which usually have just remotes connected to them. Being able to connect to multi databases would make it much easier and thus more frequently done.

Is this possible?
 

Surferosa

Portal Pro
September 2, 2009
55
5
Home Country
England England
It would be nice to use the config program from my office machine, where I rip the discs, to add movies to categories on the HTPC which usually have just remotes connected to them. Being able to connect to multi databases would make it much easier and thus more frequently done.

Is this possible?

You've sort of described my setup there mate with one exception- centralised storage. To explain, I have 4 MP clients:

Office Machine (used for config, ripping, etc- dont tend to watch too much from this though. This is the database creator)
Media Centre (main rig for movies under TV)
Kids PC 1
Kids PC 2

I have a NAS that holds all the relevant file (eg movies, music, TV, etc), art, thumbs and database information. I only have 1 version of the database- Moving Pictures 1.0 beta filters & categories has finally let me set a filter against stuff I dont want the kids watching (restricted content filter). So now, the only problem you have is that 4 clients running MP at once is likely to pretty quickly corrupt the database. The solution is centralised storage with redistribution via a batch file. So, all the clients are now setup according to this wiki.

The difference (and I think this tip should maybe be added to the wiki) is that the database files (ie the db3 files) are acessed via a local copy- rather than directly from my NAS. So, instead of running mediaportal, they run a batch file that contains the following code;
Code:
xcopy \\[mynas]\MediaPortal\Main\Database\*.db3 C:\ProgramData\Team MediaPortal\MediaPortal\database /D /C /Y
start "MediaPortal" "C:\Program Files (x86)\Team MediaPortal\MediaPortal\MediaPortal.exe"

So this copies all the database files for MediaPortal onto the client PC's. So if I have added a movie or TV program from my office PC, next time they run MP from another client PC this copies the changed file to their client PC and it then appears in the MP once its run.

The downside of this approach is two-fold. Firstly, if they have added any local media (a disc or something) then they lose that in the database update. This isnt an issue for me as they dont. Secondly, 'watched' flags are effectively lost from the client PC's. I get round this on the main rig (Media Centre) by keeping the database files pointing at the network share. The trick here is to make sure this isnt running when I change add stuff to the database (isnt an issue for me- its in the room next door so I check). That way, I retain all the watched flags from my media center rig (though unfortunately the kids pc's watched flags are still lost).

As for your issue with remotes and alike- I am running 2 config versions as well. The office machine and main rig run on Streamed MP (both 16x9 resolutions- one I use with a remote, one without). The two kids pc's dont have a remote, and are now running Blue 4x3. I dont have any issues with remotes / aspect ratios- the trick I guess is just to keep 2 copies of the config files- but setting them up on each pc is a one-time job so Im not sure how much of an issue it could be?

Edit: I should add, the other key to this is making sure that scanning for new media in the GUI is always disabled in the config(s). The only place I add new media is from mediaportal config- which I only ever run from my office rig (as it sounds like you do).
 

Bob The Great

Portal Member
October 16, 2009
11
1
Home Country
United States of America United States of America
That's an interesting approach. Rather than push the db out whenever it's updated, you have MP pull it on launch. I like that better - that way there's no possibility of over-writing a db while MP is open on one of the clients. The only downside is some un-necessary copying if the db has not changed. This would be easy enough to get around by running a .bat to compare hashes of the local and server-side db to detect changes and determine whether a copy is needed.

I think I might adopt this to sync the Moving Pictures db, and let the others remain independant. The rest of the MP db's don't change frequently, and I want to preserve the ability to have custom configs on each client.
 

ryan20021982

Retired Team Member
  • Premium Supporter
  • June 27, 2008
    655
    86
    USA
    Home Country
    United States of America United States of America
    I'm thinking about setting up a central database myself and thought of this idea

    My mediacenters dont use the internet for anything just local to access the server, so

    my server has dual nic's and I am thinking about just connecting the extra nic on the server to
    a gbit switch and connecting the mediacenters to that so they have local access and also so they
    cant update the database if they wanted to. Then point them to the server for the db file, where it
    will be updated from.

    Has anyone else done this exact setup?
     

    igu

    Portal Member
    January 21, 2008
    17
    0
    Home Country
    Canada Canada
    It would be nice to use the config program from my office machine, where I rip the discs, to add movies to categories on the HTPC which usually have just remotes connected to them. Being able to connect to multi databases would make it much easier and thus more frequently done.

    Is this possible?

    You've sort of described my setup there mate with one exception- centralised storage. To explain, I have 4 MP clients:

    Office Machine (used for config, ripping, etc- dont tend to watch too much from this though. This is the database creator)
    Media Centre (main rig for movies under TV)
    Kids PC 1
    Kids PC 2

    I have a NAS that holds all the relevant file (eg movies, music, TV, etc), art, thumbs and database information. I only have 1 version of the database- Moving Pictures 1.0 beta filters & categories has finally let me set a filter against stuff I dont want the kids watching (restricted content filter). So now, the only problem you have is that 4 clients running MP at once is likely to pretty quickly corrupt the database. The solution is centralised storage with redistribution via a batch file. So, all the clients are now setup according to this wiki.

    The difference (and I think this tip should maybe be added to the wiki) is that the database files (ie the db3 files) are acessed via a local copy- rather than directly from my NAS. So, instead of running mediaportal, they run a batch file that contains the following code;
    Code:
    xcopy \\[mynas]\MediaPortal\Main\Database\*.db3 C:\ProgramData\Team MediaPortal\MediaPortal\database /D /C /Y
    start "MediaPortal" "C:\Program Files (x86)\Team MediaPortal\MediaPortal\MediaPortal.exe"

    So this copies all the database files for MediaPortal onto the client PC's. So if I have added a movie or TV program from my office PC, next time they run MP from another client PC this copies the changed file to their client PC and it then appears in the MP once its run.

    The downside of this approach is two-fold. Firstly, if they have added any local media (a disc or something) then they lose that in the database update. This isnt an issue for me as they dont. Secondly, 'watched' flags are effectively lost from the client PC's. I get round this on the main rig (Media Centre) by keeping the database files pointing at the network share. The trick here is to make sure this isnt running when I change add stuff to the database (isnt an issue for me- its in the room next door so I check). That way, I retain all the watched flags from my media center rig (though unfortunately the kids pc's watched flags are still lost).

    As for your issue with remotes and alike- I am running 2 config versions as well. The office machine and main rig run on Streamed MP (both 16x9 resolutions- one I use with a remote, one without). The two kids pc's dont have a remote, and are now running Blue 4x3. I dont have any issues with remotes / aspect ratios- the trick I guess is just to keep 2 copies of the config files- but setting them up on each pc is a one-time job so Im not sure how much of an issue it could be?

    Edit: I should add, the other key to this is making sure that scanning for new media in the GUI is always disabled in the config(s). The only place I add new media is from mediaportal config- which I only ever run from my office rig (as it sounds like you do).



    I like your idea about "pulling" the moving pictures DB from the MP clients when they start up but I put mine in sleep mode and MP starts up each time, is there somewhere in MP where you can define a batch file to be run when it starts since I don't completely shutdown my Clients each time?

    :D

    I think I've aswered my own question. I think the solution is to use an "Eventtrigger" . What this will do is that when the trigger sees a certain Event ID in the event log it will run the specified batch file automatically after being woken up from an S3 state. Hopefully this works!

    Thanks for the idea!
     

    Surferosa

    Portal Pro
    September 2, 2009
    55
    5
    Home Country
    England England
    That's an interesting approach. Rather than push the db out whenever it's updated, you have MP pull it on launch. I like that better - that way there's no possibility of over-writing a db while MP is open on one of the clients. The only downside is some un-necessary copying if the db has not changed. This would be easy enough to get around by running a .bat to compare hashes of the local and server-side db to detect changes and determine whether a copy is needed.

    I think I might adopt this to sync the Moving Pictures db, and let the others remain independant. The rest of the MP db's don't change frequently, and I want to preserve the ability to have custom configs on each client.

    Well, that batch file above only copies the files if they have changed anyway (the /D limiter in xcopy checks and only copies files across if the date attribute is different in the files). To be honest, copying all of the files across takes a couple of seconds anyhow- I have 3000-odd video files yet the db3 files are still small enough not to be a big issue.

    Also- configs are completely independent in each client. When I first set them up, I copy the mediaportal.xml across so that most of the settings are the same. Once its localised though, I can change the skin, remote and all other settings independentely. Its just the database files that are kept in sync. The only issue I had before was not being able to restrict content to the kids pc's. The marvelous job the guys have done on MovPic 1.0 means I now have as close to a multi-seat install as I need.

    The other issue you may have is Im not sure how you would specify different a different database directory for (say) Moving Pictures & TV Series. mediaportaldirs.xml specifies the database directory which (as far as I know) is used by all the plugins? I could be wrong though...

    ryan20021982 said:
    my server has dual nic's and I am thinking about just connecting the extra nic on the server to
    a gbit switch and connecting the mediacenters to that so they have local access and also so they
    cant update the database if they wanted to. Then point them to the server for the db file, where it
    will be updated from.

    Unfortunately the gui changes some of the information inside the db3 file (for example watched tags)- and its these that will cause the corrupt database issue (even more likely if you try updating the database whilst the client is running elsewhere).

    igu said:
    I think I've aswered my own question. I think the solution is to use an "Eventtrigger" . What this will do is that when the trigger sees a certain Event ID in the event log it will run the specified batch file automatically after being woken up from an S3 state. Hopefully this works!

    Well, I did think about different ways of configuring the batch file- for me, I mapped a key to the remote that starts MediaPortal in the media rig. However, bacuse I didnt want to lose the watched tags from here, I decided to keep these pointing at the server (Im just careful to make sure its not on when Im updating- no corrupt issues so far and Ive been using it like this for a while). On the pc's, they get launched from a shortcut by the kids- which is obviously also pointing to the batch file.

    Running the batch file from an event trigger sounds cool though.. post it up here when your done if you dont mind. Could come in handy :)

    Edit: Im currently in the process of making a batch file that will run the config on my office PC. The idea here is I want to make a series of incremental backups of the db3 files so that *if* the database does die for any reason, I have a series of date stamped backups to fall back on. Belt & braces really, but why not...

    Edit: Just checked- 335 MP Movies- db3 file size 2.6MB; 2400 TV Series Episodes- db3 file size 6.7MB; 25000 Music files- db3 file size 9.6MB. So in total, even if all the files change every time, Im only pulling 20MB. Which is why its almost instant :)
     

    Bob The Great

    Portal Member
    October 16, 2009
    11
    1
    Home Country
    United States of America United States of America
    The other issue you may have is Im not sure how you would specify different a different database directory for (say) Moving Pictures & TV Series. mediaportaldirs.xml specifies the database directory which (as far as I know) is used by all the plugins? I could be wrong though...

    You wouldn't need to specify a different directory, I wouldn't think. Just call out movingpictures.db3 explicitly in your xcopy command. And the same for TVSeriesDatabase4.db3, if those are the only two that need to be copied.

    Code:
    xcopy \\[mynas]\MediaPortal\Main\Database\[COLOR="Red"]movingpictures.db3[/COLOR] C:\ProgramData\Team MediaPortal\MediaPortal\database /D /C /Y
    start "MediaPortal" "C:\Program Files (x86)\Team MediaPortal\MediaPortal\MediaPortal.exe"


    Edit: I forgot about the music db. But still, that's only 3.
     

    Surferosa

    Portal Pro
    September 2, 2009
    55
    5
    Home Country
    England England
    As an aside- Ive often thought the easiest way of getting round the watched flags issue (whle MP2 is stick in development anyhow) would be to create an app that could be used outside of the main config screen to import & export the watched flags. TV Series has this import / export but inside the plugin config, whereas this would need to be something that could be run without user input. Then I could call this from both sides of my batch file.. ie

    export watched flags from local database
    overwrite local database with copy database
    import / merge watched flags (ie watched flag - true wins when watched flags are different)
    start mediaportal

    That way, everytime mediaportal runs, the watched flags would resync when the local database is updated.

    My only issue is I havent got a scooby-do how to design an app that would do it....

    The other issue you may have is Im not sure how you would specify different a different database directory for (say) Moving Pictures & TV Series. mediaportaldirs.xml specifies the database directory which (as far as I know) is used by all the plugins? I could be wrong though...

    You wouldn't need to specify a different directory, I wouldn't think. Just call out movingpictures.db3 explicitly in your xcopy command. And the same for TVSeriesDatabase4.db3, if those are the only two that need to be copied.

    Code:
    xcopy \\[mynas]\MediaPortal\Main\Database\[COLOR="Red"]movingpictures.db3[/COLOR] C:\ProgramData\Team MediaPortal\MediaPortal\database /D /C /Y
    start "MediaPortal" "C:\Program Files (x86)\Team MediaPortal\MediaPortal\MediaPortal.exe"

    OK- yea, I see what you mean now. That would work; though if its just to save time copying- Id give it a go first (I literally cant read the command prompt text it happens so quick!).
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    One quick-n-dirty solution could be as follows:

    Adjust batch script into:

    Code:
    IF NOT EXIST "\\[mynas]\Some.Folder.With.Write.Access\Sync.Databases\HTPC.1.movingpictures.db3" xcopy "C:\ProgramData\Team MediaPortal\MediaPortal\database\movingpictures.db3" "\\[mynas]\Some.Folder.With.Write.Access\Sync.Databases\HTPC.1.movingpictures.db3" /C /Y
    xcopy "\\[mynas]\MediaPortal\Main\Database\movingpictures.db3" "C:\ProgramData\Team MediaPortal\MediaPortal\database" /D /C /Y
    start "MediaPortal" "C:\Program Files (x86)\Team MediaPortal\MediaPortal\MediaPortal.exe"

    HTPC.1.movingpictures.db3 would have to be renamed 2, 3, etc for all the other clients.

    And then somebody with more knowledge on how the watched flag is added/updated inside the SQLite database could write a little program that would merge the differences on the 'watched' status. Viewing the database via SQLite Database Browser I can see that there is a table 'watched_history' and 'movie_info__watched_history' which seem to be the only one related to the 'watched' status (but I don't know enough to say that for sure).

    Since SQLite is used by a lot of people, I figured there has to be something out already, but all I found so far are Shareware solutions such as: SQLite Compare - Diff / Merge Utility , but there has to be some open source solution out there, or maybe somebody reading this with SQL experience could join the effort :D
     

    Surferosa

    Portal Pro
    September 2, 2009
    55
    5
    Home Country
    England England
    Yea.. I second that. Nice work: if we can find a way to merge these we have a complete multi-seat solution.

    Might be worth posting in the TV Series forum as the functionality already exists in that config.

    Edit: I just checked my batch files (its a little while since I set this up) and I still host all of the databases on the server itself. So, inside mediaportal directory, I have subs of:

    Main_Database
    Office_Database
    Kids1_Database
    Kids2_Database

    No change to any of how the above works- just that Im hosting all the different Database files on the server. The xcopy still copys them from Office - Kids1 as I describe above. The reason why was I had some issues getting MediaPortal to run with a local database and networked thumbs.
     

    Users who are viewing this thread

    Top Bottom