MovPic v1.0.0 Beta: Dynamic Actors = extremly slow (1 Viewer)

[RoChess]

My wife forced me into removing the Dynamic Actor category.

It adds almost 3 minutes before MovingPictures can be used. I suspect that this has to do with the massive amount of SQL queries being done to filter the list on my collection. I tried adjusting the advanced setting to only show actors who are in 5+ movies, and even changed it to 1 (and 0), but it's still a massive delay.

Since it is still beta, I'm sure I just have to be patient, but wanted to make sure the issue is known.

 

[fforde]

Yes, we are aware of the problem, I created an issue on the tracker so we make sure this gets attention though. Star the issue if you'd like email updates as progress updates are made.

 

[RoChess]

Thank you. I must say that I found no delays in using it, but the initial load was excruciating.

I assume the main slowdown is correlating how many movies an actor has played in, to decide if they should be listed. On 1000 movies with 5 actors per movies, that is 5000 actor entries that have to be checked against the 1000 movies to see if they played in each of them. Roughly speaking that would boil down to 5 million checks, which would explain the slowdown.

Perhaps there needs to be some SQL caching added, like an extra field that keeps track on how many movies an actor has been in. That 'counter' could be updated auto during the adding/removal of a movie, or manual from inside the configuration. Then on building the category views, you use that counter to quickly filter out the actors that played in the amount of movies configured in the advanced settings (2 by default).

I'm probably oversimplifying things, but sometimes that helps

 

[tourettes]

Thank you. I must say that I found no delays in using it, but the initial load was excruciating.

I assume the main slowdown is correlating how many movies an actor has played in, to decide if they should be listed. On 1000 movies with 5 actors per movies, that is 5000 actor entries that have to be checked against the 1000 movies to see if they played in each of them. Roughly speaking that would boil down to 5 million checks, which would explain the slowdown.

Perhaps there needs to be some SQL caching added, like an extra field that keeps track on how many movies an actor has been in. That 'counter' could be updated auto during the adding/removal of a movie, or manual from inside the configuration. Then on building the category views, you use that counter to quickly filter out the actors that played in the amount of movies configured in the advanced settings (2 by default).

Sometimes huge data amounts aren't possible to be handled with "real-time" queries but instead you need to take a different approach and turn the responsibilities upside down. Instead of generating the results always on the fly it could be possible to generate that data only when inserting new data (importing a new movie) to the database. I'm not sure how Moving Pictures handles the current situation and not sure how easy such different approach would be, would it be even possible (some categories can be build based on the watched status? So it would require MP plugin side to do DB updates for categories). But in general that is pretty good way to improve performance to a different level (with especially good scalability as the UI wont create x*y*z... like queries where the performance drops radically when data amounts increase). Unfortunately it works only if the data is pretty static, meaning that application side wont do that much changes into it.

This is what was done for the EPG pretty recently, as in the worst case there could have been few second delays for a button press as the SQL queries needed to be run for the huge amount of EPG data (with complex recording rules).

 

[fforde]

Well there are some things we can precalculate and some things we can not. Because of the flexibility of the Categories and Filtering systems though and because they can be used in concert, it is impossible to precalculate members of all lists. We might be able to do this for some though. Also one of the reasons the actors category is slow is because we do not currently have an Actors table. Meaning that when checking for movies starring Tom Cruise, a "string contains" call has to be done for every movie in your collection. This obviously is far from ideal.

 

[Guzzi]

AW: MovPic v1.0.0 Beta: Dynamic Actors = extremly slow

Hi fforde,
I have another question:
While there is an option to disable the importer in the GUI I did not find any option to turn off all "workintensive stuff", that is done in the background (especially updating category assignments, updating movie status, etc.)
All this is very timeconsuming - especially when the stuff is located on a relatively slow NAS - it slows down the start of MP and makes the GUI sluggish at least for some minutes.
So: Is it - or would it be - possible to disable all those functions (or schedule them for the nighttime) so that in the GUI the plugin is only used to display infos, chose movies and start them?
I think this problem will disappear anyway when those functions will be separated in teh future (server side will do the work, client side will only do display and watching), but this is far way ;-)
Any hints?

Thanks,
Guzzi

 

[Guzzi]

AW: MovPic v1.0.0 Beta: Dynamic Actors = extremly slow

Hi fforde,
while thinking about the reasons, why MyFilms is so extremely fast and MoPics so slow, I think the difference is obvious: MyFilms offers the user to SEARCH for Movies in the DB and presenting the search result - while MoPics builds a complete view, that trequires the search for ALL infos.
While I very much like the concept of MoPics that is very flexible and allows almost anything to be defined, I think to help the user find movies with certain aspects, a search functin might me the (better performing) approch. Let the user search for "all movies having MikeB as actor" or "All movies with xyzabc ind descriptionstrin" - this should perform fast (it does in MyFilms almost instantly) and is something that could at a later point also be used to improve navigation experience in the GUI (Like showing other films with same actor as current). What do you think?

 

[FireSokar]

I have the RC1 installed and have no catagorys but ever since upgrading to the new versions of moving pictures I have a delay between 2-5 seconds when launching Mediaportal. Is there a reason that will be causing this?