Stuck at "Retrieving possible matches". (1 Viewer)

[luci5r]

Let me clarify what's going on ...

Till Friday morning, I was running MediaPortal 1.3.0.0 Alpha w/ StreamedMP 1.8.0 and MovingPictures 1.3.1, and everything was working absolutely flawless. Been working like that for weeks and no problems.

Friday morning I decided to update StreamedMP to the latest 1.8.1 & MovingPictures to the latest 1.4.0, and the fabled Sh*t hit the fan. MediaPortal started crashing and nothing was working anymore. Long story short - the culprit turned out to be a corrupted movingpictures.db3 database. I replaced the corrupted db3 with a working older backup; and fired up MediaPortal and everything started working absolutely fine.

Then - the problem started --- The MovingPictures "Configuration" stopped working. Basically it got stuck at "Retrieving Possible Matches". I've had this problem before because of a corrupted db3 database. At that time i had to use an even older backup & it had resolved the issue. This time I tried an older backup and it still didn't work. I finally resorted to deleting my MovingPictures db3 Database altogether and let it create a new one. STILL it didn't work. Same problem. Stuck at "Retrieving possible matches". No matter what I did - I couldn't get MovingPictures to add new titles to the database.

I finally resorted to finding and installed MovingPictures 1.3.1 (the older version), and once again tried with both an older backup and a fresh database --- STILL the same problem. Stuck at "Retrieving possible matches".

So at this time:

Whether I use MovingPictures 1.3.1 or 1.4.0, and whether I delete the movingpictures.db3 completely and let the plugin create a fresh database, or use a working older backup --- I get stuck at Retrieving possible matches.

MediaPortal & StreamedMP continue to work fine.

Any suggestions?

Thanks.

 

[HomeY]

Basically it's getting stuck at "Retrieving Possible Matches"

Did you check the Regex Noise Filter again??
Reference topic: https://forum.team-mediaportal.com/threads/stuck-on-retrieving-possible-matches.110065/

 

[luci5r]

Basically it's getting stuck at "Retrieving Possible Matches"

Did you check the Regex Noise Filter again??
Reference topic: https://forum.team-mediaportal.com/threads/stuck-on-retrieving-possible-matches.110065/

Damnit!! I can't believe it -- It's the same damn thing again --- Your'e Right!!
The Regex Noise Filter is corrupted again. What I don't understand is why it's doing so even if I delete my db3 completely and let the plugin create a fresh new one!! Either way, I'm working to fix it now. Thanks for pointing this one out. I'll post back how I make out.

BTW, I'm going to try MyVideos that you suggested in the other post. Thanks again.

 

[HomeY]

It might also be interesting to figure out why this keeps happening to your database...

Are you by any chance using the a Central Database as described in the Wiki?

 

[luci5r]

It might also be interesting to figure out why this keeps happening to your database...

Are you by any chance using the a Central Database as described in the Wiki?

Couple of things:

I don't believe I'm using a Central Database; I have never seen that article before and haven't followed the procedure given in that article. That said, I have a WebServer(Apache/PHP) that runs a web application which "READS" the movingpictures.db3 to display my movie library online. I use PHP to read the sqlite database, and I only have READ permissions for PHP. I must however strongly assert that although I've encountered this corrupted database 2 or 3 times now --- it has always been after an Update or some other MediaPortal related installation/update, so I would seriously be doubtful that the web application has anything to do with it. The web application runs 24/7 ... i would have this problem all the time if that was the culprit.

Secondly, I did indeed go into the movingpictures.db3 and manually fix the Noise filter row with correct values (taken from that other thread); however -- at this time "MediaPortal Configuration" crashes during launching the configuration. So my first question is, is the value for that filter still the same in 1.4.0 as it was in 1.3.1 ??

I'm using this:

(([\(\{\[]|\b)((576|720|1080)[pi]|dir(ectors )?cut|dvd([r59]|rip|scr(eener)?)|(avc)?hd|wmv|ntsc|pal|mpeg|dsr|r[1-5]|bd[59]|dts|ac3|blu(-)?ray|[hp]dtv|stv|hddvd|xvid|divx|x264|dxva|(?-i)FEST[Ii]VAL|L[iI]M[iI]TED|[WF]S|PROPER|REPACK|RER[Ii]P|REAL|RETA[Ii]L|EXTENDED|REMASTERED|UNRATED|CHRONO|THEATR[Ii]CAL|DC|SE|UNCUT|[Ii]NTERNAL|[DS]UBBED)([\]\)\}]|\b)(-[^\s]+$)?)

 

[HomeY]

If you can't recall using the Central Database, then we can assume you don't use that. That takes 1 option out of the possible causes...

About your Webserver: Since only 1 user can edit the database at the same time (SQLite dB limitation) this comes close to the cause. Maybe it's an idea to write a simple .bat file with a robocopy command that copies your MovingPictures dB to another location (you can add that .bat to Windows Scheduler so it runs daily for example) so your webserver can read it from there? There are websites like Trakt.tv or follw.it that can sync your entire collection online That way you can also 'backup' your watched status of your database, in case you have to replace it again (but let's hope not...).

Regex filter hasn't changed (as far as i know)

 

[luci5r]

If you can't recall using the Central Database, then we can assume you don't use that. That takes 1 option out of the possible causes...

About your Webserver: Since only 1 user can edit the database at the same time (SQLite dB limitation) this comes close to the cause. Maybe it's an idea to write a simple .bat file with a robocopy command that copies your MovingPictures dB to another location (you can add that .bat to Windows Scheduler so it runs daily for example) so your webserver can read it from there? There are websites like Trakt.tv or follw.it that can sync your entire collection online That way you can also 'backup' your watched status of your database, in case you have to replace it again (but let's hope not...).

Regex filter hasn't changed (as far as i know)

An important thing to note in your hypothesis is "Edit", the reality is that the PHP script doesn't actually "Edit", but only "Reads" the database. Also, like I stated - if that was the problem, I would imagine it would happen way too often - like all the time. Still -- I like your idea and the alternative, just to completely RULE out the PHP/Web Application interference. I will indeed use a batch file or something to make a copy of the db3 and let the web application use that. Works for me!

UPDATE

I do have an update. So I decided to re-install MovingPictures 1.4.0 and delete my existing movingpictures.db3 completely. Fired up the Configuration and let it create a fresh new database. At this time it is retrieving information and everything is proceeding as normal.

I don't understand, nor do I have an explanation, as to why it's working now when it didn't 15 minutes ago - but we all know that stranger things have happened. Also, I now have to re-import & establish my over 250+ titles, which is a pain, and since I have no real, solid information on a) what the problem was, and b) how I solved it --- this could potentially happen again at any time.

This all said, I think I need to start looking at alternatives to MovingPictures, like MyVideos or something. Not quite sure what else to do here.

 

[HomeY]

I would make a backup of that new database directly after you imported all movies again.

Remember that we really need full debug logs to figure out what's going on. @fforde might have some suggestions for you, but he'll need your logs.

 

[RoChess]

The problem is not always "read" permission on the database, but an SQL-lock on the table/cell. These locks can also be placed when you are just "Reading" at SQLite level. But the lock prevents data from being written then if MovPic at that exact same time needs it as well. SQLite supports concurrency, but it needs to be coded in and MovPic is not specifically written for that scenario, hence multi-user setups are unsupported. The best way to go about it is to "copy" the database at Windows API level, because then you are truly 'just reading' the file and Windows API will still be able to process any write API requests from MovPic at the same time to the source.

So just 'copy' movingpictures.db3 to say movingpictures.php.db3 and then do your PHP thing with that copy. I've got 4k+ entries and my movingpictures database is 35MB, a 'copy' action takes a split second and that is on a slow HDD. If you are setup with an SSD or have write-cache enabled it will take a few nanoseconds. To be sure that your 'copy' did not corrupt. Say for example nanosecond 1 you copy first half, then nanosecond 2 MovPic makes a change and nanosecond 3 you copy the last half, you end up with a corrupted copy. Simply run an SQL integrity command on the copy i.e.: pragma integrity_check;

If result is "OK", then you are good to go.

 

[luci5r]

The problem is not always "read" permission on the database, but an SQL-lock on the table/cell. These locks can also be placed when you are just "Reading" at SQLite level. But the lock prevents data from being written then if MovPic at that exact same time needs it as well. SQLite supports concurrency, but it needs to be coded in and MovPic is not specifically written for that scenario, hence multi-user setups are unsupported. The best way to go about it is to "copy" the database at Windows API level, because then you are truly 'just reading' the file and Windows API will still be able to process any write API requests from MovPic at the same time to the source.

So just 'copy' movingpictures.db3 to say movingpictures.php.db3 and then do your PHP thing with that copy. I've got 4k+ entries and my movingpictures database is 35MB, a 'copy' action takes a split second and that is on a slow HDD. If you are setup with an SSD or have write-cache enabled it will take a few nanoseconds. To be sure that your 'copy' did not corrupt. Say for example nanosecond 1 you copy first half, then nanosecond 2 MovPic makes a change and nanosecond 3 you copy the last half, you end up with a corrupted copy. Simply run an SQL integrity command on the copy i.e.: pragma integrity_check;

If result is "OK", then you are good to go.

@ HomeY:

So I finally got everything working again. Sat down & re-scraped all my 250+ titles (Actually didn't take as long as I thought). My foreign films collection were a bit tedious -- but it's all working good & fine right now. And yes, I made backups every step of the way; especially after I was done. I'm also going to schedule some sort of a weekly (or frequent) backup process for the db3.

@ RoChess:

I agree. Whether or not PHP is causing the issue, I'm in the process of setting a copy of the file that will be read by PHP, rather then using the actual live db3. It makes sense and is the only way to be sure. Thanks for your advice.

Thanks guys.