Episodes deleting after import (1 Viewer)

[eetaylog]

Ive had a strange problem with TVSeries for a while now. whenever i add a new series to my database and do an import for getting the new information, my database seems to randomly drop the video pointer addresses for other episodes in existing series. If for example i import the information for the whole series of Flashforward, it will drop the pointer (T:\TV Series\.....) for a couple of episodes of Lost.

This means that every time i add a new show, or update the information for existing shows, i have to go and check every single episode of all my shows to check that the address pointer is still there (by checking that the episode names arent greyed out). And when i find the odd episode thats missing, i have to manually open the database with SQLlite and enter the address manually incase another scan just worsens the problem!

im using version 2.3.3517 on MP 1.0.1 btw, i know its an older version, but im having other issues with the later versions, so this is the one thats the most stable for me.

any pointers (no pun intended!)?

 

[BennTech]

I'm having a similar issue. I'm watching something in TV Series 2.5.3, exit MediaPortal and go into My TvSeries config and notice that all the 4700+ episodes of my entire TVSeries database are gone except maybe a dozen random episodes. Happened immediately after upgrading from 2.3.2589 to 2.5.3 last night. Had to reimport entire DB. Then today I noticed a pesky series was missing (non-standard season names that needs special arrangement of parsing expressions). After reimporting that one, I notice all my series are empty again. Another half hour of reimporting everything.

Can't verify if was actually caused by import because I wasn't really expecting it erase DB. And can't verify if it's filename corruption like eetaylog because I reimported DB before coming here looking for answers.

When I have a bit more time, I'll trying troubleshooting this. I notice there's a DB backup after the upgrade. Is 2.5.3 using a new DB format (or at least a slightly modified one) even though both are v4? Could this be a corrupt DB conversion? I'll delete DB to test, but I'm looking for where TVSeries config is stored and can't find it. It's not stored in DB, is it? Don't want to delete DB just test this and loose all my settings I've tweaked over the years and long forgotten what I did.

I gotta get this fixed. Reimporting my entire database every time I go into config is a real PITA.

To answer my own question, yes, TV Series config is stored in the database, as I found out in test delete. Deleting the database and rebuilding all my settings is way too much work for me to in the foreseeable future. Also, "Empty" PB next to the database path doesn't erase the entries in the DB because DB stays same size. Thus, any potential corruption would still be present.

And the cause is definitely after import. For me, the local filename of missing entries is completely blank.

 

[eetaylog]

Ive got a feeling its something to do with the 'automatically choosed aired/series' settings in TVSeries config (online data tab).

Why dont you just make a copy of your TVSeriesdatabase4.db3 file (in your c:\...\mediaportal/database folder) before doing an import, and then if your DB messes up you can just replace it with the saved copy rather than reimporting everything.

 

[BennTech]

Yeah, making a copy is what I've been doing since the second loss of data (and hence answering my own question). Don't see how this be related to "automatically choose aired" because I don't do that. I like manually picking the order.

 

[BennTech]

OK, so I got fed with this problem and did some more digging. I turned on debug/SQL and did a simple import. After the import, my entire DB was wiped except for my new import. In the log, I found a huge select statement that basically selects my entire TV database and deletes every entry one by one. Here's my log excerpt:

00000006 - 5/31/2010 4:47:54 PM - Executing SQL: select online_episodes.CompositeID, online_episodes.EpisodeID, online_episodes.SeriesID, online_episodes.EpisodeIndex as online_episodesEpisodeIndex , online_episodes.SeasonIndex, online_episodes.EpisodeName, online_episodes.Watched, online_episodes.Summary, online_episodes.FirstAired, online_episodes.OnlineDataImported, online_episodes.GuestStars, online_episodes.Director, online_episodes.Writer, online_episodes.Hidden, online_episodes.lastupdated, online_episodes.DownloadPending, online_episodes.DownloadExpectedName, online_episodes.ThumbUrl, online_episodes.thumbFilename, online_episodes.Combined_episodenumber, online_episodes.Combined_season, online_episodes.DVD_chapter, online_episodes.DVD_discid, online_episodes.DVD_episodenumber, online_episodes.DVD_season, online_episodes.EpImgFlag, online_episodes.IMDB_ID, online_episodes.Language, online_episodes.ProductionCode, online_episodes.Rating, online_episodes.absolute_number, online_episodes.airsafter_season, online_episodes.airsbefore_episode, online_episodes.airsbefore_season, online_episodes.seasonid, local_episodes.EpisodeFilename, local_episodes.CompositeID, local_episodes.SeriesID, local_episodes.SeasonIndex, local_episodes.EpisodeIndex, local_episodes.LocalEpisodeName, local_episodes.LocalImportProcessed, local_episodes.AvailableSubtitles, local_episodes.CompositeID2, local_episodes.EpisodeIndex2, local_episodes.videoWidth, local_episodes.videoHeight, local_episodes.FileDateAdded, local_episodes.FileDateCreated, local_episodes.IsAvailable, local_episodes.ext, local_episodes.Removable, local_episodes.localPlaytime, local_episodes.VideoCodec, local_episodes.VideoBitrate, local_episodes.VideoFrameRate, local_episodes.VideoAspectRatio, local_episodes.AudioCodec, local_episodes.AudioBitrate, local_episodes.AudioChannels, local_episodes.AudioTracks, local_episodes.TextCount, local_episodes.StopTime from online_episodes left join local_episodes on (local_episodes.CompositeID=online_episodes.CompositeID)  where local_episodes.LocalImportProcessed = 2 and local_episodes.Removable = '0' and online_episodes.Hidden = 0 and online_episodes.CompositeID not in (select distinct local_episodes.CompositeID2 from local_episodes where local_episodes.CompositeID2 != '') union select online_episodes.CompositeID, online_episodes.EpisodeID, online_episodes.SeriesID, online_episodes.EpisodeIndex as online_episodesEpisodeIndex , online_episodes.SeasonIndex, online_episodes.EpisodeName, online_episodes.Watched, online_episodes.Summary, online_episodes.FirstAired, online_episodes.OnlineDataImported, online_episodes.GuestStars, online_episodes.Director, online_episodes.Writer, online_episodes.Hidden, online_episodes.lastupdated, online_episodes.DownloadPending, online_episodes.DownloadExpectedName, online_episodes.ThumbUrl, online_episodes.thumbFilename, online_episodes.Combined_episodenumber, online_episodes.Combined_season, online_episodes.DVD_chapter, online_episodes.DVD_discid, online_episodes.DVD_episodenumber, online_episodes.DVD_season, online_episodes.EpImgFlag, online_episodes.IMDB_ID, online_episodes.Language, online_episodes.ProductionCode, online_episodes.Rating, online_episodes.absolute_number, online_episodes.airsafter_season, online_episodes.airsbefore_episode, online_episodes.airsbefore_season, online_episodes.seasonid, local_episodes.EpisodeFilename, local_episodes.CompositeID, local_episodes.SeriesID, local_episodes.SeasonIndex, local_episodes.EpisodeIndex, local_episodes.LocalEpisodeName, local_episodes.LocalImportProcessed, local_episodes.AvailableSubtitles, local_episodes.CompositeID2, local_episodes.EpisodeIndex2, local_episodes.videoWidth, local_episodes.videoHeight, local_episodes.FileDateAdded, local_episodes.FileDateCreated, local_episodes.IsAvailable, local_episodes.ext, local_episodes.Removable, local_episodes.localPlaytime, local_episodes.VideoCodec, local_episodes.VideoBitrate, local_episodes.VideoFrameRate, local_episodes.VideoAspectRatio, local_episodes.AudioCodec, local_episodes.AudioBitrate, local_episodes.AudioChannels, local_episodes.AudioTracks, local_episodes.TextCount, local_episodes.StopTime from online_episodes left join local_episodes on (local_episodes.CompositeID2=online_episodes.CompositeID)  where local_episodes.LocalImportProcessed = 2 and local_episodes.Removable = '0' and online_episodes.Hidden = 0 and local_episodes.CompositeID2 != '' order by online_episodesEpisodeIndex asc 
00000006 - 5/31/2010 4:47:55 PM - [B]Success, returned Rows: 3382[/B]
00000006 - 5/31/2010 4:47:55 PM - Executing SQL: delete from local_episodes where local_episodes.EpisodeFilename = '\\MyServer\MySeriesPath\FileName1.ext'
00000006 - 5/31/2010 4:47:55 PM - Success, returned Rows: 1
00000006 - 5/31/2010 4:47:55 PM - Executing SQL: delete from local_episodes where local_episodes.EpisodeFilename = '\\MyServer\MySeriesPath\FileName2.ext'
00000006 - 5/31/2010 4:47:55 PM - Success, returned Rows: 1
00000006 - 5/31/2010 4:47:55 PM - Executing SQL: delete from local_episodes where local_episodes.EpisodeFilename = '\\MyServer\MySeriesPath\FileName3.ext'
00000006 - 5/31/2010 4:47:55 PM - Success, returned Rows: 1
[I](...deletion repeats 3382 times)[/I]

The select ends with "where local_episodes.LocalImportProcessed = 2 (not sure what "2" means) and local_episodes.Removable = '0' and online_episodes.Hidden = 0 and local_episodes.CompositeID2 != ''". I'm guessing because a file server share is not removable but also not local, every episode on my NAS (my entire media collection) is matching the select and thus getting deleted.

The question is why? Is deleting every episode on a file server a bug or a "feature"? And if it's a feature, how do I turn it off?

 

[SilentException]

This means that your files weren't accessible during scan.. It could be that you deleted them or removed removable hard drive or perhaps your NAS wasn't up.. If you want to keep references to unavailable files, turn on the "keep references" for that share..

 

[BennTech]

This means that your files weren't accessible during scan.. It could be that you deleted them or removed removable hard drive or perhaps your NAS wasn't up.. If you want to keep references to unavailable files, turn on the "keep references" for that share..

None of that applies. My NAS was up. The files were accessible. I always have "keep references" checked.

I've figured out what's happening, although I don't know if this a bug or a "feature". If I remove an import folder from my list, then every episode imported from that folder gets deleted from the DB, even if they still exist and are accessible.

So apparently in the import section we have to maintain a list of every path from which we've ever imported media? Otherwise, MP-TVSeries automatically deletes all entries on the missing import paths? Seriously? To me, that's like Winamp deleting your entire music library just because you changed the watch folders. All the media still exists--why are you deleting it? It's asinine. I don't understand the point of this "feature".

If you want to clean up missing entries, then check whether the files exist instead of wiping out files just because we removed some import path that we used ages ago. Yes, verifying files exist will take longer, so don't make it automatic every import--make it a user-initiated action as a pushbutton or a menu option. Plus that will solve the complaints in the numerous threads about removing duplicate/obsolete entries, to which the only PITA "solutions" are either to manually select hundreds or thousands of entries and delete them, or wipe out your entire DB and reimport everything from scratch.

I do have a workaround to the auto-delete: Keep all your old import paths, but uncheck "enabled". This keeps all your records (albeit by cluttering your import list). However, be warned that you will get the following "Commit failed"/"null err" bug for every episode on the disabled path (3382 times in my case):

00000006 - 6/1/2010 2:31:31 PM - Executing SQL: update local_episodes SET IsAvailable=0 where local_episodes.LocalImportProcessed = 2 and local_episodes.Removable = '0' and online_episodes.Hidden = 0
00000006 - 6/1/2010 2:31:31 PM - Commit failed on this command: <update local_episodes SET IsAvailable=0 where local_episodes.LocalImportProcessed = 2 and local_episodes.Removable = '0' and online_episodes.Hidden = 0> (SQLiteClient: TVSeriesDatabase4.db3 cmd:sqlite3_prepare16:pvm=null err:ERROR detailed:no such column: online_episodes.Hidden query:update local_episodes SET IsAvailable=0 where local_episodes.LocalImportProcessed = 2 and local_episodes.Removable = '0' and online_episodes.Hidden = 0).

This error appears to have no adverse effect. However, considering this occurs at the same location in the log as the auto-deletes, essentially replacing the deletes with an error, it seems quite evident to me that the SQL logic in this section of code is buggy and needs review, which will hopefully include a review of this auto-delete "feature".

 

[RoChess]

The whole point of import paths is to define where your collection resides, so removing an import path means you no longer want to make use of those files, so MP-TVSeries properly removes every reference to them.

I think you should be asking yourself first why you are removing import paths

It's the same as deleting the Winamp library entries and then wondering why you can't play your media anymore (eventhough MP3s still exist).

 

[SilentException]

Of course you have to leave import paths in the configuration. This is not Winamp. MP-TVSeries is constantly watching your import paths for changes and scanning them for any new episodes. Thus, if you remove import path, episodes are removed..

BTW, now that you have import paths sorted, is there any reason you keep them disabled? You rather see the error messages in the log? And, I'll take a look at the commit error..

Edit: that commit error should be already fixed in 2.5.4..