Flag Mover: backup and restore watched flags (1 Viewer)

aspik

Retired Team Member
  • Team MediaPortal
  • April 14, 2008
    1,322
    586
    @Lehmden
    Sorry, I can't reproduce it here. The show you've posted doesn't have any external identifier (TVDB ID or IMDB-ID). I'm loading only series which have an external identifier, this show is not included in the search result.

    Could you please reproduce it and this time upload logs? :)
     

    Lehmden

    Retired Team Member
  • Premium Supporter
  • December 17, 2010
    12,553
    3,934
    Lehmden
    Home Country
    Germany Germany
    Hi.
    I'm trying to reproduce it with a tiny test collection. If the series without TVDB-ID is not watched, all went well. This is how you most likely has done your test. If this series or at least one episode is set as watched, FlagMover did not finish it's job. No "WatchedEpisodes.json" is written at all...
    The log is shown this:
    [2019-06-09 07:54:53,239] [12725 ] [AMQ 'ScreenManager'] [DEBUG] - ScreenManager: Showing dialog 'BackupDialog'
    [2019-06-09 07:54:53,239] [12725 ] [AMQ 'SwitchBrowseModeAction'] [DEBUG] - WorkflowManager: Attaching GUI model '4cdd601f-e280-43b9-ad0a-6d7b2403c856' to workflow state 'de1a86f8-50d4-4cbf-8cca-557eeeacd3c4'
    [2019-06-09 07:54:53,419] [12905 ] [Thread10 ] [INFO ] - Finished refeshing News Feeds ...
    [2019-06-09 07:54:57,635] [17121 ] [Thread134] [WARN ] - ThreadPool.ProcessQueue(): Exception during processing work ''
    System.OverflowException: Der Wert für einen UInt32 war zu groß oder zu klein.
    bei System.Number.ParseUInt32(String value, NumberStyles options, NumberFormatInfo numfmt)
    bei System.Convert.ToUInt32(String value)
    bei FlagMover.Utilities.MediaItemAspectsUtl.GetTvdbId(MediaItem mediaItem)
    bei FlagMover.Services.MoverOperations.GetLibraryEpisode(MediaItem episodeMediaItem, Int32 epNumber)
    bei FlagMover.Services.MoverOperations.AddEpisodes(IList`1 watchedEpisodes, MediaItem episodeMediaItem, List`1 episodeNumbers)
    bei FlagMover.Services.MoverOperations.BackupSeries(String path)
    bei FlagMover.Models.BackupModel.<BackupLibrary>b__3_0()
    bei MediaPortal.Common.Threading.Work.Process()
    bei MediaPortal.Common.Services.Threading.ThreadPool.ProcessQueue()
    [2019-06-09 07:55:17,638] [37124 ] [Thread10 ] [DEBUG] - ThreadPool.ProcessQueue(): Quitting (inUse: 0, total: 2)
    [2019-06-09 07:55:26,919] [46405 ] [InputMgr ] [DEBUG] - ScreenManager: Preparing to close topmost dialog '432ed3e1-e341-48db-9ac9-115e351c2f9f'...
    This is the end of the story nothing else happens... For all who can't read German... "Der Wert für einen UInt32 war zu groß oder zu klein." means "The value of an UInt32 was too big or too small"


    Another question... Why did you store the "show_title" in the JSON file? Luckily the restore also did work with something like:

    {"show_imdb":null,"show_tvdb":79546,"show_title":"","season":1,"number":3}.

    or like this:

    {"show_imdb":null,"show_tvdb":79546,"show_title":null,"season":1,"number":3}
    (this format is what I will use for MP2-Buddy)

    Any other tool of this kind (Kodi, MyTVSeries,...) did not use show title at all. Show title is not unique, depends on the used language and is not needed to identify an episode. TVDB-ID, season and episode number is all that is needed. If it's not too much work you could remove it from the JSON file completely while you are fixing the "No TVDB-ID series" issue, imho... If it stays it did not matter that much as things are working nevertheless...
     

    Lehmden

    Retired Team Member
  • Premium Supporter
  • December 17, 2010
    12,553
    3,934
    Lehmden
    Home Country
    Germany Germany
    Another thing... I didn't tested it up to now as I'm focusing on series atm. but I can imagine that there also will be a similar issue with movies that don't have any IMDB-ID number... You probably may have a look there too...
     

    Lehmden

    Retired Team Member
  • Premium Supporter
  • December 17, 2010
    12,553
    3,934
    Lehmden
    Home Country
    Germany Germany
    Hi.
    @aspik

    A different issue. After restoring about 60% of my series collection I got this error:

    I first had this error in the middle of restoring:
    [2019-06-09 16:26:39,589] [6272092] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '3', Episode = '1', Show TVDb ID = '295685', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,604] [6272108] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '0', Episode = '3', Show TVDb ID = '71020', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,624] [6272128] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '2', Episode = '17', Show TVDb ID = '281470', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,648] [6272151] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '16', Show TVDb ID = '72449', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,664] [6272167] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '4', Episode = '9', Show TVDb ID = '85060', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,689] [6272192] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '3', Show TVDb ID = '71635', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,706] [6272209] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '10', Show TVDb ID = '81025', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,729] [6272232] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '3', Show TVDb ID = '261557', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,750] [6272253] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '1', Show TVDb ID = '77216', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,783] [6272287] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '12', Episode = '7', Show TVDb ID = '75682', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,811] [6272314] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '6', Show TVDb ID = '121361', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,839] [6272342] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '3', Show TVDb ID = '73710', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,857] [6272360] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '2', Episode = '1', Show TVDb ID = '266967', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,882] [6272385] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '4', Episode = '21', Show TVDb ID = '72227', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,904] [6272408] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '2', Episode = '10', Show TVDb ID = '255316', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,929] [6272433] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '16', Show TVDb ID = '71528', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,951] [6272454] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '3', Episode = '12', Show TVDb ID = '281053', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,973] [6272476] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '6', Episode = '21', Show TVDb ID = '73153', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:39,995] [6272498] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '2', Episode = '3', Show TVDb ID = '77170', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,015] [6272518] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '4', Episode = '12', Show TVDb ID = '79488', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,048] [6272551] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '3', Episode = '25', Show TVDb ID = '232731', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,065] [6272569] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '4', Episode = '3', Show TVDb ID = '248861', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,092] [6272595] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '3', Episode = '7', Show TVDb ID = '133271', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,119] [6272622] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '3', Episode = '17', Show TVDb ID = '80379', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,144] [6272647] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '3', Episode = '11', Show TVDb ID = '78817', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,157] [6272660] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '4', Show TVDb ID = '273298', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,177] [6272680] [Thread85 ] [INFO ] - FlagMover: Marking episode as watched in library. Title = ', Season = '1', Episode = '8', Show TVDb ID = '82066', Show IMDb ID = '<empty>'
    [2019-06-09 16:26:40,179] [6272682] [Thread85 ] [WARN ] - ThreadPool.ProcessQueue(): Exception during processing work ''
    System.OverflowException: Der Wert für einen UInt32 war zu groß oder zu klein.
    bei System.Number.ParseUInt32(String value, NumberStyles options, NumberFormatInfo numfmt)
    bei System.Convert.ToUInt32(String value)
    bei FlagMover.Utilities.MediaItemAspectsUtl.GetTvdbId(MediaItem mediaItem)
    bei FlagMover.Services.MoverOperations.CreateLookupKey(MediaItem episode)
    bei FlagMover.Services.MoverOperations.MatchEpisodeAndMarkAsWatched(RestoreResult result, ILookup`2 loadedEpisodes, MediaItem episode)
    bei FlagMover.Services.MoverOperations.RestoreWatchedSeries(String path)
    bei FlagMover.Models.RestoreModel.<RestoreLibrary>b__3_0()
    bei MediaPortal.Common.Threading.Work.Process()
    bei MediaPortal.Common.Services.Threading.ThreadPool.ProcessQueue()
    Then I removed everything before the last successfully restored item from the .json file and tried again to restore. Then this Timeout happens:

    [2019-06-09 16:50:19,847] [143092 ] [Thread81 ] [WARN ] - ThreadPool.ProcessQueue(): Exception during processing work ''
    System.AggregateException: Mindestens ein Fehler ist aufgetreten. ---> UPnP.Infrastructure.CP.UPnPException: Error invoking action 'X_MediaPortal_Search' ---> UPnP.Infrastructure.CP.UPnPRemoteException: Network error when invoking action 'X_MediaPortal_Search': Die Anfrage wurde abgebrochen: Timeout für Vorgang überschritten.
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.AsyncActionCallResult.GetOutParams()
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.EndInvokeAction(IAsyncResult result)
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.EndInvokeAction(IAsyncResult result)
    bei System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
    --- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
    bei System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
    bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    bei MediaPortal.Common.Services.ServerCommunication.UPnPContentDirectoryServiceProxy.<SearchAsync>d__38.MoveNext()
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
    bei System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
    bei System.Threading.Tasks.Task`1.get_Result()
    bei FlagMover.Services.MoverOperations.RestoreWatchedSeries(String path)
    bei FlagMover.Models.RestoreModel.<RestoreLibrary>b__3_0()
    bei MediaPortal.Common.Threading.Work.Process()
    bei MediaPortal.Common.Services.Threading.ThreadPool.ProcessQueue()
    ---> (Interne Ausnahme #0) UPnP.Infrastructure.CP.UPnPException: Error invoking action 'X_MediaPortal_Search' ---> UPnP.Infrastructure.CP.UPnPRemoteException: Network error when invoking action 'X_MediaPortal_Search': Die Anfrage wurde abgebrochen: Timeout für Vorgang überschritten.
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.AsyncActionCallResult.GetOutParams()
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.EndInvokeAction(IAsyncResult result)
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.EndInvokeAction(IAsyncResult result)
    bei System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
    --- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
    bei System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
    bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    bei MediaPortal.Common.Services.ServerCommunication.UPnPContentDirectoryServiceProxy.<SearchAsync>d__38.MoveNext()<---
    Result is again that FlagMover did not continue to work...

    I think this is an memory issue as the server had an OOM exception nearly at the same time...
    [2019-06-09 16:51:28,145] [46231244] [75 ] [ERROR] - Warn.: Error handling SOAP request: Eine Ausnahme vom Typ "System.OutOfMemoryException" wurde ausgelöst.
    By the way, the PC has 8 Gbyte of RAM, should be enough, I think...
     

    aspik

    Retired Team Member
  • Team MediaPortal
  • April 14, 2008
    1,322
    586
    If the series without TVDB-ID is not watched, all went well. This is how you most likely has done your test.
    Yes, that's what I've tested.
    If this series or at least one episode is set as watched, FlagMover did not finish it's job.
    tested this too and I can't reproduce it with your test data.
    I think this is an memory issue as the server had an OOM exception nearly at the same time...
    FlagMover is a client plugin. Memory issues on server side are out of control of this plugin.

    Attached in first post a new version (v2). I changed the type of the TVBD-ID to string, same as I already get it from MP2. Previously it was uint, because trakt needed this type and as I already said, this is a "side-effect plugin" of the trakt plugin. Please try and report back.
     

    Lehmden

    Retired Team Member
  • Premium Supporter
  • December 17, 2010
    12,553
    3,934
    Lehmden
    Home Country
    Germany Germany
    Hi.
    FlagMover is a client plugin. Memory issues on server side are out of control of this plugin.
    Of Course, but as client and server are on the same machine here, it probably is related to each other...

    Will try the new version tomorrow, thanks.
     

    Lehmden

    Retired Team Member
  • Premium Supporter
  • December 17, 2010
    12,553
    3,934
    Lehmden
    Home Country
    Germany Germany
    Hi.
    I've tried the new version first with my whole collection, as this was available directly. Sadly it still did not work. The same error occurs tahn before:
    [2019-06-10 12:05:45,102] [1346107] [Thread122] [WARN ] - ThreadPool.ProcessQueue(): Exception during processing work ''
    System.AggregateException: Mindestens ein Fehler ist aufgetreten. ---> UPnP.Infrastructure.CP.UPnPException: Error invoking action 'X_MediaPortal_Search' ---> UPnP.Infrastructure.CP.UPnPRemoteException: Network error when invoking action 'X_MediaPortal_Search': Die Anfrage wurde abgebrochen: Timeout für Vorgang überschritten.
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.AsyncActionCallResult.GetOutParams()
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.EndInvokeAction(IAsyncResult result)
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.EndInvokeAction(IAsyncResult result)
    bei System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
    --- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
    bei System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
    bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    bei MediaPortal.Common.Services.ServerCommunication.UPnPContentDirectoryServiceProxy.<SearchAsync>d__38.MoveNext()
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
    bei System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
    bei System.Threading.Tasks.Task`1.get_Result()
    bei FlagMover.Services.MoverOperations.RestoreWatchedSeries(String path)
    bei FlagMover.Models.RestoreModel.<RestoreLibrary>b__3_0()
    bei MediaPortal.Common.Threading.Work.Process()
    bei MediaPortal.Common.Services.Threading.ThreadPool.ProcessQueue()
    ---> (Interne Ausnahme #0) UPnP.Infrastructure.CP.UPnPException: Error invoking action 'X_MediaPortal_Search' ---> UPnP.Infrastructure.CP.UPnPRemoteException: Network error when invoking action 'X_MediaPortal_Search': Die Anfrage wurde abgebrochen: Timeout für Vorgang überschritten.
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.AsyncActionCallResult.GetOutParams()
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.EndInvokeAction(IAsyncResult result)
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei UPnP.Infrastructure.CP.DeviceTree.CpAction.EndInvokeAction(IAsyncResult result)
    bei System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
    --- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
    bei System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
    bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    bei MediaPortal.Common.Services.ServerCommunication.UPnPContentDirectoryServiceProxy.<SearchAsync>d__38.MoveNext()<---
    and FlagMover stops working without any further notice. There still are timeouts if the collection is really big..

    The test with the tiny test- collection I will do later today...
     

    Lehmden

    Retired Team Member
  • Premium Supporter
  • December 17, 2010
    12,553
    3,934
    Lehmden
    Home Country
    Germany Germany
    Hi.
    The test with the tiny collection incl. the series without any TVDB-ID was a complete success. Everything is working. Even the series is included in the JSON file, also it is not restored because of the missing ID...
    So the timeout issue with my full collection is the only remaining one...
     

    aspik

    Retired Team Member
  • Team MediaPortal
  • April 14, 2008
    1,322
    586
    The same error occurs tahn before:
    attached a new test version in first post. Please give it a try, thanks.
    The test with the tiny collection incl. the series without any TVDB-ID was a complete success. Everything is working. Even the series is included in the JSON file, also it is not restored because of the missing ID...
    then you have a different test data. The one you've posted gets imported by MP2 without the ExternalIdentifierAspect. As a result FlagMover ignores it.
    (e.g. if the series only is listed on TMDB or nowhere at all)
    it looks like on your system this series has a TMDB id. Which adds the ExternalIdentifierAspect to it. Can you please check this?

    Would it make sens to additionally check for the TMDB id for series?
     

    Users who are viewing this thread

    Top Bottom