[fixed] MP2 Importer Exception of type 'System.OutOfMemoryException' was thrown. (1 Viewer)

Terra

Portal Pro
August 25, 2009
230
56
Austria Austria
Country flag
MediaPortal 2 Version: 2.2 Final

Description
Hi guys,

So I tried MP2 again after some time and I have the issue that the MP2 Service crashes while importing my media assets.
I now restarted it at least 10 times over the last 3 days but its really annoying, not sure if it will finish any time...

I have 8GB of RAM and 312GB is free on the SSD.

Version: 2.2.1811.11777
Source: MP2.2_Final-11cc07

Here is the ServerError.log excerpt:

Code:
[2018-12-08 11:06:14,355] [2323606] [130      ] [INFO ] - ImporterWorker: 51 % completed (4 ImportJob(s), in total 11057 of 21613 so far identified resources processed)
[2018-12-08 11:06:18,623] [2327873] [111      ] [ERROR] - Unable to query
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Collections.Generic.List`1..ctor(Int32 capacity)
   at MediaPortal.Backend.Services.MediaLibrary.QueryEngine.CompiledFilter.CompileStatementParts(MIA_Management miaManagement, IFilter filter, IFilter subqueryFilter, Namespace ns, BindVarNamespace bvNamespace, ICollection`1 requiredMIATypes, String outerMIIDJoinVariable, ICollection`1 tableJoins, IList`1 resultParts, IList`1 resultBindVars)
   at MediaPortal.Backend.Services.MediaLibrary.QueryEngine.CompiledFilter..ctor(MIA_Management miaManagement, IFilter filter, IFilter subqueryFilter, Namespace ns, BindVarNamespace bvNamespace, String outerMIIDJoinVariable, ICollection`1 tableJoins)
   at MediaPortal.Backend.Services.MediaLibrary.QueryEngine.MainQueryBuilder.CreateCompiledFilter(Namespace ns, BindVarNamespace bvNamespace, String outerMIIDJoinVariable, IList`1 tableJoins)
   at MediaPortal.Backend.Services.MediaLibrary.QueryEngine.MainQueryBuilder.GenerateSqlStatement(Boolean groupByValues, IDictionary`2 miamAliases, String& mediaItemIdOrGroupSizeAlias, IDictionary`2& attributeAliases, String& statementStr, IList`1& bindVars)
   at MediaPortal.Backend.Services.MediaLibrary.QueryEngine.CompiledMediaItemQuery.AddMultipleMIAResults(ISQLDatabase database, ITransaction transaction, MultipleMediaItemAspectMetadata miam, IDictionary`2 attributes, MainQueryBuilder builder, IDictionary`2 multipleMiaValues)
   at MediaPortal.Backend.Services.MediaLibrary.QueryEngine.CompiledMediaItemQuery.AddMultipleMIAs(ISQLDatabase database, ITransaction transaction, IEnumerable`1 selectedMIAs, IList`1 ids, IDictionary`2 multipleMiaValues)
   at MediaPortal.Backend.Services.MediaLibrary.QueryEngine.CompiledMediaItemQuery.Query(ISQLDatabase database, ITransaction transaction, Boolean singleMode)
[2018-12-08 11:06:21,324] [2330574] [87       ] [INFO ] - ImporterWorker.ImportJob(#5).DirectorySaveBlock: MediaLibrary disconnected. Requesting suspension...
[2018-12-08 11:06:21,328] [2330578] [145      ] [INFO ] - ImporterWorker.ImportJob(#4): Suspended
[2018-12-08 11:06:21,329] [2330579] [145      ] [INFO ] - ImporterWorker.ImportJob(#5): Suspended
[2018-12-08 11:06:21,330] [2330580] [145      ] [INFO ] - ImporterWorker.ImportJob(#6): Suspended
[2018-12-08 11:06:21,330] [2330580] [145      ] [INFO ] - ImporterWorker.ImportJob(#2): Suspended
[2018-12-08 11:06:21,331] [2330581] [145      ] [INFO ] - ImporterWorker: Suspended (4 ImportJobs pending)
[2018-12-08 11:06:20,933] [2330183] [67       ] [ERROR] - SettingsManager: Error setting property 'PendingFanArtActions' in settings of type 'FanArtActionSettings'
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
   at System.Text.StringBuilder.Append(Char* value, Int32 valueCount)
   at System.Text.StringBuilder.AppendHelper(String value)
   at System.Text.StringBuilder.Append(String value)
   at System.IO.StringWriter.Write(String value)
   at System.Xml.XmlTextWriter.WriteStartElement(String prefix, String localName, String ns)
   at System.Xml.XmlDOMTextWriter.WriteStartElement(String prefix, String localName, String ns)
   at System.Xml.XmlElement.WriteStartElement(XmlWriter w)
   at System.Xml.XmlElement.WriteElementTo(XmlWriter writer, XmlElement e)
   at System.Xml.XmlElement.WriteTo(XmlWriter w)
   at System.Xml.XmlElement.WriteContentTo(XmlWriter w)
   at System.Xml.XmlNode.get_InnerXml()
   at System.Xml.XmlElement.get_InnerXml()
   at MediaPortal.Common.Services.Settings.SettingsFileHandler.GetValue(String entryName, Type type)
   at MediaPortal.Common.Services.Settings.SettingsManager.LoadSettingsObject(Type settingsType, Boolean useGlobaleScope, Boolean useUserScope)
[2018-12-08 11:06:19,014] [2328264] [82       ] [ERROR] - OCVVideoThumbnailer: Exception reading resource '{03dd2da6-4da8-4d3e-9e55-80e3165729a3}:////s*/m*$/m*/R*.mkv' (Text: 'Exception of type 'System.OutOfMemoryException' was thrown.')
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at OpenCvSharp.VectorOfByte.ToArray()
   at OpenCvSharp.Cv2.ImEncode(String ext, InputArray img, Byte[]& buf, Int32[] prms)
   at MediaPortal.Extensions.MetadataExtractors.OCVVideoThumbnailer.OCVVideoThumbnailer.ExtractThumbnailAsync(ILocalFsResourceAccessor lfsra, IDictionary`2 extractedAspectData)
   at MediaPortal.Extensions.MetadataExtractors.OCVVideoThumbnailer.OCVVideoThumbnailer.<TryExtractMetadataAsync>d__8.MoveNext()

thanks

Steps to Reproduce:
1. Start MP2 + Service (or PC)
2. Wait for some minutes
3. See that the Service crashed, MP2 GUI still running
 
Last edited:

Brownard

Development Group
  • Team MediaPortal
  • March 21, 2007
    1,946
    1,298
    United Kingdom United Kingdom
    Country flag
    Here is the ServerError.log excerpt:
    As @HTPCSourcer states, ideally we need a set of debug logs showing the issue. That being said, can you try the attached MediaPortal.Common.dll and see if it fixes it.
    1. Stop the server
    2. Backup the existing MediaportalPortal.Common.dll in C:\Program Files (x86)\Team MediaPortal\MP2-Server\
    3. Extract and replace with the version in the attached zip
    4. Restart the server and try and reproduce
     

    Attachments

    Terra

    Portal Pro
    August 25, 2009
    230
    56
    Austria Austria
    Country flag
    Attached are new logs with DEBUG enabled and a screenshot of the task manager during the import.
    I noticed that after I enabled verbose logging it took longer for the MP2 service to crash than in the past.

    Bildschirmfoto 2018-12-09 um 10.52.27.png


    As @HTPCSourcer states, ideally we need a set of debug logs showing the issue. That being said, can you try the attached MediaPortal.Common.dll and see if it fixes it.
    I will try this now, the logs attached are with the default 2.2 dll.
     

    Terra

    Portal Pro
    August 25, 2009
    230
    56
    Austria Austria
    Country flag
    So I tried to restart the importer again with the new DLL you provided @Brownard

    Good news, the import seems to have completed now.
    Bad news MP2 service crashed then later on. I don't know the exact time when it crashed as I left the PC unattended, but when I returned just now it was not running anymore, could not find a hint in the logs myself at what time it crashed.

    In the MP2 Server Error Logs I see a lot of:

    "OCVVideoThumbnailer: Exception reading resource '{03dd2da6-4da8-4d3e-9e55-80e3165729a3}:////*****.mkv' (Text: 'u != 0')
    OpenCvSharp.OpenCVException: u != 0"

    What is that?
     
    Last edited:

    Brownard

    Development Group
  • Team MediaPortal
  • March 21, 2007
    1,946
    1,298
    United Kingdom United Kingdom
    Country flag
    Good news, the import seems to have completed now.
    It looks like some imports were still running when the server stopped, but as you say, no indication as to what stopped it.

    All 15 of your shares are being scanned when the server first starts because automatic share refresh is enabled (Settings->Media Sources->Automatic share refresh). The server tries to import all 15 concurrently, and it looks like there are quite a few new files to process, which leads to the high memory usage. Ideally we need to queue the share refreshes to limit the number that can run concurrently to avoid cases like this.
    "OCVVideoThumbnailer: Exception reading resource '{03dd2da6-4da8-4d3e-9e55-80e3165729a3}:////*****.mkv' (Text: 'u != 0')
    OpenCvSharp.OpenCVException: u != 0"

    What is that?
    It's a library we use to grab a frame from a video file to use as a thumbnail image. It seems to have issues reading some of your videos which may be contributing to the high memory usage.
     

    Terra

    Portal Pro
    August 25, 2009
    230
    56
    Austria Austria
    Country flag
    Today it crashed again with the new DLL and I see an out of memory exception in the logs attached
     

    Brownard

    Development Group
  • Team MediaPortal
  • March 21, 2007
    1,946
    1,298
    United Kingdom United Kingdom
    Country flag
    Today it crashed again with the new DLL and I see an out of memory exception in the logs attached
    Thanks, I think I've spotted where a lot of the memory usage is. Can you try the attached, you'll need to:
    • Stop the server
    • Backup and replace MediaPortal.Common and MediaPortal.Backend in C:\Program Files (x86)\Team MediaPortal\MP2-Server
    • Backup and replace the FanArtService.* plugins in C:\Program Files (x86)\Team MediaPortal\MP2-Server\Plugins
    • Restart the server
     

    Attachments

    Terra

    Portal Pro
    August 25, 2009
    230
    56
    Austria Austria
    Country flag
    It ran quite long but then the service crashed again. This time I do not see any out of memory error in the logs, the service was just gone
     

    morpheus_xx

    Lead Dev MP2
  • Team MediaPortal
  • March 24, 2007
    11,390
    6,966
    Germany Germany
    Country flag
    @Terra can you test attached plugin?
    1. Stop MP2-Service
    2. Extract attached archive to to C:\Program Files (x86)\Team MediaPortal\MP2-Server\Plugins
    3. Remove "OCVVideoThumbnailer" from this folder (move to another folder!)
    I'd like to test if the external thumb creation by ffmpeg.exe works better for your case than the OCV version.
     

    Attachments

    Users Who Are Viewing This Thread (Users: 0, Guests: 1)

    OP Similar threads Forum Replies Date
    Lichloathe MP2 importing keyart instead of poster General 0
    J [fixed] [MP2-833] 2.2.2 server crash during import Submit: Bug Reports 50
    MausWiesel LightsOut-Connector für MP2? Allgemeines Support- und Diskussionsforum 9
    M [solved] MP2 Not discovering all movies Newcomers Forum 2
    J [fixed] [MP2-821] Importer glitches in 2.2 Submit: Bug Reports 42
    N [Finished] [MP2-791] Radio recordings do not get imported into Recordings listing MediaPortal 2 25
    raffe [solved] Export tool and import support from MP-TVSeries & Moving Pictures to MP2 General 22
    H MP2 database won't keep in synch with recordings General 3
    H [solved] MP2 Wiedergabe von Aufnahmen und Live TV geht nicht Allgemeines Support- und Diskussionsforum 6
    ge2301 [WiP] MP2Web - A platform independent web client for MP2 MediaPortal 2 58
    F Problems with new install of MP2 Newcomers Forum 3
    catalpa Bug MP2-837 Allgemeines Support- und Diskussionsforum 2
    P MP2: US user, OTA - can NOT get SchedulesDirect plugin to work on MP1 or MP2 --> how do I get OTA schedule? General 5
    bigboss97 Does anyone has a SoundGraph imon remote file for MP2 to share? General 0
    ge2301 Android based client for MP2 MediaPortal 2 3
    R MP2 GUI Issues General 7
    P [solved] MP2 new install, 2 x hdhomerun tuners, client on same PC unable to view TV & recordings Newcomers Forum 8
    Lichloathe Trakt for MP2 question General 7
    C Streaming/transcoding MP2 vs MP1? General 1
    P MP 2 - Request to add an additional TV series Filename protocal Feature Requests 0
    C Streaming to clients over Wifi. MP1 or MP2? Newcomers Forum 0
    CyberSimian MP2 synchronising of stoptime MediaPortal 1 Talk 5
    M Initial Setup MP2 (On Wndows 10) General 2
    R [solved] MP2 Client ist extrem langsam Allgemeines Support- und Diskussionsforum 5
    D [solved] EPG Buddy for MP2 - Does it work? General 5
    Similar threads

























    Top Bottom