Is there any way to speed this import up? (1 Viewer)

ltfearme

Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,751
    7,196
    Sydney
    Home Country
    Australia Australia
    See if this logging helps determine where issue is, if its in the RemoveFilesInDB() method then it may be possible to optimise it.
     

    Attachments

    • MP-TVSeries.rar
      491.1 KB

    Lyfesaver74

    Public Relations
  • Premium Supporter
  • September 25, 2010
    1,544
    1,122
    Texas
    Home Country
    United States of America United States of America
    I am going to try this right now, but I dont think it has anything to do with removing files... as I am a collector and don't really delete any shows.
     

    Lyfesaver74

    Public Relations
  • Premium Supporter
  • September 25, 2010
    1,544
    1,122
    Texas
    Home Country
    United States of America United States of America
    2016-03-15 11:20:10.188 [INFO][04]: Finished Local Filename Parsing
    2016-03-15 11:20:10.198 [INFO][04]: Executing Query: select episodefilename from local_episodes
    2016-03-15 11:20:10.278 [INFO][04]: Finished Query
    2016-03-15 11:20:10.288 [INFO][04]: Creating list to store eps from query
    2016-03-15 11:20:10.288 [INFO][04]: Finished adding episodes to list
    2016-03-15 11:20:10.298 [INFO][04]: Removing episodes from haystack that already exist in db, haystack count = 21214
    2016-03-15 11:20:18.668 [INFO][04]: Finished haystack processing
    2016-03-15 11:20:18.668 [INFO][04]: Start UpdateStatus(updateStatusEps)
    2016-03-15 11:21:40.718 [INFO][04]: Finish UpdateStatus(updateStatusEps)
    2016-03-15 11:21:40.718 [INFO][04]: Adding 0 new file(s) to Database
     

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,751
    7,196
    Sydney
    Home Country
    Australia Australia
    Okay, so the majority of time taken is in the UpdateStatus method, I will see whats going on in there.

    It also spends quite a bit of time (8 secs) in the haystack so will see if that can be improved as well.
     

    Lyfesaver74

    Public Relations
  • Premium Supporter
  • September 25, 2010
    1,544
    1,122
    Texas
    Home Country
    United States of America United States of America
    Has anyone told you how awesome you are lately? Tell your wife I said she needs to give you the "good lovin" tonight :)
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    @ltfearme would it help you if I test the same DLL to get you the timestamp results on my box to see if 95%+ of that time is also spend on the UpdateStatus method?
     

    Lyfesaver74

    Public Relations
  • Premium Supporter
  • September 25, 2010
    1,544
    1,122
    Texas
    Home Country
    United States of America United States of America
    I ran an import on my i5-4670K and got these results:

    2016-03-15 22:38:42.578 [INFO][07]: Starting Local Filename Parsing, processing 21222 files
    2016-03-15 22:38:51.041 [INFO][07]: Finished Local Filename Parsing
    2016-03-15 22:38:51.045 [INFO][07]: Executing Query: select episodefilename from local_episodes
    2016-03-15 22:38:51.136 [INFO][07]: Finished Query
    2016-03-15 22:38:51.140 [INFO][07]: Creating list to store eps from query
    2016-03-15 22:38:51.144 [INFO][07]: Finished adding episodes to list
    2016-03-15 22:38:51.148 [INFO][07]: Removing episodes from haystack that already exist in db, haystack count = 21222
    2016-03-15 22:38:53.102 [INFO][07]: Finished haystack processing
    2016-03-15 22:38:53.106 [INFO][07]: Start UpdateStatus(updateStatusEps)
    2016-03-15 22:38:54.781 [INFO][07]: Finish UpdateStatus(updateStatusEps)
    2016-03-15 22:38:54.785 [INFO][07]: Adding 0 new file(s) to Database
    2016-03-15 22:38:58.666 [DEBG][07]: Begin Parsing action: MediaInfo
    2016-03-15 22:38:58.953 [INFO][07]: All episodes already contain MediaInfo

    This is about on par with Ron's. While my other CPU is not as fast at the i5, it is no chump and still better than Ron's. So I am starting to believe that there is something wrong with my CPU and there is nothing much for you to debug, Damien :(
     

    kiwijunglist

    Super Moderator
  • Team MediaPortal
  • June 10, 2008
    6,746
    1,751
    New Zealand
    Home Country
    New Zealand New Zealand
    Could it be to do with having your HDDs sleeping and needing to wake up. Do you get the same import time if all the HDDs are awake prior to import?
     

    Users who are viewing this thread

    Top Bottom