Is there any way to speed this import up? (1 Viewer)

ltfearme

Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,751
    7,196
    Sydney
    Home Country
    Australia Australia
    Hey @Lyfesaver74,

    been a bit busy this week but I added some more logging where it looks to be spending most of the time for you. Feel free to try it out. @RoChess, you can also try it out as well if you want to compare...it may shed some clues.

    When I know exactly where the time is being taken I will try find some time to optimise it...some of this code is very old and has a high risk of regression if I go and change it :(
     

    Attachments

    • MP-TVSeries.rar
      488.1 KB

    Lyfesaver74

    Public Relations
  • Premium Supporter
  • September 25, 2010
    1,544
    1,122
    Texas
    Home Country
    United States of America United States of America
    2016-03-18 09:32:08.956 [INFO][04]: Starting Local Filename Parsing, processing 21271 files
    2016-03-18 09:32:20.640 [INFO][04]: Finished Local Filename Parsing
    2016-03-18 09:32:20.644 [INFO][04]: Executing Query: select episodefilename from local_episodes
    2016-03-18 09:32:20.729 [INFO][04]: Finished Query
    2016-03-18 09:32:20.733 [INFO][04]: Creating list to store eps from query
    2016-03-18 09:32:20.738 [INFO][04]: Finished adding episodes to list
    2016-03-18 09:32:20.742 [INFO][04]: Removing episodes from haystack that already exist in db, haystack count = 21271
    2016-03-18 09:32:29.373 [INFO][04]: Finished haystack processing, Haystack Count = 0, UpdateStatusEps = 21271
    2016-03-18 09:32:29.377 [INFO][04]: Start UpdateStatus(updateStatusEps)
    2016-03-18 09:32:29.383 [INFO][04]: Creating condition builder
    2016-03-18 09:32:29.426 [INFO][04]: Finished creating condition builder. SQL Conditions Length = 29064, Processed Conditions = 43
    2016-03-18 09:32:29.429 [INFO][04]: Applying Episode cImportProcessed/cIsAvailable to DB
    2016-03-18 09:32:31.978 [INFO][04]: Finished applying Episode cImportProcessed/cIsAvailable to DB
    2016-03-18 09:32:31.982 [INFO][04]: Applying Season cHasLocalFilesTemp to DB, this may take forever!
    2016-03-18 09:33:36.470 [INFO][04]: Finished applying Season cHasLocalFilesTemp to DB
    2016-03-18 09:33:36.474 [INFO][04]: Applying Series cHasLocalFilesTemp to DB, probably will take a long time.
    2016-03-18 09:33:53.243 [INFO][04]: Finished applying Series cHasLocalFilesTemp to DB
    2016-03-18 09:33:53.247 [INFO][04]: Finish UpdateStatus(updateStatusEps)
    2016-03-18 09:33:53.252 [INFO][04]: Adding 0 new file(s) to Database
    2016-03-18 09:33:58.399 [DEBG][04]: Begin Parsing action: MediaInfo
    2016-03-18 09:33:58.763 [INFO][04]: All episodes already contain MediaInfo
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    This new version keeps removing references on episodes that are not correct (which I know), but I can not disable the checkbox to turn it off.

    2016-03-18 17:07:31.217 [INFO][04]: *************** Gathering Local Information ***************
    2016-03-18 17:07:31.217 [INFO][04]: Starting Local Filename Parsing, processing 19506 files
    2016-03-18 17:07:40.545 [INFO][04]: Finished Local Filename Parsing
    2016-03-18 17:07:40.545 [INFO][04]: Executing Query: select episodefilename from local_episodes
    2016-03-18 17:07:40.686 [INFO][04]: Finished Query
    2016-03-18 17:07:40.686 [INFO][04]: Creating list to store eps from query
    2016-03-18 17:07:40.686 [INFO][04]: Finished adding episodes to list
    2016-03-18 17:07:40.686 [INFO][04]: Removing episodes from haystack that already exist in db, haystack count = 19506
    2016-03-18 17:07:52.389 [INFO][04]: Finished haystack processing, Haystack Count = 46, UpdateStatusEps = 19460
    2016-03-18 17:07:52.389 [INFO][04]: Start UpdateStatus(updateStatusEps)
    2016-03-18 17:07:52.404 [INFO][04]: Creating condition builder
    2016-03-18 17:07:52.420 [INFO][04]: Finished creating condition builder. SQL Conditions Length = 70499, Processed Conditions = 39
    2016-03-18 17:07:52.436 [INFO][04]: Applying Episode cImportProcessed/cIsAvailable to DB
    2016-03-18 17:07:53.748 [INFO][04]: Finished applying Episode cImportProcessed/cIsAvailable to DB
    2016-03-18 17:07:53.748 [INFO][04]: Applying Season cHasLocalFilesTemp to DB, this may take forever!
    2016-03-18 17:07:53.967 [INFO][04]: Finished applying Season cHasLocalFilesTemp to DB
    2016-03-18 17:07:53.983 [INFO][04]: Applying Series cHasLocalFilesTemp to DB, probably will take a long time.
    2016-03-18 17:07:54.217 [INFO][04]: Finished applying Series cHasLocalFilesTemp to DB
    2016-03-18 17:07:54.217 [INFO][04]: Finish UpdateStatus(updateStatusEps)
    2016-03-18 17:07:54.217 [INFO][04]: Adding 46 new file(s) to Database
    ...
    2016-03-18 17:08:47.041 [INFO][01]: Online Parsing Completed in 00:01:17.6989661
    2016-03-18 17:09:11.557 [INFO][02]: Cleaning up cached, generated Logos
     

    Users who are viewing this thread

    Top Bottom