Is there any way to speed this import up? (1 Viewer)

ltfearme

Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,687
    7,062
    Sydney
    Australia Australia
    Country flag
    Hey @Lyfesaver74,

    been a bit busy this week but I added some more logging where it looks to be spending most of the time for you. Feel free to try it out. @RoChess, you can also try it out as well if you want to compare...it may shed some clues.

    When I know exactly where the time is being taken I will try find some time to optimise it...some of this code is very old and has a high risk of regression if I go and change it :(
     

    Attachments

    Lyfesaver74

    Public Relations
  • Premium Supporter
  • September 25, 2010
    1,544
    1,122
    Texas
    United States of America United States of America
    Country flag
    2016-03-18 09:32:08.956 [INFO][04]: Starting Local Filename Parsing, processing 21271 files
    2016-03-18 09:32:20.640 [INFO][04]: Finished Local Filename Parsing
    2016-03-18 09:32:20.644 [INFO][04]: Executing Query: select episodefilename from local_episodes
    2016-03-18 09:32:20.729 [INFO][04]: Finished Query
    2016-03-18 09:32:20.733 [INFO][04]: Creating list to store eps from query
    2016-03-18 09:32:20.738 [INFO][04]: Finished adding episodes to list
    2016-03-18 09:32:20.742 [INFO][04]: Removing episodes from haystack that already exist in db, haystack count = 21271
    2016-03-18 09:32:29.373 [INFO][04]: Finished haystack processing, Haystack Count = 0, UpdateStatusEps = 21271
    2016-03-18 09:32:29.377 [INFO][04]: Start UpdateStatus(updateStatusEps)
    2016-03-18 09:32:29.383 [INFO][04]: Creating condition builder
    2016-03-18 09:32:29.426 [INFO][04]: Finished creating condition builder. SQL Conditions Length = 29064, Processed Conditions = 43
    2016-03-18 09:32:29.429 [INFO][04]: Applying Episode cImportProcessed/cIsAvailable to DB
    2016-03-18 09:32:31.978 [INFO][04]: Finished applying Episode cImportProcessed/cIsAvailable to DB
    2016-03-18 09:32:31.982 [INFO][04]: Applying Season cHasLocalFilesTemp to DB, this may take forever!
    2016-03-18 09:33:36.470 [INFO][04]: Finished applying Season cHasLocalFilesTemp to DB
    2016-03-18 09:33:36.474 [INFO][04]: Applying Series cHasLocalFilesTemp to DB, probably will take a long time.
    2016-03-18 09:33:53.243 [INFO][04]: Finished applying Series cHasLocalFilesTemp to DB
    2016-03-18 09:33:53.247 [INFO][04]: Finish UpdateStatus(updateStatusEps)
    2016-03-18 09:33:53.252 [INFO][04]: Adding 0 new file(s) to Database
    2016-03-18 09:33:58.399 [DEBG][04]: Begin Parsing action: MediaInfo
    2016-03-18 09:33:58.763 [INFO][04]: All episodes already contain MediaInfo
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,321
    1,812
    Country flag
    This new version keeps removing references on episodes that are not correct (which I know), but I can not disable the checkbox to turn it off.

    2016-03-18 17:07:31.217 [INFO][04]: *************** Gathering Local Information ***************
    2016-03-18 17:07:31.217 [INFO][04]: Starting Local Filename Parsing, processing 19506 files
    2016-03-18 17:07:40.545 [INFO][04]: Finished Local Filename Parsing
    2016-03-18 17:07:40.545 [INFO][04]: Executing Query: select episodefilename from local_episodes
    2016-03-18 17:07:40.686 [INFO][04]: Finished Query
    2016-03-18 17:07:40.686 [INFO][04]: Creating list to store eps from query
    2016-03-18 17:07:40.686 [INFO][04]: Finished adding episodes to list
    2016-03-18 17:07:40.686 [INFO][04]: Removing episodes from haystack that already exist in db, haystack count = 19506
    2016-03-18 17:07:52.389 [INFO][04]: Finished haystack processing, Haystack Count = 46, UpdateStatusEps = 19460
    2016-03-18 17:07:52.389 [INFO][04]: Start UpdateStatus(updateStatusEps)
    2016-03-18 17:07:52.404 [INFO][04]: Creating condition builder
    2016-03-18 17:07:52.420 [INFO][04]: Finished creating condition builder. SQL Conditions Length = 70499, Processed Conditions = 39
    2016-03-18 17:07:52.436 [INFO][04]: Applying Episode cImportProcessed/cIsAvailable to DB
    2016-03-18 17:07:53.748 [INFO][04]: Finished applying Episode cImportProcessed/cIsAvailable to DB
    2016-03-18 17:07:53.748 [INFO][04]: Applying Season cHasLocalFilesTemp to DB, this may take forever!
    2016-03-18 17:07:53.967 [INFO][04]: Finished applying Season cHasLocalFilesTemp to DB
    2016-03-18 17:07:53.983 [INFO][04]: Applying Series cHasLocalFilesTemp to DB, probably will take a long time.
    2016-03-18 17:07:54.217 [INFO][04]: Finished applying Series cHasLocalFilesTemp to DB
    2016-03-18 17:07:54.217 [INFO][04]: Finish UpdateStatus(updateStatusEps)
    2016-03-18 17:07:54.217 [INFO][04]: Adding 46 new file(s) to Database
    ...
    2016-03-18 17:08:47.041 [INFO][01]: Online Parsing Completed in 00:01:17.6989661
    2016-03-18 17:09:11.557 [INFO][02]: Cleaning up cached, generated Logos
     

    Users Who Are Viewing This Thread (Users: 0, Guests: 1)

    OP Similar threads Forum Replies Date
    FlatEric [solved] 5016: Same position for title and speed-info when forwarding watch/edit Videos 3
    J Clock speed vs multi-core CPU's Can MediaPortal benefit from multi core cpu's? General 0
    Lichloathe MP2 importing keyart instead of poster General 0
    P Recordings not showing up unless re-imported and not possible to delete General 0
    C Importer works super slow My TVSeries 5
    C Imported media doesn't show up immediately? General 1
    X Recording import hängt Allgemeines Support- und Diskussionsforum 0
    J [fixed] Import issue with alternate movie title matching the name of another movie Submit: Bug Reports 7
    B xmltv - cannot import tvguide.xml australia Installation, configuration support 0
    B [solved] tvguide.xml not importing australian channels Electronic Program Guide 10
    J [fixed] Importer confused on specific movies with the same title, different years and different NFO's Submit: Bug Reports 3
    azzuro [fixed] [WIP MP2.3] - Audio files not imported to MP MediaPortal 2 18
    thepahakurki TVDB import not working My TVSeries 3
    T I cannot get a basic media import-what is the best configuration? General 10
    L TV-Series kein Import Plugins & Erweiterungen 6
    J [fixed] [MP2-833] 2.2.2 server crash during import Submit: Bug Reports 50
    D Ripped movies not importing unless IMDB code is included in folder name General 4
    R [solved] Moving pictures importer wont add files from a removable drive. Is there a work around ? Moving Pictures 8
    MausWiesel LightsOut-Connector für MP2? Allgemeines Support- und Diskussionsforum 9
    fredco [solved] Large import not working Moving Pictures 15
    M [solved] MP2 Not discovering all movies Newcomers Forum 2
    J [fixed] [MP2-821] Importer glitches in 2.2 Submit: Bug Reports 42
    Golf4 [solved] Bei Import und grabben der Medieninhalte - Client ausschalten? Allgemeines Support- und Diskussionsforum 1
    cbr954rr Possibillity to export and import all client settings? General 0
    Jean-Christophe MP 2.2 - Where can I find Movies NOT imported (and why??) ? General 13
    Similar threads

























    Top Bottom