Importer works super slow (2 Viewers)

Crix1990

MP Donator
  • Premium Supporter
  • February 9, 2011
    421
    23
    31
    Germany Germany
    Country flag
    Hi everyone,

    since some time, I have the issue that the importer (the internal while Mediaportal is running) runs super slow.
    I mean the one within the config menu already takes a few minutes (ok, I have a huge series collection on my NAS), but the internal takes (felled) half an hour.
    (Main thing is, the internal importer takes much too long to be usable.)

    Can someone please take a look on the log-files to figure out, whats going on there?

    Thank you so much!
     

    Attachments

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,725
    7,120
    Sydney
    Australia Australia
    Country flag
    Looking at the logs the delay seems to be around how the plugin processes updates:

    17mins
    Code:
    2020-03-21 14:07:19.521 [DEBG][61]: <?xml version="1.0" encoding="UTF-8"?>
    <Data time="1584751431">...
    
    2020-03-21 14:24:32.050 [INFO][61]: Series with Updates: 4381
    2020-03-21 14:24:32.053 [INFO][61]: Episodes with Updates: 611
    2020-03-21 14:24:32.055 [INFO][61]: Series with Updated Fanart: 4381
    2020-03-21 14:24:32.057 [INFO][61]: Series with Updated Banners: 4381
    3mins
    Code:
    2020-03-21 14:26:14.779 [DEBG][59]: <?xml version="1.0" encoding="UTF-8"?>
    <Data time="1584759099">...
    
    2020-03-21 14:29:52.783 [INFO][59]: Series with Updates: 918
    2020-03-21 14:29:52.786 [INFO][59]: Episodes with Updates: 188
    2020-03-21 14:29:52.788 [INFO][59]: Series with Updated Fanart: 918
    2020-03-21 14:29:52.791 [INFO][59]: Series with Updated Banners: 918
    Obviously a correlation between number of series found to update online...I will need to review currently logic and see what kind of optimizations can be made.

    Thanks for reporting.
     

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,725
    7,120
    Sydney
    Australia Australia
    Country flag
    Didn't notice same issue on my system:

    1 sec
    Code:
    2020-03-22 10:59:21.557 [DEBG][11]: <?xml version="1.0" encoding="UTF-8"?>
    <Data time="1584754926">...
    
    2020-03-22 10:59:22.276 [INFO][11]: Series with Updates: 2292
    2020-03-22 10:59:22.280 [INFO][11]: Episodes with Updates: 139
    2020-03-22 10:59:22.286 [INFO][11]: Series with Updated Fanart: 2292
    2020-03-22 10:59:22.290 [INFO][11]: Series with Updated Banners: 2292=
     

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,725
    7,120
    Sydney
    Australia Australia
    Country flag
    @Crix1990, I have attached one possible optimisation which may help you. Please test over the next coulple days with any logs to see if it has helped.

    Basically I filter out all series of no interest first before attempting to make any further queries.
     

    Attachments

    Crix1990

    MP Donator
  • Premium Supporter
  • February 9, 2011
    421
    23
    31
    Germany Germany
    Country flag
    Works perfect now, thanks!
    Less than two minutes processing time!

    I created another log, just in case you want to check the result.

    Last question:
    If I have a series that matches with the online DB, but a the online DB does not have all episodes, how can I make the plugin creating these episodes (as a dummy or so).
    For example, the DB has 10 Episodes, but there was a 11th special, that has no match.
    How can I make this displayed?

    I remember that there was a way years ago.
     

    Attachments

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,725
    7,120
    Sydney
    Australia Australia
    Country flag
    Can you try disabling the following setting:
    1584915708389.png


    This will remove anything in your database after an import if not found online. Technically it's a bug and should only apply to episodes that are not also local.
     

    Chubler

    Portal Member
    July 14, 2009
    45
    24
    Australia Australia
    Country flag
    Any chance this fix can be pushed to the released version. I'm suffering from the same issue the OP reported and yes the posted fix makes a big difference in the import performance. However with then new file the config launcher fails to start with the this error:

    Config_error.png
     

    Users who are viewing this thread

    Similar threads
    OP Title Forum Replies Date
    thepahakurki TVDB import not working My TVSeries 3
    R [solved] Moving pictures importer wont add files from a removable drive. Is there a work around ? Moving Pictures 8
    fredco [solved] Large import not working Moving Pictures 15
    B Importing recordings and schedules from TVmosaic General 4
    D some songs on y albums don't import General 1
    M Importer splitting Season XXX into two parts. General 0
    J movie Importer isn't as good at MP1 Moving Pictures General 4
    R Deadlock in TV Server if recording while importing EPG TV-Server 10
    W MP2 - V2.3 MP2 2.3 Import von Audioquellen / Datenbankprobleme Allgemeines Support- und Diskussionsforum 26
    D [solved] Movie library import stuck on 23% General 2
    cbr954rr Clickfinder epg import wrong channels? General 0
    L TV-Series unvollständiger Import Plugins & Erweiterungen 5
    toricred Using TMM, How to Stop Importer From Going to The Movie Database for Data General 1
    Lichloathe MP2 importing keyart instead of poster General 0
    P Recordings not showing up unless re-imported and not possible to delete General 0
    C Imported media doesn't show up immediately? General 1
    X Recording import hängt Allgemeines Support- und Diskussionsforum 0
    J [fixed] Import issue with alternate movie title matching the name of another movie Submit: Bug Reports 7
    B xmltv - cannot import tvguide.xml australia Installation, configuration support 0
    B [solved] tvguide.xml not importing australian channels Electronic Program Guide 10
    J [fixed] Importer confused on specific movies with the same title, different years and different NFO's Submit: Bug Reports 3
    azzuro [fixed] [WIP MP2.3] - Audio files not imported to MP MediaPortal 2 18
    T I cannot get a basic media import-what is the best configuration? General 10
    L TV-Series kein Import Plugins & Erweiterungen 6
    J [fixed] [MP2-833] 2.2.2 server crash during import Submit: Bug Reports 50

    Similar threads

    Top Bottom