Importer works super slow (1 Viewer)

Crix1990

MP Donator
  • Premium Supporter
  • February 9, 2011
    421
    23
    34
    Home Country
    Germany Germany
    Hi everyone,

    since some time, I have the issue that the importer (the internal while Mediaportal is running) runs super slow.
    I mean the one within the config menu already takes a few minutes (ok, I have a huge series collection on my NAS), but the internal takes (felled) half an hour.
    (Main thing is, the internal importer takes much too long to be usable.)

    Can someone please take a look on the log-files to figure out, whats going on there?

    Thank you so much!
     

    Attachments

    • MediaPortalLogs_21_03_20__15_02.zip
      534.1 KB

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,751
    7,196
    Sydney
    Home Country
    Australia Australia
    Looking at the logs the delay seems to be around how the plugin processes updates:

    17mins
    Code:
    2020-03-21 14:07:19.521 [DEBG][61]: <?xml version="1.0" encoding="UTF-8"?>
    <Data time="1584751431">...
    
    2020-03-21 14:24:32.050 [INFO][61]: Series with Updates: 4381
    2020-03-21 14:24:32.053 [INFO][61]: Episodes with Updates: 611
    2020-03-21 14:24:32.055 [INFO][61]: Series with Updated Fanart: 4381
    2020-03-21 14:24:32.057 [INFO][61]: Series with Updated Banners: 4381

    3mins
    Code:
    2020-03-21 14:26:14.779 [DEBG][59]: <?xml version="1.0" encoding="UTF-8"?>
    <Data time="1584759099">...
    
    2020-03-21 14:29:52.783 [INFO][59]: Series with Updates: 918
    2020-03-21 14:29:52.786 [INFO][59]: Episodes with Updates: 188
    2020-03-21 14:29:52.788 [INFO][59]: Series with Updated Fanart: 918
    2020-03-21 14:29:52.791 [INFO][59]: Series with Updated Banners: 918

    Obviously a correlation between number of series found to update online...I will need to review currently logic and see what kind of optimizations can be made.

    Thanks for reporting.
     

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,751
    7,196
    Sydney
    Home Country
    Australia Australia
    Didn't notice same issue on my system:

    1 sec
    Code:
    2020-03-22 10:59:21.557 [DEBG][11]: <?xml version="1.0" encoding="UTF-8"?>
    <Data time="1584754926">...
    
    2020-03-22 10:59:22.276 [INFO][11]: Series with Updates: 2292
    2020-03-22 10:59:22.280 [INFO][11]: Episodes with Updates: 139
    2020-03-22 10:59:22.286 [INFO][11]: Series with Updated Fanart: 2292
    2020-03-22 10:59:22.290 [INFO][11]: Series with Updated Banners: 2292=
     

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,751
    7,196
    Sydney
    Home Country
    Australia Australia
    @Crix1990, I have attached one possible optimisation which may help you. Please test over the next coulple days with any logs to see if it has helped.

    Basically I filter out all series of no interest first before attempting to make any further queries.
     

    Attachments

    • MPTVSeries-4.6.3.1.zip
      957.4 KB

    Crix1990

    MP Donator
  • Premium Supporter
  • February 9, 2011
    421
    23
    34
    Home Country
    Germany Germany
    Works perfect now, thanks!
    Less than two minutes processing time!

    I created another log, just in case you want to check the result.

    Last question:
    If I have a series that matches with the online DB, but a the online DB does not have all episodes, how can I make the plugin creating these episodes (as a dummy or so).
    For example, the DB has 10 Episodes, but there was a 11th special, that has no match.
    How can I make this displayed?

    I remember that there was a way years ago.
     

    Attachments

    • MediaPortalLogs_22_03_20__14_22.zip
      472.3 KB

    ltfearme

    Community Plugin Dev
  • Premium Supporter
  • June 10, 2007
    6,751
    7,196
    Sydney
    Home Country
    Australia Australia
    Can you try disabling the following setting:
    1584915708389.png


    This will remove anything in your database after an import if not found online. Technically it's a bug and should only apply to episodes that are not also local.
     

    Chubler

    Portal Member
    July 14, 2009
    46
    25
    Home Country
    Australia Australia
    Any chance this fix can be pushed to the released version. I'm suffering from the same issue the OP reported and yes the posted fix makes a big difference in the import performance. However with then new file the config launcher fails to start with the this error:

    Config_error.png
     

    Users who are viewing this thread

    Top Bottom