Can the XmlTv Plugin handle 600+ channels?

Discussion in 'General' started by ChaosMageX, September 19, 2019.

  1. ChaosMageX

    ChaosMageX Portal Member

    Joined:
    October 23, 2015
    Messages:
    10
    Likes Received:
    0
    Gender:
    Male
    Ratings:
    +0 / 0
    Home Country:
    United States of America United States of America
    I've finally finished mapping all the channels were scanned from my HDHomeRun over-the-air and cable tuners, which amounts to over 600. I forgot to calculate the exact number because not every channel had a matching channel in the tvguide.xml file retrieved from my Schedules Direct subscription, but most of the 649 total channels I have did have a match in the EPG data.

    My question is: Can the plugin even handle writing the guide for that many channels?

    I'm asking because after I finished mapping, and then saved it and then reloaded/refreshed it to make sure it was saved, and then clicked the Import button under the General tab, the TV Setup application is now unresponsive and has been for nearly an hour now. I'm really worried, because it's even more unresponsive than apps normally are, because it doesn't even have the (Unresponsive) in its title, but it isn't responding to my clicks. I can't even move the window around the screen, as it seems to be stuck in place. Now the close and minimize buttons in the upper right corner are weirdly flickering and I've never seen an application do that before on Windows 10, or any other version of Windows for that matter. It's also running on a rather powerful computer with a NVMe SSD, an AMD Ryzen 1600 CPU, and 16 GB of RAM, so it shouldn't be freezing up like this if it's just a matter of the amount of data to process.

    EDIT: Well, it seems to have finally finished, and I can actually click stuff in the TV Setup application and move the window around the screen again. Checking the client application, it appears to have successfully imported the data for all the mapped channels. Still, why would it take that long on a SSD with a 1.5 GB/s linear read speed and a random read speed around 800 MB/s, along with a 6/12 core 3.2 GHz CPU, and 12 GB of the 16 GB of RAM available and ready to use?



    Are there logs somewhere that I can check and see why it took so long?
     
    Last edited: September 19, 2019
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. HTPCSourcer
    • Team MediaPortal
    • Administrator

    HTPCSourcer MP2 Product Manager

    Joined:
    May 16, 2008
    Messages:
    10,705
    Likes Received:
    1,118
    Gender:
    Male
    Ratings:
    +2,400 / 22
    Home Country:
    Germany Germany
    Show System Specs
    This is not a hardware thing but a limitation of the xmltv plugin, which handles every record (and possibly data field) one after the other. Every single show information is handled separately and a record is written for each single show of your 649 channels for probably 14 days, which will give you approx 649 x 14 x 24 show entries, each with ~five fields = approx 1 mio. fields to be added to the database. For every record the SQL database is opened, the information written into the different fields, etc. The process is not a mass process where all data is written at once. The xmltv plugin that comes with the TV package is probably 10 years old and has not been touched since then.

    There may also be an issue with your import settings. Best would be to have the complete EPG deleted before adding new data. The EPG read process should be visible in the server log.
     
  4. HTPCSourcer
    • Team MediaPortal
    • Administrator

    HTPCSourcer MP2 Product Manager

    Joined:
    May 16, 2008
    Messages:
    10,705
    Likes Received:
    1,118
    Gender:
    Male
    Ratings:
    +2,400 / 22
    Home Country:
    Germany Germany
    Show System Specs
  5. framug

    framug Portal Pro

    Joined:
    January 31, 2005
    Messages:
    4,180
    Likes Received:
    203
    Occupation:
    software engineer
    Location:
    South of France
    Ratings:
    +244 / 2
    Home Country:
    France France
    Same as HTPCSourcer wrote, the process of this plugin is not a mass process.
    Depending where you download your xmltv.xml file, some websites propose to have a personal file (you choose what are channels in the .xml).

    An example for french country :

    zz1.png

    It takes 2 minutes an 5 seconds for 4765 programs imported on my system :

    zz2.png

    If you have 218 000 imported programs then, it's 45 times more than I so, it would take more than 90 minutes.

    If you schedule your import when you watch TV (for example 9H00 PM) then, import could be invisible..
     
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!