How Long To Import EPG (1 Viewer)

EViS

Portal Pro
September 30, 2006
364
2
Home Country
United Kingdom United Kingdom
Having 'imported' 1000 programmes for 60 channels, how long should this data take to appear in the MP EPG?
 

Paranoid Delusion

Moderation Manager
  • Premium Supporter
  • June 13, 2005
    13,062
    2,978
    Cheshire
    Home Country
    United Kingdom United Kingdom
    My 14 day xml file takes around 3-4 minutes for 47 stations to update, but then a further 10 minutes of processing is done by ForTheRecord on series\new\missed recordings, you can see the serious cpu activity between FTR and MySQL.
     

    EViS

    Portal Pro
    September 30, 2006
    364
    2
    Home Country
    United Kingdom United Kingdom
    My data is saved to the MP database as opposed to an xml file. This then takes around 45min to 1 hour to import into the MP TV Guide gui! Obviously the problem only ever occurs when I've wiped all the channels for a new rescan or after a completely fresh restall.

    Could the problem be that I shouldn't be writing data to the database? Is an xml file the recommended method? It just seems nice to have this data in the database as opposed to the 'old school' method :).
     

    arion_p

    Retired Team Member
  • Premium Supporter
  • February 7, 2007
    3,373
    1,626
    Athens
    Home Country
    Greece Greece
    Eventually the data has to get to the database. If you use an xml file you just add another (unnecessary) step.

    The delay is most probably due to slow grabbing, not slow inserting to the database. You can verify this by temporarily switching to xmltv output. If it takes equally long to write the data to xml file, it is obviously an issue with the source sites/grabbers

    Note that some grabbers introduce an intentional delay between page requests (because the sites involved ban IPs that make a lot of requests in rapid succession). This delay setting was in some cases not honoured by WebEPG some versions back.
     

    DragonQ

    Portal Pro
    August 30, 2011
    644
    79
    Home Country
    United Kingdom United Kingdom
    Grabbing takes ages for me. Using MyDigiGuide, I have to use a "delay" of 2000 ms because the default of 1000 ms leaves loads of gaps in the EPG. However, since I have 210 channels and each day is a separate URL, this means each channel takes at least 20 seconds (for 10 days). That's 70 minutes. Surely there must be a way to speed this up? Is the delay broken somehow? Does MyDigiGuide reject frequent HTTP requests? Is there a page at MyDigiGuide that has all information for 10 days rather than going to a separate URL for each day?
     

    arion_p

    Retired Team Member
  • Premium Supporter
  • February 7, 2007
    3,373
    1,626
    Athens
    Home Country
    Greece Greece
    There is no way around this. I am not familiar with MyDigiGuide, but obviously (since you have gaps if you lower the delay) it rejects requests from a single IP if they arrive too faster than a preset value. This is commonly used by web sites to stop bots from scraping their content.
     
    Last edited:

    Users who are viewing this thread

    Similar threads

    MP1 MP2 [solved] EPG windows DE
    So, with your tips I've deep looking on the config program. And I've found the mapping for HID device. I've make the key config for up and down keys. And it's Work! Really big thanks to you and the team that give this great mediacenter. Cheers ;)
    So, with your tips I've deep looking on the config program. And I've found the mapping for HID device. I've make the key config for...
    Hello, How to scroll the line in the epg windows with titan extended skin? I see the firts 11 channel lines but I cannot get the...
    Replies
    3
    Views
    372
    Thanks for that. I found "Chnlineup" in the Sky grab and "FreesatEPG" in the Freesat grab and enabled them both in EPG Collector, and also set them as the EPG grabber for radio in MP2. But here's the odd thing: The next time I opened the XMLTV EPG mapper (where you map actual channels to channels available in the EPG), the radio...
    Thanks for that. I found "Chnlineup" in the Sky grab and "FreesatEPG" in the Freesat grab and enabled them both in EPG Collector...
    Following on from this thread: Previously I used EPG Collector very successfully to grab a 7 day OTA EPG which included the EPG...
    Replies
    4
    Views
    992
    Ok some good news after much tearing out of hair lol as i refuse to be beaten at problem solving i managed to find out where EPG Collector gets its epg from and found it gets it from 'freesathome' so cribbed the settings from that and added the freesathome channel to the radio channels and selected that channel only to collect epg...
    Ok some good news after much tearing out of hair lol as i refuse to be beaten at problem solving i managed to find out where EPG...
    Hi folks Ive just started using epg collector to import uk freesat epg into mediaportal i have had good success with it so far and...
    Replies
    7
    Views
    1K
    And this is the follow-up thread, which includes log files: PiP audio problem
    And this is the follow-up thread, which includes log files: PiP audio problem
    I just updated from 2.4. Update did not conserve my channels or epg plug-in or anything, looks like, which of course was a...
    Replies
    5
    Views
    513
    I have all of my media on a NAS. I guess it might just be a network issue, then.
    I have all of my media on a NAS. I guess it might just be a network issue, then.
    Whenever I go into the back end for Moving Pictures, it almost immediately hangs on the Movie Importer tab. If I want to go into...
    Replies
    4
    Views
    368
    Top Bottom