Best way to insert programs in database (1 Viewer)

Edalex

Community Plugin Dev
  • Premium Supporter
  • January 3, 2008
    2,931
    1,234
    Saratov
    Home Country
    Russian Federation Russian Federation
    Country flag
    Hi there!
    I'm currently developing TVServer plugin for grabbing EPG from one site and I wonder what way is better to insert programs.
    My data is full EPG for one week for all my channels. So it's 10-50 k records. Should I do it once for all programs or maybe better split insertion process by channels?
    Also since my data is full week EPG I have a problem with db updating. Insertion in clean db takes several seconds but the same data could take several hours to insert in full db because of tvserver's deduplication process. So I assume I should clear db before every import with layer.RemoveAllPorgrams(idChannel) but iteration thought all channels deleted all data in db in act but looks like tvserver doesn't know about it/ Shouls I use new tvbusinesslayer for inserting to exclude deduplication or what?
     

    mm1352000

    Development Group
  • Team MediaPortal
  • September 1, 2008
    21,577
    8,222
    Home Country
    New Zealand New Zealand
    Country flag
    I suggest you try to write a WebEPG grabber... and if you can't do that, generate an XMLTV file that can be imported by the XMLTV plugin.
     

    Edalex

    Community Plugin Dev
  • Premium Supporter
  • January 3, 2008
    2,931
    1,234
    Saratov
    Home Country
    Russian Federation Russian Federation
    Country flag
    I suggest you try to write a WebEPG grabber... and if you can't do that, generate an XMLTV file that can be imported by the XMLTV plugin.
    Right, XMLTV...
    I can make XMLTV Export but it will be 2 additional conversions (raw data ->xml in my plugin and xml->raw data in xmltv) in which I don't see any benefits but it will produce some problems with escaping/formatting data for xml compliance. First I thought I could use xmltv scheduler for importing my plugin's files in such case but now I realize I still need a trigger for my plugin execution.
    But since xmltv file is similar to mine (one file for all week for all channels) I'll look into its code for importing.
    WebEPG is not the option since a) it's ugly to use parsing for API b) I need to make some advanced data modofication like splitting episodes which is not possible with WebEPG.
     

    mm1352000

    Development Group
  • Team MediaPortal
  • September 1, 2008
    21,577
    8,222
    Home Country
    New Zealand New Zealand
    Country flag
    I can make XMLTV Export but it will be 2 additional conversions (raw data ->xml in my plugin and xml->raw data in xmltv) in which I don't see any benefits but it will produce some problems with escaping/formatting data for xml compliance.
    If you convert to XMLTV, it enables standard tools to be used for data enriching (adding series + episode info).

    First I thought I could use xmltv scheduler for importing my plugin's files in such case but now I realize I still need a trigger for my plugin execution.
    The PowerScheduler plugin can do the triggering.
    The XMLTV plugin will automatically import when it detects the file change.

    WebEPG is not the option...
    Did you think of improving it?
     

    Users who are viewing this thread

    Similar threads

    Thank you for performing this test, and confirming @joecrow 's suspicion as to the cause of the problem. In principle the same problem exists with other EPG sources, such as the broadcast EPG that I use with DVB-T/T2. When a programme first appears in the EPG (7 days ahead of its broadcast time), some programmes have a generic...
    Thank you for performing this test, and confirming @joecrow 's suspicion as to the cause of the problem. In principle the same...
    TV Server 1.25. Windows 10. This has been an issue for many years, if not since inception, and I hoped it would be resolved...
    Replies
    10
    Views
    1K
    Sorry for the late response. In my experience, the EPG grabbing of a single transponder takes 2 to 3 minutes. Afterwards it takes 1 to 2 minutes to write the data to disk. To speed up the EPG stuff (not only writing but also reading the EPG guide in MediaPortal), I changed the program table (and only that table) from InnoDB to...
    Sorry for the late response. In my experience, the EPG grabbing of a single transponder takes 2 to 3 minutes. Afterwards it takes...
    I get a few "Continuity error..." after the EPG scan finished: [2021-01-30 18:56:10,278] [4c45530] [18c8] - mhw reset [2021-01-30...
    Replies
    2
    Views
    1K
    Ok, the mc2xml using the MC version is broken. I found this for working with zap2it using a different parsing tool zap2xml xmltv install / configure for ATSC / OTA U.S.A Maybe this will help the next WMC user to get the guide to work. I bet if I went with my gut to go with TitanTV instead of trying to replicate the WMC guide, I...
    Ok, the mc2xml using the MC version is broken. I found this for working with zap2it using a different parsing tool zap2xml xmltv...
    Hello, Looking to move my Win7 Ultimate WMC box to a Win10Pro OS and started here with MP2. Got the live TV, OTA ATSC tuning...
    Replies
    8
    Views
    2K
    EPG Collector can grab the DVB EPG and save it as an XMLTV file, which both TVServer and many other TV backends can import. EPG Collector can also extract episode and season info from titles and descriptions for OpenTV (maybe MHEG5 too) and place them into the episode num tag. Perhaps you could contact Steve Bickell to see if you...
    EPG Collector can grab the DVB EPG and save it as an XMLTV file, which both TVServer and many other TV backends can import. EPG...
    Hi, I know this post is very long, so I have highlighted some parts of the text as a TL;DR version. I have been using MP 1 since...
    Replies
    2
    Views
    2K
    Hello, I would like to share this as a suggestion for configuring DVB EPG. In TV server, configure under DVB EPG: + always try to fill holes = on + always try to update existing entries = on + enable CRC check = off + grab EPG only for channels on same transponder = on Then use some SQL client to connect to the database, and...
    Hello, I would like to share this as a suggestion for configuring DVB EPG. In TV server, configure under DVB EPG: + always try to...
    Hello, I would like to share this as a suggestion for configuring DVB EPG. In TV server, configure under DVB EPG: + always try to...
    Replies
    0
    Views
    929
    Top Bottom