Need advice on writing plugin that modifies EPG database (1 Viewer)

Markos

Portal Member
September 7, 2008
11
2
Czech Republic Czech Republic
Hi,

I know this post is very long, so I have highlighted some parts of the text as a TL;DR version.

I have been using MP 1 since beta. Originally I had an issue with basically useless EPG data that was grabbed from DVB-T. The providers were not sending really useful information, e.g. no season and episode number, yet the title was for example "Friends III (3)". This prevented me to use periodic recordings in MP, so I have written small utility that connected to SQL, parsed the titles to episode info and updated the database directly. I was running this task once a day using a scheduler in windows and it worked fine.

Recently I have moved to MP2. I have noticed that it uses SQLite for the database, which basically breaks my tools. I'd like to rewrite it and I have some ideas and I'd like to hear your opinion.

Modify the old tool
The easiest way would be to reuse the existing tool and modify the DB directly, however this is quite tricky with SQLite. Perhaps stopping TV Service, running the task and starting it again would work.
The original SQL script was really fast, it processed the whole database in 1 second and was not memory hungry as it always had only 1 program in memory.
I'd like to keep this as a backup plan.

Write TV Server plugin
A plugin that updates the database using the TvDatabase assembly.
The procedure would be:
  1. Get all programs using Program.ListAll()
  2. Parse each program's title and update episode info
  3. Call Program.Persist() to save each program to database one by one
  4. (optional) Somehow hook on "EPG grabbing finished" event, it such even exists, so I can modify the data as soon as possible.
This way of writing a plugin looks like a way to go, however I think there are major drawbacks:
  1. Program.ListAll() takes everything in memory at once - there does not seem to be any way to stream the data as needed. I am worried about the plugin taking too much memory during it's maintenance.
  2. I have read in some thread that saving too many things to the DB is really slow, in terms of minutes or even hours. Perhaps I misunderstood the other thread, can somebody prove me wrong?
Tweak TvDatabase.Program source code
Download the sources of TvDatabase assembly and edit the "Program.Title" property setter to do the parsing in-place. Then build this assembly and replace the original one.

This solution actually looks really elegant, easily implemented, without any performance overheads, and each program would be modified immediately - no need to wait for the once-per-day task to run.

Do you see any disadvantage of this solution?

Tweak EPG grabber direclty
Perhaps changing how the grabber parses the data and saves it to DB is the way to go. I have not investigated how that might work or if it is overkill for this task. It is just another idea on top of my head.


Perhaps you have another idea what could I do?
 

Edalex

Community Plugin Dev
  • Premium Supporter
  • January 3, 2008
    2,922
    1,228
    Saratov
    Russian Federation Russian Federation
    Country flag
    I'm not a MP2 guy but are you sure it uses sqlite for TVServer since it's the same as in MP1?
    In your case I'd write a grabber from source which supports extended info.
    If you still want to do post-proccessing on your db every dev will say that hooking in db directly by sql is "bad design" but who knows :D
    Getting programs with TVBusinessLayer is much better than direct sql but you need to do it wise. My tvserver plugin grabbing epg from site is probably inserting programs not in a good way and I'm receiving "Dupicate program" error message for every program which changes tiem for insertion from 20-30 minutes to 5-20 hours. :D
    Download the sources of TvDatabase assembly and edit the "Program.Title" property setter to do the parsing in-place. Then build this assembly and replace the original one.
    Honestl I think in some way TVDatabase should have interface for sending program to outside for post-processing by default so if you'll code such way it could be nice addition to TVServer (y)
    Disclaimer: That was just an opinion not professional thoughts:censored:
     

    Jasmeet_181

    Portal Pro
    September 5, 2007
    399
    324
    United Kingdom United Kingdom
    Country flag
    EPG Collector can grab the DVB EPG and save it as an XMLTV file, which both TVServer and many other TV backends can import. EPG Collector can also extract episode and season info from titles and descriptions for OpenTV (maybe MHEG5 too) and place them into the episode num tag. Perhaps you could contact Steve Bickell to see if you could extend that feature to the DVB EPG.
     

    Users Who Are Viewing This Thread (Users: 0, Guests: 1)

    OP Similar threads Forum Replies Date
    S EPG timeshift advice needed please Electronic Program Guide 3
    X [solved] Display needs to be turned on for TV services to function General 1
    J IPTV Sweden, need help with m3u list MediaPortal 1 Plugins 14
    N Play a TV recording needs a double selection - twice ok on remote Television (MyTV frontend and TV-Server) 2
    L [solved] Why does Mediaportal 2 need access to my webcam? General 2
    U [solved] Beginner needs help recording live TV General 19
    mrmojo666 beginner: trying to write a plugin, need help to catch event General Development (no feature request here!) 3
    M What does MP homeserver detection need at the client end, and what does the server need to transmit? General 6
    el Filou Is Windows Media Player still needed on Windows 10 ? MediaPortal 1 Talk 0
    milagroso Remote control with EventGhost - I'm stuck, need help General 3
    W Need help getting my TV guide working again General Support 1
    morpheus_xx [Evaluate] LIVE555 library vulnerability - do we need to update?! Developer Discussions 7
    A I need help with 110 and 119 west Installation, configuration support 21
    A Need help rescaning Channels for 110 and 119 west MediaPortal 1 Talk 2
    C EPG-Buddy: so close to no help needed Electronic Program Guide 8
    A Need new 110w and 119w tp`s Newcomers Forum 0
    S [solved] Alright, I need help with BD ISO's Newcomers Forum 19
    O [Question] Advice Wanted on Developing a Client Application for a Disabled Family Member MPExtended 7
    Bob Belcher Downgrade From MP2 to MP1 Advice Installation, configuration support 6
    lacanox XMLTV not writing the tv schedule Electronic Program Guide 3
    Top Bottom