Need advice on writing plugin that modifies EPG database

Discussion in 'TV-Server Plugins' started by Markos, March 27, 2018.

  1. Markos

    Markos Portal Member

    September 7, 2008
    Likes Received:
    +2 / 0
    Home Country:
    Czech Republic Czech Republic

    I know this post is very long, so I have highlighted some parts of the text as a TL;DR version.

    I have been using MP 1 since beta. Originally I had an issue with basically useless EPG data that was grabbed from DVB-T. The providers were not sending really useful information, e.g. no season and episode number, yet the title was for example "Friends III (3)". This prevented me to use periodic recordings in MP, so I have written small utility that connected to SQL, parsed the titles to episode info and updated the database directly. I was running this task once a day using a scheduler in windows and it worked fine.

    Recently I have moved to MP2. I have noticed that it uses SQLite for the database, which basically breaks my tools. I'd like to rewrite it and I have some ideas and I'd like to hear your opinion.

    Modify the old tool
    The easiest way would be to reuse the existing tool and modify the DB directly, however this is quite tricky with SQLite. Perhaps stopping TV Service, running the task and starting it again would work.
    The original SQL script was really fast, it processed the whole database in 1 second and was not memory hungry as it always had only 1 program in memory.
    I'd like to keep this as a backup plan.

    Write TV Server plugin
    A plugin that updates the database using the TvDatabase assembly.
    The procedure would be:
    1. Get all programs using Program.ListAll()
    2. Parse each program's title and update episode info
    3. Call Program.Persist() to save each program to database one by one
    4. (optional) Somehow hook on "EPG grabbing finished" event, it such even exists, so I can modify the data as soon as possible.
    This way of writing a plugin looks like a way to go, however I think there are major drawbacks:
    1. Program.ListAll() takes everything in memory at once - there does not seem to be any way to stream the data as needed. I am worried about the plugin taking too much memory during it's maintenance.
    2. I have read in some thread that saving too many things to the DB is really slow, in terms of minutes or even hours. Perhaps I misunderstood the other thread, can somebody prove me wrong?
    Tweak TvDatabase.Program source code
    Download the sources of TvDatabase assembly and edit the "Program.Title" property setter to do the parsing in-place. Then build this assembly and replace the original one.

    This solution actually looks really elegant, easily implemented, without any performance overheads, and each program would be modified immediately - no need to wait for the once-per-day task to run.

    Do you see any disadvantage of this solution?

    Tweak EPG grabber direclty
    Perhaps changing how the grabber parses the data and saves it to DB is the way to go. I have not investigated how that might work or if it is overkill for this task. It is just another idea on top of my head.

    Perhaps you have another idea what could I do?
  2. Google AdSense Guest Advertisement

    to hide all adverts.
  3. Edalex
    • Premium Supporter

    Edalex Community Plugin Dev

    January 3, 2008
    Likes Received:
    +1,223 / 9
    Home Country:
    Russian Federation Russian Federation
    Show System Specs
    I'm not a MP2 guy but are you sure it uses sqlite for TVServer since it's the same as in MP1?
    In your case I'd write a grabber from source which supports extended info.
    If you still want to do post-proccessing on your db every dev will say that hooking in db directly by sql is "bad design" but who knows :D
    Getting programs with TVBusinessLayer is much better than direct sql but you need to do it wise. My tvserver plugin grabbing epg from site is probably inserting programs not in a good way and I'm receiving "Dupicate program" error message for every program which changes tiem for insertion from 20-30 minutes to 5-20 hours. :D
    Honestl I think in some way TVDatabase should have interface for sending program to outside for post-processing by default so if you'll code such way it could be nice addition to TVServer (y)
    Disclaimer: That was just an opinion not professional thoughts:censored:
  4. Jasmeet_181

    Jasmeet_181 Portal Pro

    September 5, 2007
    Likes Received:
    +283 / 0
    Home Country:
    England England
    EPG Collector can grab the DVB EPG and save it as an XMLTV file, which both TVServer and many other TV backends can import. EPG Collector can also extract episode and season info from titles and descriptions for OpenTV (maybe MHEG5 too) and place them into the episode num tag. Perhaps you could contact Steve Bickell to see if you could extend that feature to the DVB EPG.

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.


  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!