Need advice on writing plugin that modifies EPG database (1 Viewer)

[Markos]

Hi,

I know this post is very long, so I have highlighted some parts of the text as a TL;DR version.

I have been using MP 1 since beta. Originally I had an issue with basically useless EPG data that was grabbed from DVB-T. The providers were not sending really useful information, e.g. no season and episode number, yet the title was for example "Friends III (3)". This prevented me to use periodic recordings in MP, so I have written small utility that connected to SQL, parsed the titles to episode info and updated the database directly. I was running this task once a day using a scheduler in windows and it worked fine.

Recently I have moved to MP2. I have noticed that it uses SQLite for the database, which basically breaks my tools. I'd like to rewrite it and I have some ideas and I'd like to hear your opinion.

Modify the old tool
The easiest way would be to reuse the existing tool and modify the DB directly, however this is quite tricky with SQLite. Perhaps stopping TV Service, running the task and starting it again would work.
The original SQL script was really fast, it processed the whole database in 1 second and was not memory hungry as it always had only 1 program in memory.
I'd like to keep this as a backup plan.

Write TV Server plugin
A plugin that updates the database using the TvDatabase assembly.
The procedure would be:

1. Get all programs using Program.ListAll()
   
2. Parse each program's title and update episode info
3. Call Program.Persist() to save each program to database one by one
4. (optional) Somehow hook on "EPG grabbing finished" event, it such even exists, so I can modify the data as soon as possible.

This way of writing a plugin looks like a way to go, however I think there are major drawbacks:

1. Program.ListAll() takes everything in memory at once - there does not seem to be any way to stream the data as needed. I am worried about the plugin taking too much memory during it's maintenance.
   
2. I have read in some thread that saving too many things to the DB is really slow, in terms of minutes or even hours. Perhaps I misunderstood the other thread, can somebody prove me wrong?

Tweak TvDatabase.Program source code
Download the sources of TvDatabase assembly and edit the "Program.Title" property setter to do the parsing in-place. Then build this assembly and replace the original one.

This solution actually looks really elegant, easily implemented, without any performance overheads, and each program would be modified immediately - no need to wait for the once-per-day task to run.

Do you see any disadvantage of this solution?

Tweak EPG grabber direclty
Perhaps changing how the grabber parses the data and saves it to DB is the way to go. I have not investigated how that might work or if it is overkill for this task. It is just another idea on top of my head.


Perhaps you have another idea what could I do?

 

[Edalex]

I'm not a MP2 guy but are you sure it uses sqlite for TVServer since it's the same as in MP1?
In your case I'd write a grabber from source which supports extended info.
If you still want to do post-proccessing on your db every dev will say that hooking in db directly by sql is "bad design" but who knows
Getting programs with TVBusinessLayer is much better than direct sql but you need to do it wise. My tvserver plugin grabbing epg from site is probably inserting programs not in a good way and I'm receiving "Dupicate program" error message for every program which changes tiem for insertion from 20-30 minutes to 5-20 hours.

Download the sources of TvDatabase assembly and edit the "Program.Title" property setter to do the parsing in-place. Then build this assembly and replace the original one.

Honestl I think in some way TVDatabase should have interface for sending program to outside for post-processing by default so if you'll code such way it could be nice addition to TVServer
Disclaimer: That was just an opinion not professional thoughts

 

[Jasmeet_181]

EPG Collector can grab the DVB EPG and save it as an XMLTV file, which both TVServer and many other TV backends can import. EPG Collector can also extract episode and season info from titles and descriptions for OpenTV (maybe MHEG5 too) and place them into the episode num tag. Perhaps you could contact Steve Bickell to see if you could extend that feature to the DVB EPG.