[Plugin] EPG Cleaner & Schedule Repair (1 Viewer)

[hurley]

What is EPG Cleaner?

EPG Cleaner is a TV-Server plugin, which allows DVB EPG data to be cleaned or filtered as it is loaded into the TV-Guide.
Remove spam or prefixes and suffixes from program titles and description text, which make the TV-Guide difficult to read and cause problems with scheduling.

For example if your TV network appends words like 'New' or 'Repeat' or 'Premiere' to program titles in an inconsistent manner, or parental ratings like (12) or (16) which are different for each episode, you may need to create multiple schedules for your favourite series to avoid missing any episodes.
Examples like these will also result in series recordings being scattered over multiple folders in Recorded TV.

EPG Cleaner uses Regular Expressions (RegEX) to create powerful and highly flexible filters for your guide data to remove these unwanted parts.
Each filter may apply to all channels or only selected channels.

"RegEx sounds too complicated for me"...

Don't worry, EPG Cleaner has a Simple mode which uses wildcards like * and ? which most Windows users should be familiar with.​

• For more examples or help with RegEx or plugin settings, please download and read the user guide linked in this post.


What is Schedule Repair?

Schedule Repair is an integrated feature of EPG Cleaner designed to improve some aspects of MediaPortal's scheduling system.

MediaPortal does a great job of scheduling recordings but some schedule types don't handle dynamic guide data very well.
Schedule Repair tracks changes to program start and end times for these schedule types and updates the schedule accordingly.
If your schedules occasionally disappear from MediaPortal's TVGuide, record with the name of the previous show or miss the end of a show with last minute schedule changes, Schedule Repair can help.

_____________________________________________________________________________________________________

User Guide

A comprehensive user guide and Regular Expression reference is installed along with the plugin and is also available as a separate download here.
If you have any specific questions or feedback, please post to this forum thread.
_____________________________________________________________________________________________________

Installation

Installation is handled by the MediaPortal Extension Manager.
Just download and launch the EPGCleaner_1.0.2.0.mpe1 file from this post or the plugin repository, or start the MediaPortal Extension Manager from your desktop shortcut, browse for EPG Cleaner and click Install.

**For TV-Server only installations**

The MPEM system is not available on TV-Server only installations.
You will need to download and run the stand-alone installer contained in EPGCleanerSetup_1.0.2.0.zip from the linked downloads in this post.

**Manual Installation**

If you have any problems with the MPEM or stand-alone installers you can download the appropriate zip file package for your MP version.
Just extract the zip file's contents to your TV-Server Plugins folder, enable the plugin in TV-Server Configuration and re-start the TVService.
(Note that using this option will not add the plugin to the MP Extension Manager)
_____________________________________________________________________________________________________

MP Plugin Repository Listing: EPG Cleaner & Schedule Repair

Spoiler: Change Log

Version 1.0.2.0 (2016-05-20)

• Bugfix: Plugin config was slow to load with large number of channels

Version 1.0.1.0 (2016-03-11)

• Maintenance release
• Fixed minor bug in GUI filter table
• Improvements to EPG data pre-processing
• Code cleanup and prepare project for source release

Version 1.0.0.0 (2016-01-16)

• First official release
• Added MPEI installer

 

[hurley]

It might be a good idea to use this second post for some setup and Regex examples.
I'll get the ball rolling with my current settings for Australian TV networks.
Channel Nine has been the worst offender for guide spamming, though it's not nearly as bad as it has been in the past.

Aust. TV networks:

Title Filters:
1. "\(Inc.*\)?" (Replace with: empty) Channels: 9 network (Removes channel nine's spam)
2. " - (Double Episode|New Episode|New Season|New|Final|Premiere)" (move to description) Channels: 7,9 networks
3. "^Movie:\s" (move to description) Channels: 10 networks (This ones just for consistency across channels)

Don't include the quotes!!! and note leading/trailing spaces.

Other known examples:

To remove classification rating suffixes used in Finland eg: "Emmerdale (7)" or "Emmerdale (S)"
"\s\((S|7|12|16|18)\)$"
It means a space \s followed by any of the characters split by | between parenthesis and at the end of the line $.

To remove all 'sub-title' text after the main title eg: "Curious George And the Ants" -> "Curious George"
(?<=Curious George).*

 

[Edalex]

Hi.
I think plugin could use string.Replace methods additionaly to or instead of regex

 

[hurley]

Hi. I think plugin could use string.Replace methods additionaly to or instead of regex

From an efficiency or simplicity point of view..?
I went with regex for the flexibility, so I can be confident no matter what the TV networks stuff into my guide data, there is a regex pattern that will fix it.
You are right though, most use cases could be achieved with a simple String.Replace.

The plugin uses Regex.Replace for most operations combined with Regex.Match for moving text to the description field.
From a usage angle: A regex pattern 'foo' and replace with 'bar' will work just the same as String.Replace except that this will replace only the first found 'foo' unless the option: 'Filters process all matches' is enabled.

I was concerned about the overhead of using regex, so I did a lot of real world benchmarking the various techniques Static/Instance/Compiled etc..
Surprisingly the winner proved to be the instance constructor, which msdn states is not cached but in testing it most certainly is some 5-10 times faster after the first use and out-performs both Static and Compiled options in this use case.
I guess it goes to show, optimizing without benchmarking can be a fools errand.

There is a bit of a learning curve with regex, but the learning is worth it.

What in particular would you like to achieve that would benefit from using String.Replace?
I'm happy to take on board feedback and make adjustments, which would improve the plugin.

 

[Edalex]

Both simplicity and speed I think

 

[hurley]

I guess I preferred flexibility and user power over simplicity.
That is after all where MediaPortal wipes the floor with any other HTPC software, so fits with MP's ethos.

Are you having performance issues with the plugin? I'm not too familiar with the kind of data that needs to be processed in other regions. In particular I haven't at this stage made any allowances for languages other than English.
One performance issue of note is the debug log is a significant bottleneck on the plugin if it's enabled, due to the constant stream of writes.
The other logs queue their writes until associated tasks are complete, so are not such a problem.

The debug log should not be enabled in a production system and the important 'info' level details are written to tv.log and epg.log where required.

Edit: Having said that, you have a valid point. If regex is not required it's a waste of cpu cycles.
I might investigate the feasibility of incorporating a light String.Replace mode... no promises though...
Thanks for your feedback.

 

[azzuro]

Hello thank for your sharing and contribution.
I'm not sure about mpem and tvserver. For me it work.

And i see you are member since less than 1 year. Welcome to our community

 

[hurley]

Salut azzuro et merci.

In truth I have been lurking in the shadows for quite some years, but just recently joined.
I am happy to able to give something back to MediaPortal.
It has given me a lot including the inspiration to learn to code. Something that some years ago I thought was beyond my capabilities.
So I'm a huge fan of the open source model and sharing of knowledge for mutual benefit.

Furthermore, getting involved really makes you appreciate the enormous effort of all involved in the project.
Developers, patch writers, testers, doc writers, graphic artists, web devs, support staff, active users.......
It's really impressive what you all have achieved with an essentially unfunded, community driven project.
My hat is off to all right back to Frodo. Thank you.

In other news...

This is totally off-topic but on this day, nearly 4 billion years ago, a star exploded unleashing light 20 times as bright as our entire galaxy!! Mind blown! How utterly humbling our universe is. ..as you were...

 

[azzuro]

For information @hurley

happy to see you, now in our area.

About your plugin, don't forget to provide source file of your plugin, like you say, we are "open source" community,
and about open-source rights : all software "created for" or "using" open-source project must be shared.
This goal can have many benefits :
1st : all people can have acces to your code, for contribute patch, and many other stuff.
2nd : Official Dev, can have a eyes on your code, and maybe ask you for more contribution ( internaly ).

i can only point you (as personal opinion) to MP2 or the next TVE3.5 for providing help on coding side, but Team and Me, can't force you to do what you don't want.

So, Re-Welcome,

j'ai affaire à un francophone par hasard ?

 

[hurley]

About your plugin, don't forget to provide source file of your plugin

haha, that may well be my downfall as I don't have any formal training, just google, msdn, stackoverflow and lots of persistence.
Real devs eyes on my work will be quite intimidating, but might be a valuable lesson for me.
Nevertheless, it is my intention to share, once the plugin is proven ok and I've had a chance to clean up my source.

Non, je ne suis pas Français. Mais j'ai appris quelques mots d'une salle de chat que je visite parfois.