[Plugin] EPG Cleaner & Schedule Repair (1 Viewer)

hurley

Portal Pro
February 2, 2015
61
57
Australia Australia
What is EPG Cleaner?

EPG Cleaner is a TV-Server plugin, which allows DVB EPG data to be cleaned or filtered as it is loaded into the TV-Guide.
Remove spam or prefixes and suffixes from program titles and description text, which make the TV-Guide difficult to read and cause problems with scheduling.

For example if your TV network appends words like 'New' or 'Repeat' or 'Premiere' to program titles in an inconsistent manner, or parental ratings like (12) or (16) which are different for each episode, you may need to create multiple schedules for your favourite series to avoid missing any episodes.
Examples like these will also result in series recordings being scattered over multiple folders in Recorded TV.

EPG Cleaner uses Regular Expressions (RegEX) to create powerful and highly flexible filters for your guide data to remove these unwanted parts.
Each filter may apply to all channels or only selected channels.

"RegEx sounds too complicated for me"...
Don't worry, EPG Cleaner has a Simple mode which uses wildcards like * and ? which most Windows users should be familiar with.​

• For more examples or help with RegEx or plugin settings, please download and read the user guide linked in this post.


What is Schedule Repair?

Schedule Repair is an integrated feature of EPG Cleaner designed to improve some aspects of MediaPortal's scheduling system.

MediaPortal does a great job of scheduling recordings but some schedule types don't handle dynamic guide data very well.
Schedule Repair tracks changes to program start and end times for these schedule types and updates the schedule accordingly.
If your schedules occasionally disappear from MediaPortal's TVGuide, record with the name of the previous show or miss the end of a show with last minute schedule changes, Schedule Repair can help.

_____________________________________________________________________________________________________

User Guide

A comprehensive user guide and Regular Expression reference is installed along with the plugin and is also available as a separate download here.
If you have any specific questions or feedback, please post to this forum thread.
_____________________________________________________________________________________________________

Installation

Installation is handled by the MediaPortal Extension Manager.
Just download and launch the EPGCleaner_1.0.2.0.mpe1 file from this post or the plugin repository, or start the MediaPortal Extension Manager from your desktop shortcut, browse for EPG Cleaner and click Install.

**For TV-Server only installations**

The MPEM system is not available on TV-Server only installations.
You will need to download and run the stand-alone installer contained in EPGCleanerSetup_1.0.2.0.zip from the linked downloads in this post.

**Manual Installation**

If you have any problems with the MPEM or stand-alone installers you can download the appropriate zip file package for your MP version.
Just extract the zip file's contents to your TV-Server Plugins folder, enable the plugin in TV-Server Configuration and re-start the TVService.
(Note that using this option will not add the plugin to the MP Extension Manager)
_____________________________________________________________________________________________________

MP Plugin Repository Listing: EPG Cleaner & Schedule Repair

Version 1.0.2.0 (2016-05-20)
  • Bugfix: Plugin config was slow to load with large number of channels
Version 1.0.1.0 (2016-03-11)
  • Maintenance release
  • Fixed minor bug in GUI filter table
  • Improvements to EPG data pre-processing
  • Code cleanup and prepare project for source release
Version 1.0.0.0 (2016-01-16)
  • First official release
  • Added MPEI installer
001_SetupWizard.png 001_SettingsMain.png 002_EPGCleanerLog.png 003_ScheduleRepairLog.png 004_DebugLog.png
EPGCleaner_Icon_64.png
 

Attachments

Last edited:

hurley

Portal Pro
February 2, 2015
61
57
Australia Australia
It might be a good idea to use this second post for some setup and Regex examples.
I'll get the ball rolling with my current settings for Australian TV networks.
Channel Nine has been the worst offender for guide spamming, though it's not nearly as bad as it has been in the past. :censored:

Aust. TV networks:

Title Filters:
1. "\(Inc.*\)?" (Replace with: empty) Channels: 9 network (Removes channel nine's spam)
2. " - (Double Episode|New Episode|New Season|New|Final|Premiere)" (move to description) Channels: 7,9 networks
3. "^Movie:\s" (move to description) Channels: 10 networks (This ones just for consistency across channels)

Don't include the quotes!!! and note leading/trailing spaces.



Other known examples:

To remove classification rating suffixes used in Finland eg: "Emmerdale (7)" or "Emmerdale (S)"
"\s\((S|7|12|16|18)\)$"
It means a space \s followed by any of the characters split by | between parenthesis and at the end of the line $.

To remove all 'sub-title' text after the main title eg: "Curious George And the Ants" -> "Curious George"
(?<=Curious George).*
 
Last edited:

Edalex

Community Plugin Dev
  • Premium Supporter
  • January 3, 2008
    2,923
    1,233
    Saratov
    Russian Federation Russian Federation
    Country flag
    Hi.
    I think plugin could use string.Replace methods additionaly to or instead of regex
     

    hurley

    Portal Pro
    February 2, 2015
    61
    57
    Australia Australia
    Hi. I think plugin could use string.Replace methods additionaly to or instead of regex
    From an efficiency or simplicity point of view..?
    I went with regex for the flexibility, so I can be confident no matter what the TV networks stuff into my guide data, there is a regex pattern that will fix it.
    You are right though, most use cases could be achieved with a simple String.Replace.

    The plugin uses Regex.Replace for most operations combined with Regex.Match for moving text to the description field.
    From a usage angle: A regex pattern 'foo' and replace with 'bar' will work just the same as String.Replace except that this will replace only the first found 'foo' unless the option: 'Filters process all matches' is enabled.

    I was concerned about the overhead of using regex, so I did a lot of real world benchmarking the various techniques Static/Instance/Compiled etc..
    Surprisingly the winner proved to be the instance constructor, which msdn states is not cached but in testing it most certainly is some 5-10 times faster after the first use and out-performs both Static and Compiled options in this use case.
    I guess it goes to show, optimizing without benchmarking can be a fools errand.

    There is a bit of a learning curve with regex, but the learning is worth it. :)

    What in particular would you like to achieve that would benefit from using String.Replace?
    I'm happy to take on board feedback and make adjustments, which would improve the plugin.
     

    hurley

    Portal Pro
    February 2, 2015
    61
    57
    Australia Australia
    I guess I preferred flexibility and user power over simplicity.
    That is after all where MediaPortal wipes the floor with any other HTPC software, so fits with MP's ethos.

    Are you having performance issues with the plugin? I'm not too familiar with the kind of data that needs to be processed in other regions. In particular I haven't at this stage made any allowances for languages other than English.
    One performance issue of note is the debug log is a significant bottleneck on the plugin if it's enabled, due to the constant stream of writes.
    The other logs queue their writes until associated tasks are complete, so are not such a problem.

    The debug log should not be enabled in a production system and the important 'info' level details are written to tv.log and epg.log where required.

    Edit: Having said that, you have a valid point. If regex is not required it's a waste of cpu cycles.
    I might investigate the feasibility of incorporating a light String.Replace mode... no promises though...
    Thanks for your feedback.
     
    Last edited:

    azzuro

    Test Group
  • Team MediaPortal
  • May 10, 2007
    9,636
    5,267
    France - IDF
    France France
    Country flag
    Hello thank for your sharing and contribution.
    I'm not sure about mpem and tvserver. For me it work.

    And i see you are member since less than 1 year. Welcome to our community
     

    hurley

    Portal Pro
    February 2, 2015
    61
    57
    Australia Australia
    Salut azzuro et merci.

    In truth I have been lurking in the shadows for quite some years, but just recently joined.
    I am happy to able to give something back to MediaPortal.
    It has given me a lot including the inspiration to learn to code. Something that some years ago I thought was beyond my capabilities.
    So I'm a huge fan of the open source model and sharing of knowledge for mutual benefit.

    Furthermore, getting involved really makes you appreciate the enormous effort of all involved in the project.
    Developers, patch writers, testers, doc writers, graphic artists, web devs, support staff, active users.......
    It's really impressive what you all have achieved with an essentially unfunded, community driven project.
    My hat is off to all right back to Frodo. Thank you. (y)

    In other news...

    This is totally off-topic but on this day, nearly 4 billion years ago, a star exploded unleashing light 20 times as bright as our entire galaxy!! Mind blown! How utterly humbling our universe is. ..as you were...
     

    azzuro

    Test Group
  • Team MediaPortal
  • May 10, 2007
    9,636
    5,267
    France - IDF
    France France
    Country flag
    For information @hurley

    happy to see you, now in our area.

    About your plugin, don't forget to provide source file of your plugin, like you say, we are "open source" community,
    and about open-source rights : all software "created for" or "using" open-source project must be shared.
    This goal can have many benefits :
    1st : all people can have acces to your code, for contribute patch, and many other stuff.
    2nd : Official Dev, can have a eyes on your code, and maybe ask you for more contribution ( internaly ).

    i can only point you (as personal opinion) to MP2 or the next TVE3.5 for providing help on coding side, but Team and Me, can't force you to do what you don't want.

    So, Re-Welcome,

    j'ai affaire à un francophone par hasard ?
     

    hurley

    Portal Pro
    February 2, 2015
    61
    57
    Australia Australia
    About your plugin, don't forget to provide source file of your plugin
    haha, that may well be my downfall as I don't have any formal training, just google, msdn, stackoverflow and lots of persistence.
    Real devs eyes on my work will be quite intimidating, but might be a valuable lesson for me.
    Nevertheless, it is my intention to share, once the plugin is proven ok and I've had a chance to clean up my source.

    Non, je ne suis pas Français. Mais j'ai appris quelques mots d'une salle de chat que je visite parfois. :)
     

    Users who are viewing this thread

    Similar threads
    OP Title Forum Replies Date
    M Epg selection plugin? Feature Requests 2
    C Issue with plugin initialisation My TVSeries 7
    morpheus_xx [Finished] "What's New" plugin for MP2 upgrades MediaPortal 2 20
    da3dsoul [Request] An MVP Plugin example for custom settings in MP2 Plugin Development 2
    da3dsoul MP2 Plugin - Getting Started (Directory Tree and Player) Plugin Development 27
    M Installing Plugins Newcomers Forum 2
    joecrow 1.25.0 Radio Time plugin fails after Resume from Sleep with MP 1.25 Bugreports 10
    P MP2: US user, OTA - can NOT get SchedulesDirect plugin to work on MP1 or MP2 --> how do I get OTA schedule? General 5
    E aktuelle englische Forums Webseite von - Showtime Plugin v3 Plugins & Erweiterungen 4
    mrmojo666 [Tv Server Plugin ]Direct Show Video Control Plugin 0.0.0.1 for Hauppauge PVR cards MediaPortal 1 Plugins 0
    joecrow MultiShortcut Plugin (Uavailable) in DWHD skin General Support 58
    vapourEyes Building TV Plugin General Development (no feature request here!) 49
    Golf4 Nach Windows-Updates läuft TV-Plugin nicht mehr Allgemeines Support- und Diskussionsforum 0
    joecrow How about a Launcher plugin for MP1? MediaPortal 1 6
    D [solved] TVServer Plugin must be missing something. General Development (no feature request here!) 1
    framug Nouveau plugin XmlTvFR pour récupérer le guide télé français Kazer / ZguideTv (WebEPGfr) 28
    emphatic Allow remote control mapping of all plugins Improvement Suggestions 0
    D Where should plugins be located on hard drive? General Support 0
    D how to disengage from one version of a plugin to use another Newcomers Forum 0
    cbr954rr How to disable a pre-configured plugin? General 2
    mrmojo666 [Plugin]Marantz Control Plugin 0.0.0.2 MediaPortal 1 Plugins 2
    mrmojo666 beginner: trying to write a plugin, need help to catch event General Development (no feature request here!) 3
    framug [Rejected] Fix PowerScheduler plugin for MP2 single seat Submit: Code Patches 11
    doskabouter [Question] Stale plugins Developer Discussions 9
    M MP1 stops responding after Starting Plugins General Support 14

    Similar threads

    Top Bottom