Tweaking DVB EPG (2 Viewers)

[Vasilich]

hi @hurley,
thanks for reply.
my intent was to ADD something, not to remove
e.g. all matches of "[a-z][A-Z]" replace with smth like "[a-z]\n[A-Z]" (it means every combination of small letter following with capital letter replace with same combination with carriage return in between).
Is it possible?

 

[hurley]

Try using 'lookarounds' http://www.rexegg.com/regex-lookarounds.html

Regex Expression: (?<=[a-z])(?=[A-Z]) Replace with: \r\n

** The \r\n doesn't seem to be getting processed by the textbox in the filter tester, but it should work for the actual data. I'll fix the textbox in next update.

Edit: Just tested this in practice with a description filter. (I assume that's where you want to use it!?)
The \r\n gets inserted between any lower case char followed by upper case char but it seems MediaPortal doesn't Edit: My sloppy code didn't process the \r\n correctly. The \r gets removed and the \n gets printed to the description text.
Will have to do some digging to see whats going on there...
---
Fixed below in v0.1.0.41
Though you might have occasional problems if there's a name like McGuire or DeCristo etc.. in the description text.
This might work better though:

Regex Expression: (?<=[^A-Z][a-z])(?=[A-Z]) Replace with: \r\n

-----------------------------------------------------------------------------------------------------------------

Edit @Vasilich
I've since discovered a much simpler and more efficient way to do this in line with your original idea of using substitutions:

Regex Expression: ([^A-Z][a-z])([A-Z]) Replace with: $1\r\n$2

 

[hurley]

Updated: EPGCleaner Plugin v0.1.0.41 - 2015-06-25

• BugFix: Newline character "\r\n" in Replace with field was getting escaped when it shouldn't.

Updated: EPGCleaner Plugin v0.1.0.42 - 2015-06-27

• BugFix: Possible null reference exception if Replace with field is left blank in v0.1.0.41 (sorry about that)
• Added: Generic special character handling routine (works for: \r \n \t)

 

[hurley]

Updated: EPGCleaner Plugin v0.1.0.51 - 2015-09-12 (beta)

Been a while between updates but I've made some more progress towards a version 1.0 release.
This is mainly a polishing up of code with the most significant change being another overhaul of the logging side.
I've switched to xml for the EPG Cleaner and Schedule Repair logs which allows for much faster loading.
Old logs from v0.1.0.40+ will be automatically upgraded, so the first load will be slow.

Also spent/wasted a lot of time making a custom ListView control for the log display that has collapsible grouping and works in both WinXP and Win7. It's quite neat however and has a Win7 style theme.

Other changes are minor and mainly GUI related. Things like:
- Scroll-synchronised textboxes
- More consistent colours for selected items etc.
- Fixed some layout inconsistencies in WinXP vs Win7 (un-tested in other OS's)
- etc, etc, etc...

I ran some tests on a process throttling option but I don't believe it's needed, however most debugging is being done on a quad core desktop so I may revisit this later.
Also added a Schedule Repair configuration page, but I'm not 100% happy with it, so left that out for now too.

Question: Would users like access to the nitty gritty schedule repair options or prefer it to just be taken care of. 'More buttons please' or 'Just press go' ... ?

I do appreciate this plugin is already somewhat complicated and will get on to that help/user guide once things have settled.

That's it for now..

Edit: Removed version 0.1.0.51 of plugin. See post below for version 1.0 Release Candidate

 

[alanwads]

Could somebody possible please provide some instructions of how to use this epgcleaner.dll file, i.e. where to copy it to and how to access its features?

Many thanks.

 

[hurley]

Hi @alanwads, sorry I missed your post.

Re Installation: There are two versions of the EPGCleaner.dll in sub-folders within the rar file. One is for pre 1.6 and one for 1.6+ versions of MediaPortal. Choose the appropriate dll and place it in the TV Server Plugins folder: "C:\Program Files\Team MediaPortal\MediaPortal TV Server\Plugins".

Open TV-Server Configuration, go to the Plugins page and enable the plugin. You may need to restart the TVService which can be done from the Manual Control page or via Administrative Tools -> Services.

Re Usage: In brief, the Regex Expression column on the plugin's Settings tab page is where you would input an expression pattern for filtering program titles. The Replace with column holds the replacement text for any matches of the given expression. (This may be left empty to remove the matched expression text or input a replacement value here. Clicking the check box to the left of this column will move the matched text to prefix the program's description field)

The Channels(s) column allows you to only apply this filter to the selected channels. Clicking this text box opens a compact drop-down menu of available channels to select or you can click the button to its right to open the Channel Selector dialog if you have a lot of channels to navigate.

You can use the filter tester at the bottom of the settings page to test your regex patterns.
To do this, enter a program title which contains text you want to filter into the text box, select the filter you want to test and click the Test button. If the test is successful Match will be displayed below the test button and the resulting program title displayed below the input textbox.

I do intend to complete this plugin with an installer and user guide at some point, but it's been downgraded on my priorities somewhat due to the limited interest/feedback, I presume it's not particularly useful to many users.

If you have any specific questions about usage or regex expression's, or an example of a program title you'd like to filter, I'd be happy to help.

hurley.

 

[alanwads]

Hi Hurley,

Thanks for the reply. I got it up and working and it's doing exactly what I wanted; removing the text "New: " from EPG listings which was making the recording of new series a real pain.

Alan

 

[hurley]

Ah good news, I'm glad to hear it's working. Just a tip, you could tighten up your regex for that example (if you haven't already) with the 'start of line' character ^ ie: ^New:\s
Thanks for your feedback.

 

[hurley]

Updated: EPGCleaner Plugin v0.1.0.61 - 2015-12-24 (Version 1.0 Release Candidate)

Pre Christmas effort to get this thing to version 1...
Quite a few changes in this release, though I've lost track of a lot of the smaller ones.
The big news is I have finally completed that user guide as a Windows chm file and there is now a user friendly universal installer which handles all MePo versions.
This will be wrapped in an MPEI package for the official Version 1.0 release in coming weeks...

If I don't find any bug's or get reports of errors I'll set up a download page and forum thread for the plugin.
Thank you to @noonereallycares for allowing me to commandeer your thread.
And thank you to all those who have tried the early alpha/beta versions, I hope it's a useful addition to your MediaPortal bag of tricks. Merry Christmas to All!!

Edit: Had one of those 'light-bulb' moments you have when you stop focusing on something for a few days.
Implemented a threading related change which yielded a significant performance improvement so I'm updating this RC to v0.1.0.61
Also made a small change to the installer to ensure TV-Service restart when importing a saved configuration file.

hurley.

Spoiler: ChangeLog

Major Changes

• Dropped the title similarity algorithm.

It's results were too un-predictable for me and with effective use of title filters, the variations in program titles almost never happens.
The goal here is better reliability so it had to go.

• Changed the way EPG data is processed

Previously all programs on an updated channel were filtered, but the epg data can sometimes have holes which may cause programs to be filtered twice.
Now, existing programs must be matched to the new epg data.

• Description fields are always updated

MediaPortal has a hard coded setting which causes description fields to only be updated if the new value is longer than the existing value.
(While this may be appropriate in certain situations it really should be a user selectable option).
This can cause similar problems as above with multiple filtering of the same description text.
Description fields are now always refeshed where required in an epg pre-processing routine.
This pre-processing also removes duplicate and zero length entries which seem to occasionally exist in the epg data.

• Added option to process all regex matches per filter

More info on this in user guide. In most cases leave this option un-checked.

• Removed the global case insensitive filters option

Individual filters can be made case insensitive if required with special regex syntax.
More info on this in the Regular Expression Reference in the user guide.

• Upper to title case and Remove/Separate short description prefix

These options have both been improved. They will never be 100% accurate but aren't detrimental to scheduling if they get it wrong.
Handling of acronyms, surnames and the ABC networks description #hashtag's have been improved.

• Added user selectable Max Schedule Adjustment value

This Schedule Repair option is set at a very conservative 20 mins by default, but I personally set this value to around 2 hrs.

• Recording End Time repair

This feature which applies to Once and Everytime type schedules has been improved by accessing the actual EndTime property of the RecordingDetail using Reflection, rather than the hackish approach of changing the post record interval.

• ToolTips in Windows 7

Seems ToolTips were sometimes/always not working properly under Win7. This bug now resolved.

• User Guide

A comprehensive user guide is now installed along with the plugin dll.
This can be opened using the buttons at top right of the plugin's configuration page in TV-Server Configuration.
The button with the red 'R' opens the guide at the Regular Expression Reference page.

• Universal Setup Wizard

This Setup Wizard handles installing, uninstalling and backup/import of plugin settings.
For users of earlier MediaPortal versions it will install the .NET 3.5 version of the plugin. (Please run the installer again if upgrading to MP 1.6 or later)
The wizard also removes any obsolete settings from Alpha and Beta releases that would be left behind in the database and deletes all logs etc should you choose to use the uninstall option.
When you use the Setup Wizard there is no need to worry about stopping/restarting the TV-Service or enabling the plugin. This is all taken care of by the installer.

......If I remember any other important changes I'll update this post.

 

[hurley]

Update: EPG Cleaner plugin now has it's own thread and repository listing. Please direct further queries there. Thank you, hurley.

• Forum Thread: [Plugin] EPG Cleaner & Schedule Repair
• Repository: EPG Cleaner & Schedule Repair