UK RadioTimes TV grabber and channels.dat update script (2 Viewers)

[yourmumquake]

Appologies but out of the three sub-forums: EPG, WebEPG or XMLTV, I'm not sure if this is the right place for the following query or if there is any interest:

I'm sure there may already be something similar and if so, will be better coded but just in case and this has taken me a while to get right, I would like to share a script (striclty speaking it is two seperate scripts) if there is any interest...

In a nut shell, the script updates the 1-xmltv_radiotimes_com.xml with the latest channels from RadioTimes URL and it then updates the channels.xml with this latest channel detail so that 'auto map' populates (in its mysterious way) any un-mapped channels with a priority on the RadioTimes details.

Another feature is to provide only UK listings in the 'Selection' box when adding manually.

Although primarily for UK, there should be enough detail in the script to transform for others.

Shout back and if any takers will add to this post - if I can...

 

[yourmumquake]

REPLACED ZIP: I noticed a 'bad' character in the original Mediaportal provided xml file and errors in log

Attached zipped file, contains RadioTimes.bat and RadioTimes.exe!

If AutoHotkey (AutoHotkey - Free Mouse and Keyboard Macro Program with Hotkeys and AutoText) is installed then RadioTimes.bat batch file is all that is required – it will generate its own autohotkey (AHK) script and then make a call to it. I have included a compiled autohotkey script (RadioTimes.exe) for those that do not have autohotkey but you’ll need to keep both bat and exe in same directory or edit the bat file to place where required.

The batch file has only been tried on XP, so don’t know what errors (or not) may occur on Vista or Windows 7. At worst, run batch on a PC with XP (having Mediaportal is not a requirement but you’ll need to create a few directories – see top if batch file for the dir’s) and copy the two resulting (channels.xml and 1-xmltv_radiotimes_com.xml) files across.

Also, I’ve only created and used this script with MP beta 1.1 – but I don’t believe either files have changed in operation from MP 1.0...

Take a look through the batch file in notepad and you should get a good idea as to what is going on and being achieved – but in essence if the 1-xmltv_radiotimes_com.xml is more than 6 days old the script will check the RadioTimes web site and update both the 1-xmltv_radiotimes_com.xml and channels.xml files with the latest information – there will be a brief message to alert of any updates and also leave a text file on desktop for easily identification an update is outstanding.

I am interested in any feedback on the way the channels.dat file is edited. You’ll note that when using WebEPG with the ‘new’ channels.xml there is clear indication the data is from RadioTimes using this file which makes it easier to determine manual updates, hopefully if you use you’ll see what I mean. I have not noticed any problems so far but really interested in feedback.

The script runs for about a 30 seconds in total and there will be a period of about 1 second where the mouse and keyboard are locked whilst saving the latest radiotimes channels details.

There are several methods to hide the cmd window whilst batch is running as there are several methods in regularly running the batch file. I can detail how I achieve hiding and regularly running but guess you’ll use your own methods that you care for.

 

[Splash_Womble]

This sounds interesting but I'm not sure quite what to do with it. I'm running XP with 1.10 beta so it should be similar to your setup.
When I try to find some channels, for example 'Yesterday' it isn't in the MP list but if I download the radio times list using the web address in your RadioTimes.bat file I see it's is there. I'm assuming this RadioTimes.exe will update the list?
I don't seem to have any success with automap so I've done them all manually.
Where do I put your 2 files? How do I run them?
Do you know if it's possible to get any radio listings? I don't see them but they are on the radio times website?

 

[yourmumquake]

Hi Splash_Womble and thanks for taking time to look at and comment.
I guess you don't have or use AutoHotkey, which is not a problem as you'll need to use both the batch (.bat) and exe. You can place them anywhere, for example create another directory under Program Files e.g. C:\Program Files\Team Mediaportal\XMLTV - place both files in this directory and either run the batch file directly by just double clicking on it or set an XP Task to run the bat file on a regular bases. Alternatively there is an option within Mediaportal setup to run a command in either powerschedular if enabled or WebEPG.
Both the batch and exe will automatically update both the channels.xml and radiotimes xml files located in the ‘all user’ Team Mediaportal WebEPG directory– none of the original files will be deleted, they will be renamed with an extension of .org – so if anything goes wrong, you can change back easily. The ‘automap’ appears to work by comparing the grabber xml files channel names to the scanned in TV card’s channel names, which is very hit and miss as you have found out as the web sites and TV providers don’t appear to work to a standard naming convention.

Since generating this programme, I have become more familiar with Autohotkey and am just finishing a re-write but using Autohotkey entirely and have also included DigiGuide xml (which has a richer description of programmes). This new script makes direct calls to the RadioTimes and Digiguides web servers and therefore does not rely on using a browser as used in this thread’s script – so hoping will be more robust.
I hope to finish this new script and test in the next week but will start a new thread as it includes both RadioTimes and Digiguide. Let me know how you get on and if you are interested in checking out the new script.

I am still hoping for feedback from ‘developers’ on how I change the channels xml (to help with ‘automapping’) and if there are any concerns…

 

[Splash_Womble]

OK I've had a quick play but it doesn't work for me. It creates all the files but the 1-xmltv_radiotimes_com.xml is corrupt. The channels.xml is OK. When the batch file runs I can see loads of errors but I don't know how to log them for you. PS I've updated to latest SVN

 

[yourmumquake]

Many thanks for trying and I'm sorry to hear it does not work for you.

I just tried the latest SVN (24102 – out yesterday) and downloaded the above zip file on this thread. It still works for me, although that does not help you! I can work with you if you are willing to spend time and have the patience – let me know and I’ll add some logging and ask you to check for certain files etc.

Alternatively, if you are keen to have an updated RadioTimes and MyDigiuide XML file, I have more or less finished the re-write and would welcome feedback using this instead. I’ll add a new zip to this reply – there are two files, you only need to run the ‘compiled’ RadioTimes_DigiGuide.exe, the ahk file is the script which can be opened with notepad for you to take a look at if interested. Most of the code should be fairly self explanatory. If you do try out please undo (put all back as per original) the changes that may have been made with the earlier script – channels.xml and also all the grabber directories and files, alternatively re-run the SVN should set it back to the original state.
Hopefully, all being well, you should see a text file on your desktop showing if any of the three files (channels.xml, 1-xmltv_radiotimes_com.xml, 2-www_mydigiguide_com.xml) have been updated and which channels were added.

 

[Splash_Womble]

I've tried yesterdays version and it runs OK, produces the list of new channels and I can then add them in the TV setup. BUT when I run the 'grab now' it only grabs the new channels, even though the old ones still have grab instructions next to them?
It's getting close though, well done. (I shouldn't bother with the Bleeb one as 2 are enough).

 

[yourmumquake]

To make use of the new xml files I think you'll need to 'clear-mapping' on your previous channels and re-map (hopefully auto-mapping will save you some work here), save and grab again. Always take a look in the 'tv.log' (and 'error.log') in the log directory, check the WebEPGImporter section and see what errors (if any) or detail is being recorded.

I change the channel id's to the actual name of the channel (as per web-site data) which likely conflicts with the original naming convention, e.g. the original MP name for BBC1 (or BBC ONE on DVB-T) is "bbc1@bbc.co.uk", where as I use the name as shown on the RadioTimes or MyDigiGuide web sites; "BBC1".

I was hoping someone more knowledgeable could comment on how these changes might cause issues anywhere else, if there are no other issues and requires just this initial re-mapping (it should only need to be done the once on first time of using the newly created xml file(s)), I'm happy living with it as I only use WebEPG to download the main-stream and Film channels (about 30-40) and let the TV EPG grabber fill in the rest.

Also, as I don't detect if the desktop text file has been viewed for new channels and WebEPG updated using any new additional channels, it will have to be deleted manually once satisfied all new available channels have been mapped in WebEPG - in order to provide a visual indication further channels have been added, this is more for if you setup an automated task to regularly run this prog.

If it is a big head-ache for you, as you may have many channels mapped using WebEPG, I can look at changing the prog to only add new channels to the existing (original) xml files (rather than re-name all channels) which should fix your issue.

Let me know how you get on and thanks for persevering.

(I am still updating this script to include more options/robust on the download of web data as the method used has a few potential pit-falls: The download might appear to succeed even when the remote file doesn't exist - this is because many web servers send an error page instead of the missing file, this error page is what will be saved in place of Filename. Internet Explorer 3 or greater must be installed for this function to work. Firewalls or the presence of multiple network adapters may cause this function to fail.)

 

[Splash_Womble]

Sorry for the late reply, been away this week. After I cleared everything out it works perfectly. 99% of all UK TV and Radio Channels. Fantastic. Just need the radio guide to select the channels properly then I'll be all systems go. Thanks again.

 

[clanmackenzi]

Guys,

Looks very interesting but sounds a bit complicated.

Are you doing this because WebEPG in MP1,1beta does not work as well as 1.02 did with XMLTV. I ask because I cannot get the auto update to work properly - see my thread here

https://forum.team-mediaportal.com/webepg-136/webepg-auto-grab-problem-mp1-1beta-74290/#post543606

Are your script files a solution to this problem.

Cheers