WebGrab+Plus a new xmltv grabber (6 Viewers)

[WG++Maker]

I have made an advanced /multisite/incremental EPG grabber and like to share it with MediaPortal users

Some background:
* I live in a rural area on the Canary Islands where the internet connections are just dialup speed (no ADSL or 3G). On occasion I can use a satellite downlink . Lets say the speed varies between 2-50kB/s.
* Under these conditions I have been using several epg grabbers, but it always took 'hours' to just the daily update.
* The above was the reason I decided to make my own epg grabber which limits downloading to an absolute minimum without loosing richness of data. With it, the daily download for 25 channels in full detail takes about 10 minutes (sat downlink) to 1 hour (in dial-up)

Some features:
* Incremental grabbing. It only grabs what is new or what needs updated because it is changed or missing.
* Can grab from multiple tvguide sites in one run. Each site is programmable through a SiteIni file (I have made a few Dutch, Belgium, Spanish and English siteini files so far). These siteinifiles can be made by the user with the help of debugging modes and the documentation. Of course I am prepared to help or make one if requested
* The output follows the xmltv standard.
* Most xmltv elements are supported, like start and stop times, title (+ original title), sub-title, category, rating, star-rating, credits (director, actor, presenter), (production)date and episode. All of which are just a choice of the user and programmable per channel.
* All kinds of other options, like, days to grab, one-show-update-only and site-index-page-only (less detail but super fast) and xmltv-element-value filtering and modification.
* Written in C# , executable less that 1MB. Tested for XP, Vista and W7. It runs in command mode, no GUI , no drivers to install. FreeWare, Absolutely non commercial, no malware or virus (I wouldn't even know how to get them in it !?)

In function it comes close to tvxb but tests show it is about 2 to 3 times faster while providing a more detailed epg.

The program is in its first beta and available for download on
WebGrab+Plus, an advanced XMLTV grabber - ServerCare home
Further there is an active user group in Spain on
WebGrab+Plus Nuevo Grabber para generar una gran guia EPG - TodoHTPC - TodoHTPC

Jan

 

[richy759]

Hi Jan,
Having a play with your webgrab++ program, I must say it looks good. There doesn't seem to be a radiotimes.com.ini but there is refernence to radiotimes in the WebGrab++.config.xml file. Is this missing from the latest release? Do you have a file you could attach that I can use to get this going?

Thanks,
Rich

 

[WG++Maker]

Rich,

I have removed the support for radiotimes because the results were very dissapointing and unreliable.
For now the prefered tvguide site for the UK is tvguide.co.uk, which comes into two flavours: The standard full detail and the ultra fast (a fraction of a second/program) index-only. This site is rather unique in that it provides a near complete epg in this super fast mode.

Jan

 

[richy759]

Jan,
I have added all the channels I can from tvguide.co.uk but I can't seem to see details for any of the BBC HD channels (BBCHD and BBC ONE HD). You can add them to the guide but they never appear. I was using hover over to get the channel ID, is there another way that you know of?

managed to find BBC HD channels, just missing 301 and BBC news now...

Cheers,
Rich

 

[WG++Maker]

Rich ,
see personal message about a complete channel list for tvguide.co.uk.
BBC news = 66, 301 doesn't exist

 

[doornjoostje]

Wouldn't it be nice to incorpocate the faster grabbing into the webepg of mp 1.2.0?

 

[WG++Maker]

Latest version 1.0.2 went public today @
WebGrab+Plus, an advanced XMLTV grabber - ServerCare home

Supported tvguide sites:

Belgium: tv.be, mijn-tv-gids.be
Brazil: hagah.com.br
Netherlands: tvgids.nl, gids.publiekeomroep.nl, tvgids.upc.nl tvtv.nl
Spain: plus.es, laguiatv.com, teleprogramma.tv
Sweden: dagenstv.com, svt.se
UK: tvguide.co.uk, radiotimes.com, tvtv.co.uk

 

[WG++Maker]

There is a new version of the program 1.0.4 that went public a couple of days ago.
WebGrab+Plus, an advanced XMLTV grabber - ServerCare home

Three new supported sites are added:

For Denmark : tvguide.dk
For USA and N/S America's : directv.com
Belgium : yelo.be

And of course the sites from the list on the previous post

There is also a special script that facilitates the use of this grabber in Media Portal and solves a few issues with the Media Portal xmltv importer.

 

[richy759]

I have noticed that I get a very repeatable "Error downloading page: the remote server returned an error: (500) internal server error" on some channels when grabbing from tvguide.co.uk. It retries but eventually goes onto the next item. This problem doesn't see like the web page isn't there (I'm not familiar with the error codes) but perhaps there is some problem with the request. Any ideas?
Regards,
Richard

 

[WG++Maker]

Richy,

As you can read below , this problem is created by the server (tvguide.co.uk).

Fixing 500 errors - general
This error can only be resolved by fixes to the Web server software. It is not a client-side problem. It is up to the operators of the Web server site to locate and analyse the logs which should give further information about the error.

These thing seem to happen now and then and (hopefully) dissappear soon.
Although it shouldn't make much difference and assuming you run WG++ V1.0.3 or higher you can try to add mode w in the configfile. It activates the alternative Grabengine Wget.

WG++Maker