WebGrab+Plus a new xmltv grabber (5 Viewers)

Christoph21x

Portal Pro
January 31, 2009
50
14
Berne
Home Country
Switzerland Switzerland
== EDIT 2012-09-19: New Version 1.0.1 of SwapCat available under http://21x.eu/downloads/SwapCategory.html ==
Attachment version 1.0 deleted. - See Post https://forum.team-mediaportal.com/threads/webgrab-plus-a-new-xmltv-grabber.88425/page-19#post-916902 for the most recent version.

Ok - solved the "multiple categories" for me: My EPG showed categeory information like "USA 2009" (especially coming from grabbed web sites like tvtoday.de, tvtv.* etc. - but maybe you encounter the same.
If anyone encounters the same issues as described on the previous pages, he might want to have a look at this helper application - see attachment.
The easiest way: download - read the SwapCat.readme.txt from within the package and TRY OUT!

If you have any question or suggestion, post it here - I'll read it.

More information:

I wrote a tiny application to solve my issues - maybe also interesting to others (see attachment below) , facing a similar issue:

Using several scrapers on several internet guide sites and a variety of front-ends may lead to the fact, that various EPG's may show "irrelevant" category information as the main category for a show in Fortherecord.
Reason is, that 4TR only uses the FIRST category definde for its EPG, recordings and suggestions.

Gettting started: Download and try out the attached ZIP archive, read the SwapCat.readme.txt for an quick start AND extensive explanatory.

Most of the scrapers support (as defined) "multiple fields" for the category information, whereas most of the EPG viewers (including Fortherecord and MediaPortal) only reference the first entry of all those categories (per programme).
If you encounter categories like "USA 2009" in your main EPG - you should use SwapCat.
If you encounter categories like "Thriller" in your EPG and the online view of the grabbed website shows something like "Thriller; Comedy; Thriller Comedy" and you want to see the latter in your EPG (meybe it's even the Sub-Category, depending on your grabber, and you want to see rather THIS) - you should use SwapCat.
(For a quick check, whether SwapCat will be able to serve you, read the SwapCat.readme.txt of the archive and jump to the section "2.1. Quick start guide".)

Two basic modes are available: the interactive and the automated run.
The interactive mode will let you do step by step whatever you want or simply run an analysis on the given source file. In this mode you may safely run this analysis (only) on your source TV guide XML file without changing anything - it will tell you, if SwapCat will be of use for you on the given source file - and then invoke run the swap.
You may select the target TV guide XML file (which MUST be different from the source) and select a logging file to give you more details on analysis or swapping.
The automated run is performed when you call SwapCat with the appropriate command line arguments - see "2. Usage".

SwapCat will open your source TV guide XML file (read-only; no changes are done), parsing the file and find multiple field-entries of categories in your XMLTV file.
On finding them (if not running analysis only) will make SwapCat swap the order of the category occurences within a program. The last entry will become the first one, the order will be inverted. Everything else in your TV guide source XML will simply be copied to the target.

EXAMPLE:
Code:
    Your source TV guide XML file shows:  ...will convert to... order in your newly created TV guide target XML file:
 
    <category>USA 2009</category>                                  <category>Thriller-Comedy</category>
    <category>Thriller</category>                                      <category>Comedy</category>
    <category>Comedy</category>                                    <category>Thriller</category>
    <category>Thriller-Comedy</category>                        <category>USA 2009</category>
 
    Your EPG shows as category:                                          Your EPG shows as category:
    "USA 2009"                                                                    "Thriller-Comedy"

Usage of SwapCat - interactively or automated with command line parameters:
'SwapCat.exe': starts the interactive SwapCat (supports analyzing a source)
'SwapCat.exe <guide source xml file> <swapped target guide xml file>': automated run: swaps the order into a newly created guide xml file, switches to interactivity on error
'SwapCat.exe <guide source xml file> <swapped target guide xml file> <log file>': automated run: swaps the order into a newly created guide xml file, writes log file but exits on error

Details are described in the package. Interaqctive run, automated run with error detection, unconditional automated rn with log creation... see it in the package.

If you have any questions or suggestions, please post them here.

Kind regards - Christoph

Kind regards from Switzerland - Christoph
 
Last edited:

shummi

Portal Pro
February 24, 2010
51
2
Home Country
Denmark Denmark
Hi Jan

I am fetching from ontv, and at http://ontv.dk/tv-guide/tv1000-classic
The latest program starts at 22:00 and next mornings program start at 8:00, so i have set <skip>12,1</skip> in config file.

So now all programs are in XML file, but the last program haven't any stop time, so it continue until next morning, actually it should stop at 23:59
Code:
  <programme start="20120721220000 +0200" stop="20120722080000 +0200" channel="Viasat Film Classic">
    <title lang="da">Red Dawn</title>

I am basically using the original ontv scrapper from pack 10, and i doesn't contain a stop param.
probably because i doesn't show on the index page.

I have tried to take it from the sub-page, but it doesn't work, is there another way to archive this?
Code:
stop.scrub {single()|<div class="content" id="content">| - | |</p>}

Thanks in advance /Shummi
 

tacker

New Member
July 25, 2012
1
0
44
Home Country
Algeria Algeria
hi,
i do have a working setup of webgrab which provides me a xml file trough the WebGrab++.config.xml file.
now I would like to use the mdb postprocessor function in the new version of webgrab.
my question is what do I need to put in the WebGrab++.config.xml or what do I need to do now to get for excample the IMDB information inside my xml file.
where do I do the settings for the the mdb postprocessor?
thank you in advance.
 

Django.edwards

Portal Pro
October 22, 2009
457
54
Eupen
Home Country
Belgium Belgium
Hi, take a look at the following line in the config.xml:
<postprocess run="y" grab="y">mdb,imdb.com</postprocess>

you should enable the postprocess run and the grab.
Be aware that the imdb data will be written in another file in the mdb directory. The original guide.xml is not touched with imdb data. There is a imdb.config.xml file which described the config.
 

WG++Maker

Portal Pro
October 25, 2010
130
112
La Gomera, Canary Islands
Home Country
Spain Spain
New Version of WebGrab+Plus V1.1.1

This version adds a few major features:
1. Another postprocessor REX (Reallocate and Edit Xmltv) that can
- Move the content of xmltv elements to other xmltv elements
- Merge the content of several xmltv elements
- Add comments/prefi/postfix text
- Remove or create xmltv elements
E.g. If the PVR software do not recognize some of the xmltv elements grabbed (like actors) it can add
these to the description and remove the now useless actor xmltv element.
2. Improved MDB postprocessor (automatically adds IMDB data to your xmltv file)
3. Added support for LINUX (in combination with MONO) and tested in W8
4. A command line parameter that specifies and overrules the default homefolder-path of the program for user that prefer another location of it.
5. Support for sites that use the POST HttpWebRequest method, further extending the range of websites.
6. The installation packages contain ini files for 126 TvGuide websites in about 60 countries worldwide!!

... and , on top of that, many small improvements and bugs fixed as usual.

Download links:
For LINUX:
http://www.servercare.nl/Docs/WebGrab+PlusV1.1.1LINUX.rar
For Windows:
http://www.servercare.nl/Docs/WebGrab+PlusV1.1.1MSI.rar
The documentation:
http://www.servercare.nl/Docs/Documentation.V1.1.1.pdf
All the available siteini files:
http://www.servercare.nl/Docs/SiteIni.Pack.11.rar
The list of all available supported TvGuide sites:
http://www.servercare.nl/Docs/SiteIni-List-Pack.11.pdf

For more info and support:
http://www.servercare.nl/Lists/Posts/ViewPost.aspx?ID=98
WebSite in preparation:
http://www.webgrabplus.com

Many thanks to everyone contributing to the development, especially Alberto and Willy.

WG++Maker ... Jan
 
Last edited:

Der_Bub

MP Donator
  • Premium Supporter
  • June 17, 2008
    830
    60
    Cologne
    Home Country
    Germany Germany
    Hello,

    Since V1.1.0 I have a "perfomance Problem": Everytime WebGrab is running, small stuttering appears to TV within MediPortal.

    I reduced Process priority to idele (and increased MP), but it still has some stutter.

    Do you have any Idea to reduce this? Does any Part/Option of WebGrab needs outstanding system power and is it possible to configure/disable it?

    Thanks!
     

    pellegj

    Portal Pro
    May 7, 2009
    106
    32
    Home Country
    Sweden Sweden
    Hello,

    Since V1.1.0 I have a "perfomance Problem": Everytime WebGrab is running, small stuttering appears to TV within MediPortal.

    I reduced Process priority to idele (and increased MP), but it still has some stutter.

    Do you have any Idea to reduce this? Does any Part/Option of WebGrab needs outstanding system power and is it possible to configure/disable it?

    Thanks!
    Hi!
    Are you sure it's when WebGrab is running, and not after WebGrab has run, when MP is importing the guide data?
     

    Users who are viewing this thread

    Top Bottom