WebGrab+Plus a new xmltv grabber (6 Viewers)

[Paranoid Delusion]

Hi Jan

Just trying this out but getting a exception on first run using Win7 32bit, no c:\programdate folder is created, read through this thread and cannot find any answer, did I miss something, have pretty much every windows update going, so have no idea what is wrong.

ps. your dependency checker showed nothing.

 

[WG++Maker]

Hi Jan

Just trying this out but getting a exception on first run using Win7 32bit, no c:\programdate folder is created, read through this thread and cannot find any answer, did I miss something, have pretty much every windows update going, so have no idea what is wrong.

ps. your dependency checker showed nothing.

View attachment 93989

I assume that you used the msi installer !?

The fact that no C:\ProgramData\ServerCare\WebGrab folder is created suggest that you accendently tried to install WebGrab+Plus by running 'setup.msi' from the deploy folder!
Instead you should run 'Install.bat' which is located in the WebGrab+PlusV1.0.9MSI home folder among all the ini files!!
(Explanation: The install.bat file creates the WebGrab homefolder C:\ProgramData\ServerCare\WebGrab, and fills it with all the necessary files, when that is done it automatically starts setup.msi from the deploy folder)


hope this helps WG++Maker ... Jan

Hello Jan,

first of all: Thanx for this great plugin!

I'm trying to configure a belgian website for my epg. It is the french version of the mijn-tv-gids.be, situated at mon-programme-be.
It seems to work so far, but I have some difficulties to insert the episode numbers. I have attached the Mediaportal tv.log where you can see the error, and the .ini files I use.
Maybe you can provide me some help on this topic.

Thanks in advance

Django

MediaPortal requires the episode in the 'xmltv_ns' format (google xmltv.dtd for a definition) to recognise as episode data. If it is nor in that format it remains just data for the user to interpret.
I am modifying your siteini to convert the episode into xmltv_ns format and will come back here if its ready.

WG++Maker ... Jan

 

[WG++Maker]

Django:

attached the revised 'mon-programme-tv.be.ini' that has the episode in xmltv_ns format.

Be aware that all the episode and season vales are 1 less than in the site! That is because of the xmltv_ns definition that is index 0 based (episode 1 = 0)

thanks for contributing your siteini!

WG++Maker .. Jan

 

[redeye19987]

Hi Jan,

I am trying to get the new included Australian .ini working. It seems like everything works ok for the first channel in my list then after that it errors and pulls back no more. Below is the error on each channel:

Channel ABC3 site -- YOURTV.COM.AU -- update mode incremental
Unable to update Channel ABC3

System.NullReferenceException: Object reference not set to an instance of an object.
at WebGrab.Scrub..ctor(Config cnf, ChannelToUpdate ctu, Boolean Silent)
at WebGrab.Program.UpdateChannel(String strIndex, ChannelToUpdate Chan, XmlTarget xTarget)
at WebGrab.Program.Main(String[] args)

Any clues? I am kind of desperate to get this working as my other grabber just died.

Cheers, Justin

 

[WG++Maker]

Hi Jan,

I am trying to get the new included Australian .ini working. It seems like everything works ok for the first channel in my list then after that it errors and pulls back no more. Below is the error on each channel:

Channel ABC3 site -- YOURTV.COM.AU -- update mode incremental
Unable to update Channel ABC3

System.NullReferenceException: Object reference not set to an instance of an object.
at WebGrab.Scrub..ctor(Config cnf, ChannelToUpdate ctu, Boolean Silent)
at WebGrab.Program.UpdateChannel(String strIndex, ChannelToUpdate Chan, XmlTarget xTarget)
at WebGrab.Program.Main(String[] args)

Any clues? I am kind of desperate to get this working as my other grabber just died.

Cheers, Justin

Hi Justin,

I found your problem.
YOURTV.COM.AU uses a new feature that (apparently) was not properly tested. It saves the indexpages of the first channel if the are the same as for the next channels. This saves grab time.
All works ok if you disable this feature. To do that,
1. change in YOURTV.COM.AU.INI the following line:
site {ratingsystem=AU|subtitlestype=ClosedCaption|loadcookie=yourtv.cookie.txt|keepindexpages}
into:
site {ratingsystem=AU|subtitlestype=ClosedCaption|loadcookie=yourtv.cookie.txt}
2. save the changed YOURTV.COM.AU.INI , make sure to save it using utf-8 encoding.

I assume you have done the loadcookie for paychannels !?

WG++Maker .. Jan

 

[redeye19987]

Thanks for your speedy reply Jan!

All is now working as expected. Many thanks for your fix.

Yes I have the cookie file set up. Just on that though, I originally did this through a firefox plugin and when I ran the grab it only picked up certain episodes in the first channel. When I was did the export through Internet explorer however it picked up everything. Do you think that this is relating to the above issue or is it that the firefox export is somehow no good?

Many thanks again!

Justin.

 

[WG++Maker]

Thanks for your speedy reply Jan!

All is now working as expected. Many thanks for your fix.

Yes I have the cookie file set up. Just on that though, I originally did this through a firefox plugin and when I ran the grab it only picked up certain episodes in the first channel. When I was did the export through Internet explorer however it picked up everything. Do you think that this is relating to the above issue or is it that the firefox export is somehow no good?

Many thanks again!

Justin.

Interesting! I can't imagine what caused that. Maybe the site was refreshing the schedules while you grabbed? I think it's unlikely that Firefox cookies are different from the IE ones (they originate from the site as you know). If you have time you could have a look. WG++ filters the cookie file and keeps only the relevant ones for the site. So after first run there are only a few left, so easy to compare.
Another issue related to the yourtv.com.au cookies : Some of them have a rarther short expiry date. (A month from the moment of creation, if I remember well) If WG++ finds one that is expired, it changes this to a later date in the future. I have not been able to test if such a modified cookie is accepted by the site. If not, you have to recreate the cookie file. Pls tell me if that is necessary at that time.

Another small issue: I noticed a leftover debug switch in the ini file. This makes your log file unnecassary long, so better to remove it.
Change:
title.scrub {single(debug)|title="View show information"|">|</a>|</a>}
in:
title.scrub {single|title="View show information"|">|</a>|</a>}


WG++Maker ... Jan

 

[redeye19987]

Hi Jan,

Yes there do seem to be some differences between the IE file and the Firefox file. I ran both exports at similar times. If you like, I can send them to you if you PM me your email?

I have made the .ini file change as you suggested, I did actually think it looked unnecessarily long for each channel.

I also found another bug for you! Channel names that include the ampersand symbol (&) cause the program to error. I have two of them. This will also cause the Guide.xml to error on loading so am wondering if you have included any checks for these special characters that do not work well in an xml? I had this problem in my old grabber HERE

Cheers,

Justin

 

[redeye19987]

I have this now up and running for the full seven days for all available Free to Air and Foxtel channels in Melbourne Australia(153 channels). Works like a charm! For those interested I have attached my config and .ini files to save a little hassle so if you are in Melbourne just delete the ones you don't want.
Also my .ini file is the most up to date and working as of right now as there have been some changes/fixes by Jan.

Thank you to Jan for making this available to Mediaportal users.

Enjoy!

 

[WG++Maker]

I have this now up and running for the full seven days for all available Free to Air and Foxtel channels in Melbourne Australia(150 channels), with the exception of just two, Crime & investigation and Discovery Home & Health due to a character conflict bug I have mentioned in my last post above. Works like a charm! For those interested I have attached my config and .ini files to save a little hassle so if you are in Melbourne just delete the ones you don't want.
Also my .ini file is the most up to date and working as of right now as there have been some changes/fixes by Jan.

Thank you to Jan for making this available to Mediaportal users.

Enjoy!

Hi Redeye,

re the & ampersand character in xml.:
Assuming you have created the channellist file for your region and provider with enabling the two lines for that in yourtv.com.au.ini , you will get the following two channel entries for the one's containing the & :

<channel update="i" site="yourtv.com.au" site_id="Crime &amp; Investigation" xmltv_id="Crime &amp; Investigation">Crime &amp; Investigation</channel>
<channel update="i" site="yourtv.com.au" site_id="Discovery Home &amp; Health" xmltv_id="Discovery Home &amp; Health">Discovery Home &amp; Health</channel>

If you use these entries as such you will get xmltv listing containing lines like this:

..............
<channel id="Crime &amp; Investigation">
<display-name lang="en">Crime &amp; Investigation</display-name>
<url>http://www.yourtv.com.au</url>
</channel>
<programme start="20111222020500 +1000" stop="20111222025500 +1000" channel="Crime &amp; Investigation">
<title lang="en">Motorway Cops</title>
<sub-title lang="en">The Sixth Sense</sub-title>
..............

In fact there is nothing 'illegal' (concerning xml format) and this should be accepted by any xml reader (the &amp; string is the normal substitution any xml writer will produce for the & character) which should convert it back to a sole & character.

If there are problems (may be the xmltv importer of MP doesn't accept this &amp; ?) you can change the channel entries like this:
<channel update="i" site="yourtv.com.au" site_id="Crime &amp; Investigation" xmltv_id="Crime and Investigation">Crime and Investigation</channel>
<channel update="i" site="yourtv.com.au" site_id="Discovery Home &amp; Health" xmltv_id="Discovery Home and Health">Discovery Home and Health</channel>

Replace the &amp; in the xmltv_id and the display_name with 'and' , but leave them in the site_id!! (they are needed as channel recognition in the grabber).

For the rest of the xmltv elements, there is a character filter build into the program that blocks all illegal xml characters, so don't worry there.

Hope this helps WG++Maker .. Jan