My Films 5 - Grabber Scripts/Interface (2 Viewers)

[Dadeo]

My Films 5.x includes many new grabber scripts as well as a new Grabber Interface which allows you to customize options for each grabber, modify existing grabbers, or even create new grabbers to grab data from your favorite movie website.

Grabbers apply only to Ant Movie Catalog (AMC) users who can update their AMC database in My Films.

Please report any issues you find with grabbers or the grabber interface in this thread.

More importantly, please add any modified or new grabbers here!

 

[z3us]

Re: MyFilms 5 - AMC Updater

A bug: in the grabber configuration- detail page, if u press the "web" button after previewing any param, u get a Unhandled exception.
And a question: in this page, using imdb.es scrapper, if u select "description", it shows the complete plot, so I assume a redirection for this field is configurated. Where?
Thanks!

 

[Dadeo]

Re: MyFilms 5 - Grabber Scripts/Interface

Hi Z3us - sorry I keep moving your posts but we are trying to keep AMCUpdater, Grabber Engine and Plugin issues separate.

I can confirm the exception. The Web button works both on Search and Details page. It also works on Search page after previewing a parameter but not after previewing a paramater on Details page (seems to happen on any parameter I tried). Here's the Windows Exception error:
[COLLAPSE]
************** Exception Text **************
System.ComponentModel.Win32Exception: The system cannot find the file specified
at System.Diagnostics.Process.StartWithShellExecuteEx(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start()
at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start(String fileName)
at Grabber_Interface.GrabConfig.btnLoadDetailInWeb_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
[/COLLAPSE]
About the Description/Full Plot - the redirection shows in the Sub Page box beside the parameter - i.e. 'URL Redirection Description'. So then you just preview the parameter for 'URL Redirection Description' and you can see the exact link used to grab the full plot. Is that what you meant?

 

[z3us]

Re: MyFilms 5 - Grabber Scripts/Interface

No problem, Dadeo.
Another things:
- in the grabber configuration- detail page, it seems to not be using the url prefix for getting the url cover, so it results in a relative path. Is that the supposed behaviour?

- And, when u browse for loading a grabber, it always put in the name field: MyFilms\*.xml, and no file is showed. I think it should be * or *.xml

 

[z3us]

Re: MyFilms 5 - Grabber Scripts/Interface

Im creating a scrapper for :: CINeol :: , a spanish site. The main advantage of this site is that it has big spanish covers of recent films. But there is a problem when extracting info with non-english characters.

Example: Matrix

¿La realidad es lo que vemos? ¿Es posible que todo sea un sueño? Neo recibe una noche una extraña visita en su casa que hace preguntarse si está viviendo un sueño. Poco a poco va viendo que todo no es como parece y que en realidad formamos parte de un complejo programa informático cuya única función es tenernos entretenidos para proporcionar energía, estamos dentro de: The Matrix.

imports this:

¿La realidad es lo que vemos? ¿Es posible que todo sea un sueño? Neo recibe una noche una extraña visita en su casa que hace preguntarse si está viviendo un sueño. Poco a poco va viendo que todo no es como parece y que en realidad formamos parte de un complejo programa informático cuya única función es tenernos entretenidos para proporcionar energÃ*a, estamos dentro de: The Matrix.

Is there a easy fix for this?

 

[Guzzi]

AW: Re: MyFilms 5 - Grabber Scripts/Interface

Im creating a scrapper for :: CINeol :: , a spanish site. The main advantage of this site is that it has big spanish covers of recent films. But there is a problem when extracting info with non-english characters.

Example: Matrix

¿La realidad es lo que vemos? ¿Es posible que todo sea un sueño? Neo recibe una noche una extraña visita en su casa que hace preguntarse si está viviendo un sueño. Poco a poco va viendo que todo no es como parece y que en realidad formamos parte de un complejo programa informático cuya única función es tenernos entretenidos para proporcionar energía, estamos dentro de: The Matrix.

imports this:

¿La realidad es lo que vemos? ¿Es posible que todo sea un sueño? Neo recibe una noche una extraña visita en su casa que hace preguntarse si está viviendo un sueño. Poco a poco va viendo que todo no es como parece y que en realidad formamos parte de un complejo programa informático cuya única función es tenernos entretenidos para proporcionar energÃ*a, estamos dentro de: The Matrix.

Is there a easy fix for this?

Could be codepage problems. Actually, this should be set automatically based on the content of the web site. It could be that this is not working, if the headers are missing certain information.
If you post the scraper script, I can have a look at it.

 

[Guzzi]

AW: Re: MyFilms 5 - Grabber Scripts/Interface

No problem, Dadeo.
Another things:
- in the grabber configuration- detail page, it seems to not be using the url prefix for getting the url cover, so it results in a relative path. Is that the supposed behaviour?

iirc, relative path is only used for the jump from search page to first base detail page.
I am open to discussion, if a relative adressing should be supported - I myself always used absolute pathes, as we now support cascaded pages and thus we should be clear, to WHICH page a relatoive adressing should point (always the base page? Or the selected page for the current parameter?)

- And, when u browse for loading a grabber, it always put in the name field: MyFilms\*.xml, and no file is showed. I think it should be * or *.xml

I can confirm - workes fine on XP, but shows the behaviour you describe on Win7. I fixed that for next release. Thanks for reporting.

 

[Guzzi]

AW: Re: MyFilms 5 - AMC Updater

A bug: in the grabber configuration- detail page, if u press the "web" button after previewing any param, u get a Unhandled exception.
And a question: in this page, using imdb.es scrapper, if u select "description", it shows the complete plot, so I assume a redirection for this field is configurated. Where?
Thanks!

Confirmed - there was a check that should only load the preview param if it is a web link, that was not working properly - fixed for next version.
Also removed the exception to be thrown (in case there is an improper content loaded).
Thanks for reporting.

 

[z3us]

Re: AW: Re: MyFilms 5 - Grabber Scripts/Interface

Im creating a scrapper for :: CINeol :: , a spanish site. The main advantage of this site is that it has big spanish covers of recent films. But there is a problem when extracting info with non-english characters.

Example: Matrix

¿La realidad es lo que vemos? ¿Es posible que todo sea un sueño? Neo recibe una noche una extraña visita en su casa que hace preguntarse si está viviendo un sueño. Poco a poco va viendo que todo no es como parece y que en realidad formamos parte de un complejo programa informático cuya única función es tenernos entretenidos para proporcionar energía, estamos dentro de: The Matrix.

imports this:

¿La realidad es lo que vemos? ¿Es posible que todo sea un sueño? Neo recibe una noche una extraña visita en su casa que hace preguntarse si está viviendo un sueño. Poco a poco va viendo que todo no es como parece y que en realidad formamos parte de un complejo programa informático cuya única función es tenernos entretenidos para proporcionar energÃ*a, estamos dentro de: The Matrix.

Is there a easy fix for this?

Could be codepage problems. Actually, this should be set automatically based on the content of the web site. It could be that this is not working, if the headers are missing certain information.
If you post the scraper script, I can have a look at it.

Attached is the script. Its almost finished, only 2 problems:
- Special characters, as I commented
- Importing actors and roles. I will keep trying

I also upload a modified imdb.es script with working import of tagline. I suppose a similar modification should be done with the other localizations of imdb

 

[Guzzi]

AW: Re: AW: Re: MyFilms 5 - Grabber Scripts/Interface

Attached is the script. Its almost finished, only 2 problems:
- Special characters, as I commented

I debugged it and can confirm, that the automatic encoding detection for this website does not work. The content requires UTF-8.
I have made a fix for next version that should cope with that.

- Importing actors and roles. I will keep trying

To grab content with "match groups" (like persons with roles), the Replace field needs the setting "#REGEX#". I know, it's not well documented yet - attached changed script that properly pulls actors with roles.

I also upload a modified imdb.es script with working import of tagline. I suppose a similar modification should be done with the other localizations of imdb

As far as I have seen, there is no tagline on the localised IMDB pages. So either a change like you've done is required to get it - or taglines just don't get added.
Just keep in mind, that each redirection page that is defined means loading another webpage for retrieving data for a movie.
So this will slow down the grabbing. I have no personal preference with that - so other users using localized scripts might comment on that.
However, I updated IMDB.ES with the one you've sent.