FilmInfo+ - A german movie details scraper with auto grouping (3 Viewers)

[xadox]

Wie immer besten Dank für das schnelle Update Merlyn

 

[RoChess]

Wenn ich einen Rescan von bereits hinzugefügten Filmen mache, werden lediglich der Score und die Popularity Werte aktualisiert. Ich habe dann von Zeit zu Zeit einfach alle Filme markiert und einen Rescan gemacht. So hatte ich immer die aktuellsten IMDB Bewertungen.

Das ist eine Funktion von MovingPictures, und nicht vom Script selbst. Ein Scraper Script besteht aus drei Teilen, einem für die Filmsuche, einem für die Filminfos und einem optionalen dritten für das Filmposter. Beim Refreshen wird nur der Teil für die Infos neu ausgeführt. Man kann nicht einstellen, welche Infos dabei aktualisiert werden sollen und welche nicht. Aufgrund des Umfangs von FilmInfo+ werden hierbei so gut wie alle Informationen neu geladen. Daran kann ich nichts ändern.

What they might be after, is what I did with IMDb+ as well, provide a scraper-script option that only allows certain fields to be updated (votes, popularity, certification and empty fields). It has a small speed benefit as well, but the main reason is to not loose any customisation, such as manual changes to title/etc.

Moin.
Gibt's eigentlich schon Pläne dies super Script in MP2 zu "integrieren"?

Nein. Ich bezweifle auch, dass fforde und die anderen Entwickler das jemals auch nur in Erwägung ziehen werdem, da sie dann dafür zuständig wären, den Scraper Up to Date zu halten. Die guten Jungs haben ohnehin schon genug zu tun

There is more then likely not going to be a Moving-Pictures for MP2 (hardly a need for it, since majority of function is added to MyVideos). I am however going to be looking at how MyVideos on MP1.5/MP2 handles imports and see if I can adjust IMDb+ to be compatible. Will let you know, so you can use the groundwork for FilmInfo+. My new HTPC is finally done (Win 8.1, MePo 1.5, etc), and is currently running finaly disParity update, so I will be able to have a closer look at the inner workings.

If I have to rewrite the IMDb+ plugin to make it work, I will look at allowing FilmInfo+ to be part of it, so you will have the power of plugin with auto-scraper updates at your disposal as well. I figured you would fork the IMDb+ plugin by now, but C# is not for everbody and without the help I got for mine it would have never happened for me either.

This is more Christmas holiday or early next year type of timeline though, it has been a very busy year for me, so need to catch up on some projects around the house and family time as well.

 

[Merlyn]

Die HTML Tags im Writersfeld tauchen bei mir nicht auf. Ich konnte das bislang nicht reproduzieren. Verwendet ihr eventuell eine ältere Version des Scrapers?[DOUBLEPOST=1382774131][/DOUBLEPOST]

What they might be after, is what I did with IMDb+ as well, provide a scraper-script option that only allows certain fields to be updated (votes, popularity, certification and empty fields). It has a small speed benefit as well, but the main reason is to not loose any customisation, such as manual changes to title/etc.

There is more then likely not going to be a Moving-Pictures for MP2 (hardly a need for it, since majority of function is added to MyVideos). I am however going to be looking at how MyVideos on MP1.5/MP2 handles imports and see if I can adjust IMDb+ to be compatible. Will let you know, so you can use the groundwork for FilmInfo+. My new HTPC is finally done (Win 8.1, MePo 1.5, etc), and is currently running finaly disParity update, so I will be able to have a closer look at the inner workings.

If I have to rewrite the IMDb+ plugin to make it work, I will look at allowing FilmInfo+ to be part of it, so you will have the power of plugin with auto-scraper updates at your disposal as well. I figured you would fork the IMDb+ plugin by now, but C# is not for everbody and without the help I got for mine it would have never happened for me either.

This is more Christmas holiday or early next year type of timeline though, it has been a very busy year for me, so need to catch up on some projects around the house and family time as well.

Thanks a lot, RoChess! For a Plugin like yours I would actually have to learn C# first, and with other hobbies, work and this strange real life thingy I just dont have the time for that. So I basically have boxed the FilmInfo+ Plugin idea.

What do you mean by "provide a scraper-script option that only allows certain fields to be updated"? How did you do that?

 

[Helios61]

Die HTML Tags im Writersfeld tauchen bei mir nicht auf. Ich konnte das bislang nicht reproduzieren. Verwendet ihr eventuell eine ältere Version des Scrapers?

Nein, die aktuelle Version!

Gruß
Helios

 

[RoChess]

What do you mean by "provide a scraper-script option that only allows certain fields to be updated"? How did you do that?

If you look at IMDb+ source, you will see I have the scraper-script option: "global_options_refresh_all_fields"

This controls if all fields get updated, or just some. It is a simple a boolean flag (true/false), and then inside the rest of the script I check that value with simple <if test="${global_options_refresh_all_fields}=true"> check to decide if I allow that block of code to be executed. Because I also need to check if the existing value is empty (for a fresh import or when that info gets updated later) and there is no ADN/OR support for scraper-scripts, I came up with the following solution:

 <!-- Verify conditional update. There is no AND or OR support, so this will have to do -->
 <if test="${movie.directors}="><set name="update_directors" value="true" /><set name="new_import" value="true" /></if>
 <if test="${global_options_refresh_all_fields}=true"><set name="update_directors" value="true" /></if>
 <if test="${update_directors}=true">..... (do the actual update of "Directors") ....</if>

The "Directors" info does not exactly change much once you initially scraped this info, same with the remaining crew/title/etc. Fields I always want to update, such as certification (movie ratings do change over time), score+votes, etc, I just skip the check and use the old code as-is.

The ${new_import} boolean flag is probably not needed for you, but it was needed to fix a bug for IMDb+.

Keep in mind to document it properly. I made the mistake of making the default value of IMDb+ to not update all the fields (which is what I use myself), but this caused a lot of complaints from users that changed configuration options that should have caused a different title, but then the title field would not update.

 

[Merlyn]

Nice workaround.

 

[Cyrus1896]

What do you mean by "provide a scraper-script option that only allows certain fields to be updated"? How did you do that?

If you look at IMDb+ source, you will see I have the scraper-script option: "global_options_refresh_all_fields"

This controls if all fields get updated, or just some. It is a simple a boolean flag (true/false), and then inside the rest of the script I check that value with simple <if test="${global_options_refresh_all_fields}=true"> check to decide if I allow that block of code to be executed. Because I also need to check if the existing value is empty (for a fresh import or when that info gets updated later) and there is no ADN/OR support for scraper-scripts, I came up with the following solution:

 <!-- Verify conditional update. There is no AND or OR support, so this will have to do -->
<if test="${movie.directors}="><set name="update_directors" value="true" /><set name="new_import" value="true" /></if>
<if test="${global_options_refresh_all_fields}=true"><set name="update_directors" value="true" /></if>
<if test="${update_directors}=true">..... (do the actual update of "Directors") ....</if>

The "Directors" info does not exactly change much once you initially scraped this info, same with the remaining crew/title/etc. Fields I always want to update, such as certification (movie ratings do change over time), score+votes, etc, I just skip the check and use the old code as-is.

The ${new_import} boolean flag is probably not needed for you, but it was needed to fix a bug for IMDb+.

Keep in mind to document it properly. I made the mistake of making the default value of IMDb+ to not update all the fields (which is what I use myself), but this caused a lot of complaints from users that changed configuration options that should have caused a different title, but then the title field would not update.

Genau das ist es, was ich in meinem vorherigen Post meinte. Beim erneuten Scannen von bereits hinzugefügten Filmen eben nur gewisse Felder zu updaten.

Score / Popularity / FSK (eventuell sinnvoll, damit man mal ein drchgängiges System hat, wenn man bereits mit mehreren Scrapern gearbeitet hat.)

Das wäre wirklich Wahnsinn, wenn das in diesem Scraper funktionieren würde.

 

[Merlyn]

Ich muss schauen, in wie weit ich das wie von RoChess beschrieben umsetzen kann. Das geht leider nicht mal eben zwischendurch.
So wie es bei mir zeitlich im Moment aussieht, wird das bis zum neuen Jahr warten müssen.

Falls sich sonst jemand von euch damit auseinander setzen und den Scraper updaten möchte, nur zu! Ich bin für jede Hilfe dankbar

 

[badboyxx]

Kann man das plugin auf die neuen Funktionen von MovPic 1.6 updaten oder ist es ein größerer Aufwand.
- Collections
- Plot keywords
uvm.

 

[Data45000]

In den Filmen von Star Trek III, IV und co gibt es bei den Summarys immer "Quelle: CovertextBlablaba" Sieht nicht schön aus.
Ebenso bei:
Wiki auf großer Fahrt

Weitere Frage, wie kommt es, dass bei der Sprache immer eignartige Sprachen eingetragen sind. Bei Star Trek Deutsch English dann Chinesich oder Schwedisch... ?