My Films 6 - Grabber Scripts/Interface (3 Viewers)

[eRJe]

I believe IMDB made some changes again? When importing or updating new movies, the cover art doesn't download anymore. I works when I manually update the movie cover art within MP and select a different source.

btw, I use the latest IMDB-full grabber script.

 

[Dadeo]

I can confimr that eRJe - I tried to fix it but it's beyond me have to wait for @Guzzi
I also wasn't able to fix the formatting (extra spaces) when there are multiple Directors (e.g. The Way Way Back)

 

[Guzzi]

Will be fixed soon, sorry for the delay.

 

[Dadeo]

@eRJe @Robert123 and everyone missing covers using IMDB grabbers! Thanks to Guzzi's help/instructions I figured out the cover image, so I updated IMDB.xml and IMDB-Full.xml and they should grab covers again now. Also fixed the formatting (extra spaces) when there are multiple Directors. You should be able to update via the Update Grabber Scripts button on the Import Tab in Setup.

I'm working on learning and updating other grabbers as well, but it may take some time. I'll post here again when updates are available.

 

[Dadeo]

UPDATE YOUR GRABBERS IN SETUP ON THE IMPORT TAB!

I have updated most of the 36 grabber scripts now (except OFDB related grabbers which Guzzi will do soon ). It was shocking how many were partially or totally broken and no one reported an issue. It makes me wonder - is anyone is using them? Should we keep maintaining them?

Obsolete Grabbers:
Some grabbers have been removed:

• All the 'localized' IMDB grabbers since those sites have been discontinued - it's been almost a year and IMDb still has not implemented localized data.
• TMDB v3 grabbers (DE, EN, ES) since all TMDB grabbers have been updated to v3 API now
• IMDB-TMDB cover has been replaced by TMDB-Covers using v3 API - and can be used in Cover Manager to download multiple covers from TMDB until Guzzi has updated the internal grabbing to v3 API
• TMDB-DE-Person has been renamed to TMDB-Person and set to English since TMDB does not provide localized data for persons yet.
• Media-Passion - has changed sites and now requires registration and credits to use
• World of Video - closed their online shop on 30.06.2013.

If you have any of the above set as your default grabber, please update your Setup.
Guzzi is working on a method to automatically move obsolete grabbers to the backup folder when you update them. For now, you can manually move them from your ProgramData\Team MediaPortal\MediaPortal\scripts\MyFilms folder to the backup folder if you don't want them to appear in MyFilms Setup or GUI.

IMDB/IMDB-Full
I have changed IMDB to use IMDB Mobile for searching as it is much faster (IMDB-Full still uses akas.imdb.com). That makes IMDB grabber particularly useful for 'quick' updates by field in GUI! The main differences between IMDB and IMDB-Full now are, IMDB-Full provides:

• more search results for 'rare' films
• Translated Titles/AKAs using Grabber override options - though IMDB should grab the correct AKA/Title for your country (if there is one) based on your IP address
• Producers
• More actors (and # can be selected via Grabber options)
• Certifications - can be selected via grabber options, though IMDB should grab the correct Certification for your country (if there is one) based on your IP address

New Grabbers:

• MoviePosterDB - multi-covers - accessible in Cover Manager - though it doesn't grab all posters from 'groups' yet.
• AdoroCinema - Portuguese
• Filmweb - Polish
• Kinopoisk - Russian
• Moviemeter - Dutch
• TMDB-NL/PT/RU localized for Dutch, Polish and Russian

Please note: TMDB v3 API does not automatically fall back to English. Guzzi is working on a method to solve that. Hopefully TMDb's localized data will improve over time. You can improve it (for all media centres and movie catalogs) by adding missing data to TMDB!

I have updated the sortable table for Supported Grabbers in wiki with the latest changes.

If there are any other grabbers, or improvements you would like to see, please post them here. Thanks!

 

[ltfearme]

TMDB v3 API does not automatically fall back to English. Guzzi is working on a method to solve that.

IMO you should just leave it as is, it will encourage more people to update online with their native language

 

[Dadeo]

Yep - I always try and encourage everyone to do that so we can be less dependent on IMDb TMDb API is so much better and easier to work with, and with backdrops/localized posters all in one place, it is much faster.

However I may be a bit biased since I use the English data

 

[Dadeo]

NFO Grabber Update:
I have updated the NFO grabber for MyFilms, to make it more generic, and support different NFO formats, not just EmberMediaManager. To get it just Update Grabbers on the Import Tab in Setup.

I have done BASIC testing using NFO files saved by the following apps:

• Ember Media Manager (EMM) 1.3.0.18
• tinyMediaManager (TMM) 2.5.1 and 2.6 (not yet released)
• Media Companion (MC) 3.5.9
• Media Center Master (MCM) 2.1.0 (free version) using XBMC NFO format
• MediaElch 2.0.1

if you use any of the above apps, please test with the updated NFO grabber!
There may still be some minor issues, because there are so many different options available I couldn't survive testing every possible variation.
Just create a new config using Setup Wizard (AMC Catalog type, 'use nfo grabber' selected) and see if you get better results/more data than with the previous NFO grabber.

The purpose of the NFO grabber is (like MyFilms itself), to allow you to use the movie catalog/media manager app of your choice as a database back end. The NFO grabber is NOT intended to convert from other plugins/media centers. However I did test with:

• MP Videos - 1.5.0 - 'standard' NFO tags are supported but not all the extra tags used by MP Videos.
• MovPicNFO - 1.6.0 - it does not conform 100% to XBMC NFO standard - e.g <writer> not <credits>, AKAs saved to <originaltitle>, <studios> not <studio>, and multiple occurences separated by a | (even at the beginning of a field/tag). So, you may get better results using the MovPics XML plugin and catalog option in MyFilms.

Changelog
Here are the major changes to the previous NFO grabber:

• Director(s) - multiple directors are grabbed even if they are in separate occurrences/tags.
• Producer(s) - grabbed if your media manager supports it - only if the <credits> data (which is usually only Writers) indicates it is a producer. Producers will be grabbed from TMM v2.6.
• Writer(s) - multiple writers are grabbed now even if they are in separate occurrences/tags.
• Language(s) - Spoken Languages are grabbed using TMM 2.6. If you want language from the media info (e.g. audio tracks) select that option in AMCU DB Fields tab.
• Certification - preferred - grabs only the actual rating (not the country) to support the display of certification logos. If your media manager does not write <certification> to NFO, then <mpaa> data is grabbed, but it may include text depending on the media manager/options you choose.
• Studio(s) - all occurrences of Studio are grabbed now even if they are in separate occurrences/tags.
• IMDB Top250 (Rank) - grabbed from NFO files that support it. Note: data only exists IF the movie is in IMDB's Top 250
• TMDB_ID - grabbed only if the NFO file supports it, rather than grabbing the ID which is usually the IMDB ID
• MultiFanart -only \extrafanart is grabbed now, not \extrathumbs as they are often duplicates.
• Runtime - now grabbed from NFO if it exists and no runtime already exists (i.e. has not been imported from Media Info)

IMPORTANT Notes:

• NFO grabber supports the following filenames:
  • <movie filename>.nfo - (note: for multi-part movies that means moviename-CD1.nfo where the first filename is moviename-CD1.avi)
  • <movie filename>.jpg - poster/cover (although you can choose folder.jpg in AMCU DB fields settings)
  • <movie filename>-fanart.jpg for backdrop/fanart (although it also supports images in \extrafanart subfolder).
• Trailers are not grabbed from NFO because most apps save only a trailer URL to NFO, not local trailer path/filenames.
• Collection Covers are not grabbed because there are still so many variations in how they are stored by media managers. However, you can easily add Collection Covers in MyFilms 6.1 (or 6.0.3 in TestMode) in GUI > Updates > Download Collection Cover, if you use a skin that supports the latest features! (i.e. includes MyFilmsDialogImageSelect.xml)
• MediaInfo is not grabbed from NFO files (<fileinfo> tags), since, like most apps, MyFilms already imports it via MediaInfo.

I've added some feature requests to MyFilms Issue Tracker, and Guzzi is continuing to improve the Grabber Engine so that we can do even more with NFO grabber in future.

If you need to find the above info in future, I added it to the Wiki > Grabber Scripts > NFO Grabber.

Any questions?

 

[bwg]

if you use any of the above apps, please test with the updated NFO grabber!

Thanks Dadeo! I use EMM 1.3.0.18 and have just tested with the updated NFO grabber.

All seems to work well, the only difficulty I still have is that when no "Original Title" is added in Ember, it re-uses the title in the <originaltitle> tag in the .nfo (moving "A, The, Le, etc. to the end where applicable). This is redundant info - I only want an Original Title when there is an alternate title (tanslations, alternates for different countries, etc.). I realise this is an issue with Ember, but what makes it worse is that the grabber prefers the "Original Title" parameter, so I end up with e.g. "Counselor, The" going in as my "Original Title" in the amc database. I prefer it to read "The Counselor" and let AMC / MyFilms intelligently manage the switch to "Counselor, The" using the Formatted Title.

Apart from that, there are 3 customisations that I made;

• The Ember nfo contains the Plot Outline and the Plot. I like to have both of these, but the default grabber just looks for plot. So I amend the "Description" parameter to look for the <outline> and the "Comment" parameter to look for the <plot>. There is no comments tag in the Ember .nfo (note: this is just a personal preference).
• I like to add the tagline to the start of the "Description" (i.e. plot outline, see point above), it seems to fit well when browsing movies
• with multifanart, I have switched it back to look for the "extrathumbs", as my entire collection has been scraped with Ember using this format, and it would take a loooong time for me to rescrape all images in local folders back to "extrafanart" (it is a one-at-a-time process in Ember unfortunately, no mass rewrite option)

Great work to strip the country from Certification, it is clever to utilise MPAA if <certification> tag is blank. I appreciate this as certification for Australia is often not available when I am scraping.

 

[Dadeo]

Thanks for the feedback bwg - it's really great to know it is working well for you in a real world/production environment.

Re Original Title - it's not just EMM The problem is that many apps don't use Translated Titles or AKAs (it's not defined in XBMC NFO standard ), thus <title> could be a localized title or the original title. Sometimes <title> and <originaltitle> are the same, sometimes <originaltitle> is only written if different.... MF NFO grabber (or author ) has no way to know wtf is written where!

Since MyFilms let's you select either Original Title or Translated Title (<title> in NFO is the closest) as your main (master) title, it means you MUST have data in the title field you select for every record. So that's why the NFO grabber User Options maps OriginalTitle to Translated Title if there isn't one already. Did you try removing that option to see if it helps?

I only want an Original Title when there is an alternate title (tanslations, alternates for different countries, etc.)

So I assume you use Translated Title as your main (master) title in MF config? If you use Original Title, and some EMM NFOs don't have one, those movies won't import right? But how do you know if the <title> is an alternate title/translation or the actual original title?

If you can attach an example or two of a problematic NFO where you get duplicate and/or incorrectly formatted titles. I can see what we might be able to do.

Re Plot Outline - yeah AMC (thus MF) doesn't support that field, so your Desc/Comments workaround is a good solution for now.
Re Tagline - yes it is nice to display it at start of Desc. We used to do that in MF before we had an extended field in AMC for tagline. In theory you could do that in Skin - but not easy to format it with a LF after
RE - multifanart - Crap! I missed that. I know EMM has an option (Scrapers - Images & Trailers) to use either \extrafanart or \extrathumbs or both. I just didn't think about the use case where you select only \extrathumbs. DUH! You could probably change it and rename \extrathumbs to \extrafanart (so you don't have to download images again) - but your solution is easier. In future I think we can add an options to grab from \extrathumbs and map it to multifanart field if it doesn't already have images from \extrafanart.

Nice to know you have managed to use the Grabber Editor so well to customize the NFO grabber for your preferences/setup!

I appreciate this as certification for Australia is often not available when I am scraping.

You may have seen in MF Issue Tracker I added a feature request: Issue 365: Certification logos by country to extend the Logos manager so it displays the correct logos for your country. Since MF already allows you to select the country you want to use for Certification, it only makes sense to extend that to logos imo. Not the point you were making I know - doesn't help if you can't grab Aussie Certs!