this view could not be built (1 Viewer)

[MJGraf]

I would have thought there would be one for Each Type of Media....

Nope, there is always only one Importer.
There may be multiple Imports running in parallel if you add multiple MediaSources.
But even if you select all MediaCategories for one MediaSource, there is still only one Import - the import just applies more MDEs if you check more MediaCategories

 

[MJGraf]

@JSchuricht : The code is anything but clean, yet (and therefore not yet pushed to GitHub), but I need a guinea pig
Would you be so kind and test if the attached binary works for your use case? To install it:

• Make a backup of your database - this is test code and anything might go wrong
• Stop MP2-Server service
• Make a backup of [Mp2-Server-Program-Directory]\plugins\NfoMetadataExtractors\NfoMetadataExtractors.dll
• Replace the file you just backed-up with the one contained in the 7z-file attached to this post
• Restart MP2-Server
• Add a (movie)MediaSource and see how the import works...

As usual - anyone else if welcome to test as well. It should now be sufficient to have any kind of text file as nfo-file, which at any place contains a valid IMDB-ID for a movie...
Thanks!

 

[JSchuricht]

So there is good and bad with the new nfo extractor.

The good: Sweeney Todd shows up in movies.

The bad: A lot fewer movies showed up after the import.

I ran the import 3 times, questioning the number I wrote down on the first one for movies but 2608 on first import. Second and third were 2607, videos shows 3488 and my original MP2 setup is showing 3071 in movies. This is all using the same source with the same credentials.

So my basic structure is a SMB share called movies, sub folder A-Z, movie folder named after IMDB name, movie and nfo inside. A lot of movie files aren't named well and many nfo files are simply called imdb. So far every movie I have checked that didn't show up in movies has a nfo with a different name than the movie and a badly named movie file but resorts to looking up by folder name where it fails.

I think there are two main issues I am fighting right now. First the nfo's need to be named the same as the movie file vs just existing in the same folder, not quite sure how that will workout with a raw DVD rip containing a VIDEO_TS folder. Second, I think something changed that has reduced accuracy in the scrapper. Perhaps it is the addition of month and day as breese pointed out. That may explain how even more movies are missing and the weirdness I saw on 10/23 when I started looking at the item counts.

 

[MJGraf]

First the nfo's need to be named the same as the movie file vs just existing in the same folder

It doesn't need to be named as the movie name. It can also be named "movie.nfo". If the names of your nfo-files are not completely random, please do the following:
Stop MP2-Server Service.
Go to C:\ProgramData\Team MediaPortal\MP2-Server\Config
Open the file MediaPortal.Extensions.MetadataExtractors.NfoMetadataExtractors.Settings.NfoMovieMetadataExtractorSettings.xml
There is a section in it looking like this:

  <Property Name="MovieNfoFileNames">
  <ArrayOfString xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <string>movie</string>
  </ArrayOfString>
  </Property>
  <Property Name="NfoFileNameExtensions">
  <ArrayOfString xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <string>.nfo</string>
  <string>.xml</string>
  <string>.txt</string>
  </ArrayOfString>
  </Property>

Besides the movie name + .nfo, MP2 will look for any combination of names and extensions from this file. So if your nfo-files are called "imdb.nfo", just add below <string>movie</string> a new line with <string>imdb</string>.

While you have this file open, please also set these three settings to true:

  <Property Name="EnableDebugLogging">
  <boolean>true</boolean>
  </Property>
  <Property Name="WriteRawNfoFileIntoDebugLog">
  <boolean>true</boolean>
  </Property>
  <Property Name="WriteStubObjectIntoDebugLog">
  <boolean>true</boolean>
  </Property>

This will on the next import generate a new (and rather extensive) NfoMovieMetadataExtractor.log file in your log directory, which will certainly help to figure out the rest...

Then save the config file, restart MP2-Server remove the old MediaSource, add it again and wait until the import is finished. After that, please post the result and the logs.
Thanks for your help!
Michael

 

[MJGraf]

As to the matching based on movie name, I have found at least one movie name that currently doesn't work.
We use a regular expression on the full path of a movie file to match movie title and year. The regular expression we use is:

(?<title>[^\\|\/]*?)\s*[\[\(]?(?<year>(19|20)\d{2})[\]\)]?[\.|\\|\/]*

If you apply this to the following path:

\\BOOMER\Movies\#\1911 (2011)\abd-1911-1080p.mkv

(which you can test online here: https://regex101.com/ )
you can see that the movie name match is empty. This is because the title part : "[^\\|\/]*?)" is "lazy" and accepts zero characters. I first thought about making it "greedy" instead of "lazy", but I suppose this is intentionally made "lazy" so that if there are two four-digit numbers starting with 19 or 20, we don't have any of them in the title and only the first one in the year.
But admitting zero character titles doesn't make sense to me. If we replace "*?" with "+?" so that the title must have at least one character, the path above matches correctly. Regular Expression would then look like:

(?<title>[^\\|\/]+?)\s*[\[\(]?(?<year>(19|20)\d{2})[\]\)]?[\.|\\|\/]*

@morpheus_xx : Any thoughts?

 

[morpheus_xx]

Any thoughts?

Yep: I you fix one case, the probability is high to break another

So if you do modifications, please let the Unit tests run for this part (maybe some fail already ). In future the regexp are planned to be configurable via settings, so it could be worth to implement this now if modifications are required.

 

[MJGraf]

In future the regexp are planned to be configurable via settings, so it could be worth to implement this now if modifications are required.

Great minds think alike

 

[Lehmden]

Hi.
I lately have found another issue... If a movie is available on TheMovieDB (our primary source) but did not have a "tt" number the importer did not know what to do. No matter if the TMDB-ID is available in .nfo and/or matroska tags the identification often went wrong...

As Example... I have a concert movie "Roxette Live Travelling the World" from 2003 that is available on TMDB with ID "249407". This ID is stored in .nfo and matroska tags. But this concert movie isn't listed at IMDB, for this it did not have a tt number. Nobody knows why (at least I don't know), but the importer likes to recognise this as the movie "Fanny Hill" from 1983 with "tt0087237" and TMDB-ID "58843"...
I've attached the .nfo and the mediainfo output so you can see that the TMDB-ID is properly tagged into matroska tags too...

 

[MJGraf]

If a movie is available on TheMovieDB (our primary source) but did not have a "tt" number the importer did not know what to do. No matter if the TMDB-ID is available in .nfo and/or matroska tags the identification often went wrong...

The answer is as easy as sad...
The NfoMovieMDE does read the <tmdbid>, <tmdbId> and <thmdb> tags from nfo-files which may all contain the TheMovieDb-ID and stores it in MovieAspect.ATTR_TMDB_ID.
But the MovieMDE (via OnlineLibraries) does not use MovieAspect.ATTR_TMDB_ID to find the movie online. It only uses the IMDB-ID, if present, or title and year if there is no IMDB-ID.
The good news is that it shouldn't be too complicated to make it use the TmdbId for matching if any of the MDEs applied before has found one.
In the link above, we need in addition to the MatchByImdbId-method a "MatchByTmdbId"-method.
MatchByImdbId calls TheMovieDbWrapper.GetMovie(string imdbId, out Movie movieDetail), which online searches by IMDB-ID. But there is already a TheMovieDbWrapper.GetMovie(int id, out Movie movieDetail) method, which searches online by Tmdb-ID.
So the changes should be limited to MovieTheMovieDbMatcher.
I have no idea why this wasn't implemented before, but I suppose that when the MovieMDE was written, there were not too many ways to find such IDs, because the NfoMDEs were not yet there and IIRC also the Matroska-Tag-Reader was added later. And finding a Tmdb-ID by way of a string operation on the file or folder name is likely to fail, because it is just a number - without any identifying prefix like "tt".
Anyway - I will see what I can do...

EDIT: MatroskaMatcher currently also only supports IMDB-ID - not TMDB-ID. But MatroskaReader, which is used by MatroskaMatcher, can read TMDB-ID from MKV-Files. So supporting this one as well (in case the TMDB-ID is tagged in the mkv-file, but there is no nfo-file) shouldn't be too complicated...

 

[Lehmden]

Hi.

I will see what I can do...

Thanks...
I believe this is not a very seldom use case especially in future. TheMovieDB is growing rapidly and it's free. IMDB is commercial and as far as I know you need to pay for becoming a community member... I never added any data to IMDB, but often do so on TheMovieDB. And I'm not the only one who does the same... So I believe there will be more and more movies that are not listed on IMDB but listed on TheMovieDB... Not the mainstream stuff, this always will have a tt number for sure, but for the more exotic or special movies this situation will become more common, I'm sure...
A similar development can be seen in series too. TVDb has lots of restrictions what kind of show to add. TheMovieDB is way much more "tolerant" here so the number of TV shows that are available on TheMovieDB but not on TVDB will constantly grow too, I'm sure...