[further infos missing] Linked folders not importing (1 Viewer)

[sridharb]

Hi, great timing. I just hit the breakpoint in VS. But interesting things to report. When I created the test directory with one symbolic link, it automatically imported it (I was running the usual server, not the debug version). After it finished, I saw that it successfully created the movie. Now, the problem becomes harder. First I have to understand by going through the code why it succeeds in this test scenario and then try to understand possibly why it fails in the other scenario where it has multiple movies to import and symlinks (looks like only sometimes) fail.

 

[sridharb]

Here are my observations after debugging:

The breakpoint that you suggested seems to be hit by multiple threads which makes debugging difficult - possibly one worker thread per filesystem item (like a file). That apart, I put a breakpoint in line 191. It didn't hit that. But that could be because of some other issues - please see log below:

[2019-06-27 10:53:19,393] [66779 ] [39 ] [INFO ] - ImporterWorker: Scheduled ImportJob(#1) (Activated) (Path ='{e88e64a8-0233-4fdf-ba27-0b44c6a39ae9}:///H:/TestSymLink/', ImportJobType='Refresh', IncludeSubdirectories='True')
[2019-06-27 10:53:19,489] [66875 ] [37 ] [INFO ] - ImporterWorker.ImportJob(#1).DirectoryUnfoldBlock: Successfully processed 3 MediaItems; time elapsed: 00:00:00.0930814; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 10:53:19,517] [66903 ] [42 ] [INFO ] - ImporterWorker.ImportJob(#1).DirectorySaveBlock: Successfully processed 3 MediaItems; time elapsed: 00:00:00.0196919; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 10:53:19,646] [67032 ] [41 ] [INFO ] - ImporterWorker.ImportJob(#1).FileUnfoldBlock: Successfully processed 58 MediaItems; time elapsed: 00:00:00.0479595; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 10:53:20,148] [67534 ] [39 ] [INFO ] - ImporterWorker.ImportJob(#1): Activated
[2019-06-27 11:00:09,276] [476662 ] [96 ] [ERROR] - Warn.: UPnP: Could not enter write lock. Caller: at UPnP.Infrastructure.WriteContext..ctor(ReaderWriterLockSlim rwLock, Int32 timeout)
at UPnP.Infrastructure.ReaderWriterLockExtensions.EnterWrite(ReaderWriterLockSlim _lock, Int32 maxMs)
at UPnP.Infrastructure.CP.SSDP.SSDPClientController.HandleNotifyPacket(EndpointConfiguration config, IPEndPoint remoteEndPoint, HTTPVersion httpVersion, String date, String cacheControl, String location, String server, String nts, String usn, String bi, String ci, String sp, String st, String& error)
at UPnP.Infrastructure.CP.SSDP.SSDPClientController.HandleNotifyRequest(SimpleHTTPRequest header, EndpointConfiguration config, IPEndPoint remoteEndPoint)
at UPnP.Infrastructure.CP.SSDP.SSDPClientController.HandleSSDPRequest(SimpleHTTPRequest header, EndpointConfiguration config, IPEndPoint remoteEndPoint)
at UPnP.Infrastructure.CP.SSDP.SSDPClientController.OnSSDPMulticastReceive(IAsyncResult ar)
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Net.ContextAwareResult.CompleteCallback(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.ContextAwareResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

[*Lot more of this warning*]

[2019-06-27 11:07:17,537] [904923 ] [125 ] [INFO ] - ImporterWorker: 86 % completed (1 ImportJob(s), in total 50 of 58 so far identified resources processed)
[2019-06-27 11:07:21,574] [908960 ] [125 ] [INFO ] - ImporterWorker.ImportJob(#1).MediaItemLoadBlock: Successfully processed 58 MediaItems; time elapsed: 00:14:01.8698120; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 11:07:22,120] [909506 ] [112 ] [INFO ] - ImporterWorker.ImportJob(#1).MetadataExtractorBlock: Successfully processed 52 MediaItems; time elapsed: 00:14:02.3007820; MaxDegreeOfParallelism(InnerBlock) = 40
[2019-06-27 11:07:22,121] [909507 ] [37 ] [INFO ] - ImporterWorker.ImportJob(#1).MediaItemSaveBlock: Successfully processed 0 MediaItems; time elapsed: 00:14:02.1925243; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 11:07:22,122] [909508 ] [112 ] [INFO ] - ImporterWorker.ImportJob(#1).RelationshipExtractorBlock: Successfully processed 0 MediaItems; time elapsed: 00:14:02.0852744; MaxDegreeOfParallelism(InnerBlock) = 40
[2019-06-27 11:07:22,125] [909511 ] [37 ] [INFO ] - ImporterWorker.ImportJob(#1): Completed
[2019-06-27 11:07:22,127] [909513 ] [37 ] [INFO ] - ImporterWorker.ImportJob(#1): Disposed

Now, compare that with the log from the time that the server "automatically" imported it upon creation of the media source:

[2019-06-27 10:33:15,681] [61150709] [86 ] [INFO ] - Media item 5ce11987-4ced-443b-916d-391ed62476d3 with name TestSymLink (TestSymLink) imported (36 ms)
[2019-06-27 10:33:15,694] [61150722] [83 ] [INFO ] - Media item 549ab6d7-64fb-4f68-b17a-9fc24f9c24f1 with name 001 - Seven Samurai (001 - Seven Samurai) imported (8 ms)
[2019-06-27 10:33:15,702] [61150730] [83 ] [INFO ] - Media item f0f8e425-ebc7-4fa1-acfc-fe53d016c547 with name .actors (.actors) imported (6 ms)
[2019-06-27 10:33:15,703] [61150731] [92 ] [INFO ] - ImporterWorker.ImportJob(#1).DirectorySaveBlock: Successfully processed 3 MediaItems; time elapsed: 00:00:29.8557333; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 10:33:15,736] [61150764] [114 ] [INFO ] - ImporterWorker.ImportJob(#1).FileUnfoldBlock: Successfully processed 58 MediaItems; time elapsed: 00:00:29.7813029; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 10:33:15,740] [61150768] [179 ] [INFO ] - ImporterWorker.ImportJob(#1).MediaItemLoadBlock: Successfully processed 58 MediaItems; time elapsed: 00:00:29.6774798; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 10:33:16,815] [61151843] [169 ] [INFO ] - PluginManager: Plugin 'GenreProvider' (id '367c772e-dfc5-4be1-b292-7fb6a2d08097') activated.
[2019-06-27 10:33:17,905] [61152933] [162 ] [INFO ] - OCVVideoThumbnailer: Successfully created thumbnail for resource 'H:\TestSymLink\001 - Seven Samurai\Seven.Samurai.1954.redacted3.avi'
[2019-06-27 10:33:17,982] [61153010] [169 ] [INFO ] - OCVVideoThumbnailer: Successfully created thumbnail for resource 'H:\TestSymLink\001 - Seven Samurai\Seven.Samurai.1954.redacted2.avi'
[2019-06-27 10:33:17,995] [61153023] [139 ] [INFO ] - ImporterWorker: 43 % completed (1 ImportJob(s), in total 25 of 58 so far identified resources processed)
[2019-06-27 10:33:17,997] [61153025] [139 ] [INFO ] - ImporterWorker: 86 % completed (1 ImportJob(s), in total 50 of 58 so far identified resources processed)
[2019-06-27 10:33:17,997] [61153025] [97 ] [INFO ] - ImporterWorker.ImportJob(#1).MetadataExtractorBlock: Successfully processed 58 MediaItems; time elapsed: 00:00:31.8174594; MaxDegreeOfParallelism(InnerBlock) = 40
[2019-06-27 10:33:21,713] [61156741] [181 ] [INFO ] - Media item 96a17a34-fd46-48e6-abcd-831c9eae9a2d with name Seven Samurai (1956) reconciled (3897 ms)
[2019-06-27 10:33:22,916] [61157944] [172 ] [INFO ] - ImporterWorker.ImportJob(#1).MediaItemSaveBlock: Successfully processed 3 MediaItems; time elapsed: 00:00:36.6281907; MaxDegreeOfParallelism(InnerBlock) = 1
[2019-06-27 10:33:22,916] [61157944] [84 ] [INFO ] - ImporterWorker.ImportJob(#1).RelationshipExtractorBlock: Successfully processed 3 MediaItems; time elapsed: 00:00:36.5203169; MaxDegreeOfParallelism(InnerBlock) = 40
[2019-06-27 10:33:22,918] [61157946] [84 ] [INFO ] - ImporterWorker.ImportJob(#1): Completed
[2019-06-27 10:33:22,920] [61157948] [84 ] [INFO ] - ImporterWorker.ImportJob(#1): Disposed

You may ask me why are there 58 media items in this folder. This is mostly how it is after tMM downloads metadata. There is an .actors folder, there are files like banner.jpg, clearart.jpg etc. Importantly, there is movie.nfo - the file I want MediaPortal to read. There are three parts to this particular version, so three .avi files in that folder.

When I see it in the Movies tile in MP2, I see only one tile with the movie poster/synopsis etc. This is good, if it realizes that the other files are continuation. If it doesn't then, how can I let MP2 know that this particular movie is in three separate files?

More importantly, two questions: why did it not hit that breakpoint? Why is it apparently failing to create the mediaitem sometimes in symbolic link folders?

Thanks,
Sridhar

 

[Lehmden]

Hi.

This is good, if it realizes that the other files are continuation.

If you have named the parts correct, then MP knows how to stack the 3 videos into one movie...
See:
Multi-Part Files (Stacked Files)
for proper naming schemes...

 

[sridharb]

Hi.

If you have named the parts correct, then MP knows how to stack the 3 videos into one movie...
See:
Multi-Part Files (Stacked Files)
for proper naming schemes...

Thanks, I will ensure that the files are renamed appropriately.

 

[sridharb]

Ok, did a few more experiments. I created a folder with only what is downloaded by tMM (the other folder possibly had a mix of content downloaded by MP2 as well as tMM).

I created a new source for the symlink folder and it imported upon creation. The metadata looks fine.

To test whether it is using the NFO file (my settings are to download no data from online sources to make MP2 use the information that was downloaded by tMM), I changed the title of the movie (in the <title> section) to something distinctively different. I deleted the media source and readded it again. It imported again.

But, the changes of the NFO file were not used - i.e. I saw the same "good" information as before.

Is this consistent with the NFO file breakpoint not being hit earlier? The NFO file is called movie.nfo. I believe that this is one of the files that it should accept.

Where could this be going wrong?

Thanks,
Sridhar

 

[Lehmden]

Hi.

movie.nfo

I think this will not be recognized, also I'm not 100% sure. Please try to name the .nfo exactly as the video file (excerpt for "_CD1" or similar.
I only use this naming and it always would be read ever since the .nfo support was added to MP2 Alpha back in the days. And all other HTPC tools I'm aware of (MP1, Kodi, Emby, Plex,...) are reading this naming too..

Example: "12 Monkeys (1995).mkv" with "12 Monkeys (1995).nfo"

Aside this I would delete or move away the database (c:\ProgramData\Team MediaPortal\MP2-Server\Database\Datastore.s3db) if you want to test such things. As far as I remember the metadata is not removed from DB if you remove the file, only the file itself is. This way a later re-import is faster but this is not good for this kind of testing.

 

[sridharb]

Ok, on both suggestions.

What about the other files (like the banner, poster.jpg). Is there a specific format that MP2 looks for? In tMM, the defaults are to use filenames like "poster.jpg, banner.jpg etc.) but it also has options to use formats like "movie name -poster.jpg"). I am guessing that it probably would use "movie name"-poster.jpg more likely, but where could I confirm this?

Regards,
Sridhar

 

[Lehmden]

Hi.
There are chapters in the Wiki where the naming is described.

Setting up Media Items

In General MP2 is "more tolerant" than Kodi when it comes to naming of local fanart. Excerpt for very little things you can use exactly the same naming than Kodi asks for. The default settings in my Tool Media-Buddy (it's a different tool than tMM but it also adds .nfo and local fanart to video and audio files separately or even while you are recode or remux the video files. some sort of "Handbrake, MKVMerge and tMM with one mouse click in one tool") are set to match both, MP2 and Kodi...

For movies there is {videoname}.nfo, banner.jpg. poster.jpg, fanart.jpg, clearart.png, clearlogo.png, discart.png and the only different thing between MP2 and Kodi, the movie thumb (as on fanart.tv) must be landscape.jpg for Kodi and has to be thumb.jpg for MP2. I've raised a feature request already to allow "landscape.jpg" for the movie thumb in MP2 too. But this is not jet implemented, also I think it will be added some not too far away day...

The naming above only is possible if you have a "1 movie = 1 folder" setup (like I have and if I understand right, you also have). If you store all (many) movies all together in one folder then you need to add "{videoname}-"before the base filename. E.g. instead of "poster.jpg" you need to name it "12 Monkeys (1995)-poster.jpg" then...

For series there is one tvshow.nfo for each series and poster.jpg, banner.jpg, fanart.jpg, clearart.png, clearlogo.png seasonxx-poster.jpg, seasonxx-banner.jpg and for each episode there is a {videoname}.nfo and a {videoname}-thumb.jpg. Series need to have their own directory. Inside you can use sub- folders for seasons (I do) but you don't need to. The naming above is working both ways.

 

[sridharb]

Thanks Lehmden for the clarification. I have verified that apart from {videoname}-thumb.jpg, the defaults in tMM were fine. I had chosen MP2 as the media player in the TV section. I guess, this is the only feedback that I should possibly give back to tMM.

 

[sridharb]

With the linked folder working "fine" in the test, I decided to dump the earlier database and start from zero.

After creating all the media sources, it started to import everything automatically, ran for a while and ran out of memory. Then I kicked off the import of each category one by one and now, it is done.

I do see all the movies in all the linked folders except for a handul - which are much much lower than before, so, this is something I can investigate further.

However, the most important problem left is that it is not showing any "banner/poster/fanart" etc. All it shows is the movie synopsis, genre, year etc. I am guessing this "text" information is from the NFO file. But all the other files (banner.jpg, fanart.jpg, clearart and whatnot) are not being used. I see that they are named like Lehmden describes above - i.e. {videoname}.nfo, banner.jpg. poster.jpg, fanart.jpg, clearart.png, clearlogo.png, discart.png and not {videoname}-banner.jpg etc.

I have asked MP2 not to download any fanart - by which I thought it should use what is already there. But it seems to not.

Any clues?

Thanks,
Sridhar