Network Share Not Accessible from Importer (1 Viewer)

[Kotik]

I am getting a really strange issue from the Moving Pictures Importer.

Let me 1st give u a heads up, i transferred my whole MP DB from 7 to 10 and everything is working fine, i even updated moving pictures after the transfer and all my movies are working great within MP itself, yesterday i decided to add some new movies and TADA

The issue is that while i can normally access the NAS share folder from this PC, the importer itself is unable and it is driving me nuts, meanwhile if i manually add the share path to the importer then it is able to pick up the path and import the movie, althought it wont even download any backdrops for it, but non the less it will add the movie this way to the collection. Accessing the same path over windows is working fine and the funny part is that when moving pictures importer is trying to access the folder then windows itself is unable to access this folder, but if i close moving pictures importer and try to access the folder from windows it is working normally.

I need help.

Logs attached and screenshots that demonstrate the error itself and also show that windows can access the folder.

 

[Kotik]

I just found out that the issue is due to my movingpictures.db3. I went for a fresh installation of MP on a second pc just to see if i got some security issues that ptevent my main HTPC from accessing these folders over moving pictures importer, after i was done with the installation i fired up the importer on an emtpy .db and i WAS able to access the folders. The moment i copied over my .db3 backup file i got exactly the same error.
Guys i need help, i dont want to import all over again my movies, please someone

 

[HTPCSourcer]

The moment i copied over my .db3 backup file i got exactly the same error.

Probably a security error at file level. Copying a file from A to B is not copying the respective NTFS privileges that were set on A. Verify the security properties of the db3 and compare to A to see what may be missing.

 

[Kotik]

This is exactly what i thought at first. But i did a couple more tests and found one something interesting. If i remove from the .db3 half of my movies then i can again add a folder with no issues. My next step was to restore again the full .db3 and remove the other half. Guess what... it is working, so it is not a corrupt db. Something on my file server side is blocking too many connections from the same ip. At least this is what i think since this would also explain why windows are unable to open the same folders when the importer is running.
The way i see it is that once u run the importer it will start making concurrent connections to the file server, file server will then block it as if it was a flooding attack and after that even windows are unable to access the folders. In order to allow windows access i have to turn off the importer and then everything is working.
What could contribute to the issue is that i got a separate folder for each movie so as a result i got multiple folders that movingpictures got to scan.
Any ideas guys?

How can i prevent my file server from blocking concurrent connections?
Now that i think of it, it is maybe cause i moved my file server to windows 7 from windows xp. Maybe 7 got some extra security layers in place?
I did try with firewall off but the issue remains same.
--- Post updated ---
Found the solution.
I had to 1st check system logs on my Windows 7 file storage PC, there i discovered that i am getting Error 2017 each time i run the MovingPictures importer, after googling the error i found out THIS,
i followed it and after a reboot everything is working.

Thanx for ur idea @HTPCSourcer, apparently Windows 7 are not designed to handle such a heavy load, in any case i will run it now with the tweaks and report after a week or so regarding the stability, in case someone else runs into the same issue.

P.S. How and why movingpictures importer is generating such a heavy load is unknown to me but maybe a dev could find out a more elegant solution at the importer level?

 

[RoChess]

The Moving-Pictures importer works with concurrent threads, you can adjust the thread count in advanced settings. Repeat your scenario by changing it from the default 5 (if memory serves me) to say 3, and see if that also fixes it.

A single import thread also kicks in an additional one for the MediaInfo part, and MediaPortal thumbnail generation.

The latter is what usually kills performance as well, which is why I always disable it.

Try that first, because it requires a lot of IO during import.

http://wiki.team-mediaportal.com/1_...MediaPortal_Configuration/13_GUI/3_Thumbnails <-- The "Autocreate thumbs" checkbox.

 

[Kotik]

Forget my solution above, while it seemed to work under window 7 it doesn't work under windows 8.1 and there are no errors reported in the windows 8.1 logs. I am running out of ideas but I got a theory.

I took a deeper look into the moving pictures log files and I found something interesting:

27-Oct-2016 20:50:20 Info [ MovieImporter]: Started MovieImporter
27-Oct-2016 20:50:20 Info [ MovieImporter]: Initiating full scan on watch folders.
27-Oct-2016 20:50:20 Info [ MovieImporter]: Started watching '\\NAS-PC\Disk 1\Alien Quadrilogy' (Network) - Path is now being monitored for changes.

As u can see till here we are all good but later on when importer is trying to monitor more folders I get this:

27-Oct-2016 20:50:20 Info [ MovieImporter]: Started watching '\\NAS-PC\Disk 2\Wall-E' (Network) - Path is now being monitored for changes.
27-Oct-2016 20:50:20 Info [ MovieImporter]: Stopped watching '\\NAS-PC\Disk 2\Wall-E' (Network) - Reason: Insufficient system resources exist to complete the requested service

The Moving-Pictures importer works with concurrent threads, you can adjust the thread count in advanced settings. Repeat your scenario by changing it from the default 5 (if memory serves me) to say 3, and see if that also fixes it.

A single import thread also kicks in an additional one for the MediaInfo part, and MediaPortal thumbnail generation.

The latter is what usually kills performance as well, which is why I always disable it.

Try that first, because it requires a lot of IO during import.

http://wiki.team-mediaportal.com/1_MEDIAPORTAL_1/141_Configuration/MediaPortal_Configuration/13_GUI/3_Thumbnails <-- The "Autocreate thumbs" checkbox.

I tried with 3 and got the same issue, I then tried with 1 and that didn't help either, I got thumbs off in MP setup so that is excluded, any ideas?

Attached is moving pictures logfile.

@RoChess Is there any way to reset the "Initiating full scan on watch folders"? it seems that MP is choking while trying to get that info.

 

[Kotik]

More information,

I decided to start with a fresh movingpictures.db3 so I can make sure that my previous .db3 is not corrupt.

Started importing folders and guess what..... same issue, I started calculating the amount of shared folders and it is 63 so it makes no sense, but I know that if I remove 1 shared folder from the importer and try to add a new one then it will work perfectly fine until of course I get more than 63 shared folder that are being accessed, then same error.

Something is wrong with the way the importer is accessing these folders, it behaves as if it triggers some security threshold of either the client pc or the server side. I don't remember this being the issue with windows 7 32bit, but it is an issue for both my 8.1 64bit and 10 64bit clients. It might be the 64bit architecture?

I am slowly becoming paranoid, I replaced my network switch but the issue remains, I tried different Ethernet cards and still got the same results. Windows log files are clean and it is becoming extremely hard to troubleshoot this problem.

@RoChess some ideas would be more than welcome.

 

[RoChess]

It could be a single corrupt file somewhere.

And corrupt could mean a lot, because it can trigger a crash in MediaInfo, or MediaPortal, etc.

You would have to do the n/2 method of trial-and-error.

So add 63/2 = ~32 import paths, and if the first group works, then add the 2nd group, if that fails, then only add 32/2 = 16 import paths... until eventually you narrowed down the 1 path (or 2+) that is causing a problem.

Until eventually you narrow it down to a single file that is possibly causing you all these issues.

Also running into a limit of 64 or 64-1 would not surprise me, so isn't there anyway for you to combine import paths? Use NTFS folder mapping otherwise.

 

[Kotik]

The thing is that the files themselfs are not corrupt, it doesnt really matter in which order i add the shared folders, the moment i hit a limit i am unable to access shared folders at all, and by "at all" i mean not even from windows themselfs, but i can access the same exact folders at the same time from a different PC.

It is as if the storage PC is blocking any further connection attempts that are being made from the pc that is running the importer. If i close the impprter and access the shares over windows it then works.

The importer somehow is triggering something. Let me try to explain how it looks like, as if the importer is considered to ddos attack the storage, it is this or some kind of network resources running low with the importer.


Sent from my GT-I9100 using Tapatalk

 

[Kotik]

I tried playing with extra registry settings on the storage PC but nothing seems to help. The only thing i didnt try is moving all the movies under 1 folder and try to import that one but even if that works that essentialy means that all the work i have done up till now in organizing my collection structure was a waste of time just because moving pictures is trigering something. It was working perfectly fine when my storage disks where located on a windows xp pc and i cannot seem to understand what happened since then, too many things were upgraded since that time and i never had to import a new movie so i never noticed the issue up till recently.

This is exactly why i decided to back up my previous .db3 and start fresh but it looks like it doesnt help.

Sent from my GT-I9100 using Tapatalk