[fixed] ISOResourceProvider Exception on some ISOs only (1 Viewer)

[MJGraf]

While I was testing the Import of Videos today, I had some problems with two DVD-ISOs. And only these two, all my other DVD-ISOs work like a charm. I'm using the dev-branch of GIT. I added a global share with only one of the not-working ISOs in the path and started a ReImport. What I found out so far:

I set a breakpoint in Line 199 of the ISOResourceAccesor (ResourceExists-Method):
string[] dirList = _isoProxy.IsoReader.GetFileSystemEntries(dosPath, SearchOption.TopDirectoryOnly);

---------------------------------

The first time I get there, I've got the following locals:
path = "VIDEO_TS"
dosPath = "\\"
dosCombined = "\\VIDEO_TS"
This seems ok and I let it go...

---------------------------------

The second time i get there, i looks like this:
path = "VIDEO_TS.IFO"
dosPath = "\\VIDEO_TS"
dosCombined = "\\VIDEO_TS\\VIDEO_TS.IFO"

This time I get an exception:

[2011-10-16 20:31:54,982] [266843 ] [Importer ] [INFO ] - MovieMetadataExtractor: Exception reading resource '{e88e64a8-0233-4fdf-ba27-0b44c6a39ae9}:///C:/_MP/Quellcode/Media/NOMADEN_LUEFTE.iso>{112728b1-f71d-4284-9e5c-3462e8d3c74d}:///' (Text: 'The entry "\VIDEO_TS" does not exists in the paths table')
[2011-10-16 20:31:55,107] [266968 ] [Importer ] [INFO ] - ImporterWorker: Importing 'C:\_MP\Quellcode\Media\NOMADEN_LUEFTE.iso > /VIDEO_TS' (2 more resources pending)
[2011-10-16 20:31:55,138] [267000 ] [28 ] [WARN ] - IsoResourceAccessor: Error reading child directories of '{e88e64a8-0233-4fdf-ba27-0b44c6a39ae9}:///C:/_MP/Quellcode/Media/NOMADEN_LUEFTE.iso>{112728b1-f71d-4284-9e5c-3462e8d3c74d}:///VIDEO_TS'
System.IO.DirectoryNotFoundException: The entry "\VIDEO_TS" does not exists in the paths table
bei ISOReader.IsoReader.GetFileSystemEntries(String path, SearchOption recursive)
bei ISOReader.IsoReader.FilterEntries(String path, Boolean directory, SearchOption recursive)
bei ISOReader.IsoReader.GetDirectories(String path, SearchOption recursive)
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetChildDirectories()
[2011-10-16 20:31:55,154] [267015 ] [28 ] [WARN ] - IsoResourceAccessor: Error reading files of '{e88e64a8-0233-4fdf-ba27-0b44c6a39ae9}:///C:/_MP/Quellcode/Media/NOMADEN_LUEFTE.iso>{112728b1-f71d-4284-9e5c-3462e8d3c74d}:///VIDEO_TS'
System.IO.DirectoryNotFoundException: The entry "\VIDEO_TS" does not exists in the paths table
bei ISOReader.IsoReader.GetFileSystemEntries(String path, SearchOption recursive)
bei ISOReader.IsoReader.FilterEntries(String path, Boolean directory, SearchOption recursive)
bei ISOReader.IsoReader.GetFiles(String path, SearchOption recursive)
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetFiles()

Looks like there should be one more "\"?!?

---------------------------------

The third time i get there, i looks like this:
path = "VIDEO_TS"
dosPath = "\\VIDEO_TS"
dosCombined = "\\VIDEO_TS\\VIDEO_TS"

This looks really strange to me because there is no folder "VIDEO_TS" under "VIDEO_TS". Is there something wrong with the recursion? As a result, I get this:

[2011-10-16 20:37:03,294] [575156 ] [Importer ] [INFO ] - MovieMetadataExtractor: Exception reading resource '{e88e64a8-0233-4fdf-ba27-0b44c6a39ae9}:///C:/_MP/Quellcode/Media/NOMADEN_LUEFTE.iso>{112728b1-f71d-4284-9e5c-3462e8d3c74d}:///VIDEO_TS' (Text: 'The entry "\VIDEO_TS" does not exists in the paths table')
[2011-10-16 20:37:03,341] [575203 ] [Importer ] [WARN ] - IsoResourceAccessor: Error reading files of '{e88e64a8-0233-4fdf-ba27-0b44c6a39ae9}:///C:/_MP/Quellcode/Media/NOMADEN_LUEFTE.iso>{112728b1-f71d-4284-9e5c-3462e8d3c74d}:///VIDEO_TS'
System.IO.DirectoryNotFoundException: The entry "\VIDEO_TS" does not exists in the paths table
bei ISOReader.IsoReader.GetFileSystemEntries(String path, SearchOption recursive)
bei ISOReader.IsoReader.FilterEntries(String path, Boolean directory, SearchOption recursive)
bei ISOReader.IsoReader.GetFiles(String path, SearchOption recursive)
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetFiles()
[2011-10-16 20:37:03,372] [575234 ] [Importer ] [WARN ] - IsoResourceAccessor: Error reading child directories of '{e88e64a8-0233-4fdf-ba27-0b44c6a39ae9}:///C:/_MP/Quellcode/Media/NOMADEN_LUEFTE.iso>{112728b1-f71d-4284-9e5c-3462e8d3c74d}:///VIDEO_TS'
System.IO.DirectoryNotFoundException: The entry "\VIDEO_TS" does not exists in the paths table
bei ISOReader.IsoReader.GetFileSystemEntries(String path, SearchOption recursive)
bei ISOReader.IsoReader.FilterEntries(String path, Boolean directory, SearchOption recursive)
bei ISOReader.IsoReader.GetDirectories(String path, SearchOption recursive)
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetChildDirectories()
[2011-10-16 20:37:03,529] [575390 ] [Importer ] [WARN ] - IsoResourceAccessor: Error reading files of '{e88e64a8-0233-4fdf-ba27-0b44c6a39ae9}:///C:/_MP/Quellcode/Media/NOMADEN_LUEFTE.iso>{112728b1-f71d-4284-9e5c-3462e8d3c74d}:///VIDEO_TS'
System.IO.DirectoryNotFoundException: The entry "\VIDEO_TS" does not exists in the paths table
bei ISOReader.IsoReader.GetFileSystemEntries(String path, SearchOption recursive)
bei ISOReader.IsoReader.FilterEntries(String path, Boolean directory, SearchOption recursive)
bei ISOReader.IsoReader.GetFiles(String path, SearchOption recursive)
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetFiles()
[2011-10-16 20:37:03,716] [575578 ] [Importer ] [INFO ] - ImporterWorker: Importing 'C:\_MP\Quellcode\Media\NOMADEN_LUEFTE.iso > /AUDIO_TS'

---------------------------------

Then we get there a fourth time with this:
path = "VIDEO_TS"
dosPath = "\\AUDIO_TS"
dosCombined = "\\AUDIO_TS\\VIDEO_TS"

This looks also very strange, but there's no more exception.

So if that does ring a bell for you programming guys, feel free ;-)

cheers,
Michael

 

[MJGraf]

-------------------------------------------------

Ok, what I found out for now is: The problem is either my "problem"-ISO not being ISO9660-compliant or there's a deeper error in ISOReaderBase...

What happens is that it is checked, whether "\\VIDEO_TS" exists in the root directory of the ISO. Result: true.
Then it's checked whether VIDEO_TS.IFO exists within that directory. There the problem begins:
ISOReaderBase tries to read the contents of the "\\VIDEO_TS" directory. The result, however, is "\\AUDIO_TS", which of course isn't true.
The root of the problem is therefore, that ISOReaderBase isn't able to read the content of the directory "\\VIDEO_TS"

That means that either my ISO isn't ISO9660-compliant or ISOReaderBase has a bug. Anyway, VCD mounts and reads this ISO correctly - which doesn't help either, because I'm not sure, whether VCD corrects some things in non-compliant ISOs and our ISOReader doesn't.

I'm not sure whether there is an ISO9660 expert among you guys. If not, although I like the idea of not relying on thrid party apps, it may be better in this case to have a (maybe alternative) ISOResourceProvider, which just uses an installed VCD or Daemon-Tools to mount the ISO-file and thereby gives acces to its contents.

What do you guys think? Am I the only one with ISO-problems?

cheers,
Michael

 

[MJGraf]

---------------------------------------

Just a thought: Would this be an alternative?

*»*File*»*discutils*»*DiscUtils*»*Iso9660*»*C# / CSharp Open Source

The license looks good to me.

And we would not only be able to read ISOs, but also to write them (e.g. for an DVD-to-image-plugin). Even the whole package of "discutils" (homepage seems to be .NET DiscUtils) looks very interesting. That would enable MP2 to also read other kinds of disk images or the like.

If that's a stupid idea, don't refrain from telling me

cheers,
Michael


Edit: And I just stumpled across this post: http://discutils.codeplex.com/discussions/265391 Seems like it could help MP2 to access Bluray-ISOs as well...

------------------------------------------------------

I found the reason. And I found out that indeed I seem to have missed at least 10 years of technical development, which I've only realized as user not as programmer. When I bought my first SCSI 2xspeed CD-Rom drive for about 1.000 Marks, "UDF" wasn't well known, yet

All my ISO-files have both, ISO-9660 and UDF "file systems". However, the two problematic ones, when opened with ISO-Buster, only show the "AUDIO_TS" folder, when accessed via ISO-9660. The "VIDEO_TS" is not shown, which explains why the "VIDEO_TS.IFO" file is not found and also explains (partly), why we get the "AUDIO_TS" folder as result, when the content of the "VIDEO_TS" folder should be returned. However, when I access those ISO-files with ISO-Buster via UDF, the whole folder structure is accessible (in particular AUDIO_TS and VIDEO_TS folder). This in turn explains, why VCD opens the ISO-file without problems. My BluRay-ISO btw (as also mentioned in my link above) only has UDF (seems to be part of the spec), which is why it doesn't work, either.

That means what we need is an ISOResourceProvider, which can handle UDF and ISO-9660....

How about these discutils above? Could anyone with more knowledge than me have a look at the license, whether it would be ok to use that code under the MP2 license? And of course Albert: Do you think it makes sense to have a deeper look into it? You're the "master of desaster" and if you don't like it, I wouldn't pursue it any further. If it's ok, I'd have a look into it and see if I find the time to put some code together (not promising anything, my RL-job eats me up... )

Michael

 

[Albert]

AW: ISOResourceProvider Exception on some ISOs only

From a first quick investigation, I see one error: Your Media directory doesn't belong under Quellcode.
But that's not the problem here, off course ;-)

I'll check the problem at the end of the week. Thanks for the report!

 

[MJGraf]

Just as a short note, while trying to understand the code of ISOResourceProvider, I think I found a typo in ISOResourceProvider.cs (line 143 of the dev branch: IsResource-Method):

Isn't here a not-operator missing?

if (string.IsNullOrEmpty(resourceName) || baseResourceAccessor.IsFile)
return false;

If the base resource is NOT a file (i.e. an ISO-file) then it's not a resource for this ResourceProvider. Should in my understanding be:

if (string.IsNullOrEmpty(resourceName) || !baseResourceAccessor.IsFile)
return false;

The method is never invoked at the moment, so it doesn't have any consequences, yet.

Michael

 

[Albert]

AW: ISOResourceProvider Exception on some ISOs only

MJGraf, I just fixed some more problems in IsoResourceAccessor/Provider.
Can you please check if the problems here still exist? If yes, can you please give me a complete stack trace which also shows the callers of the methods in IsoResourceAccessor? During debug, you can get the complete stack trace by evaluating the property "Environment.StackTrace".
Thank you!

 

[Smeulf]

---------------------------------------

Just a thought: Would this be an alternative?

*»*File*»*discutils*»*DiscUtils*»*Iso9660*»*C# / CSharp Open Source

Just a quote from .NET DiscUtils :

Implementation of the ISO, UDF, FAT and NTFS file systems is now fairly stable

Maybe it worth to try if it can works or not, and wait after a more stable release, if done because the last update seems dated to July 2010, and I don't know if the project is still active or not.

Cheers.

Smeulf.

 

[MJGraf]

ok guys, this is my first patch ever, so please be kind with me

What I did is: replace the included ISOReader with the discutils as mentioned above. It is surely anything but ready, yet, but for me, it works like a charm. Only bluray isos still give me some exceptions, but I think this is due to copy protection. Anything else just works - at least for me I will provide you with a "nicer" patch, but I'm leaving for holidays on Wednesday and I won't be able to do that before I leave and I thought, maybe someone wants to test already in the meantime...

Albert: Thanks a lot. Will try to test your patch today. But i think, the problem is that ISOReader just can't read udf filesystems and most of the DVDs today (and all the bulrays) seem to use udf instead of iso9660. But nevertheless, will try and report back.

Smeulf: I think they underestimate a little bit with "fairly stable" (like MP 0.23 some time ago ). The discussions there are still active and the code seems to be developed further. I tested my patch with 26 isos I have here and except for the one bluray iso, it just worked...

As mentioned above, I'm not a fan of having too much third-pary-code and I tried to just include the iso9660-reader and the udf-reader from discutils in MP2. But my programming knowledge is just not enough for that. Therefore I've included the discutils.dll as a whole (and of course the respective license.txt, which seems to be the only requirement for using the code).

Let me know what you think...

Michael

 

[MJGraf]

@ Albert:

With your changes, the exceptions from post one are still there. What I did is:

Start MP2-Server (current dev-branch)
Start MP2-Client (also current dev-branch)
Trigger Share Reimport

The share is a test-share, which only has two files: a test.avi and one of the problematic ISOs.

Server-Log of the complete reimport is attached, server-error.log is empty.

Then I tried to get you a complete stack trace as per your post above. I set a brekpoint in line 261 of IsoResourceAccessor.cs (GetChildDirectories() where the first exception above is catched) and a second breakpoint in line 245 (GetFiles() where the second exception above is catched). I triggered a reimport and when I got to the respective brekpoints, I evaluated Environment.StackTrace. The results are:

1. GetChildDirectories():

bei System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
bei System.Environment.get_StackTrace()
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetChildDirectories()
bei MediaPortal.Common.Services.Dokan.VirtualDirectory.get_ChildResources() in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\Dokan\VirtualDirectory.cs:Zeile 42.
bei MediaPortal.Common.Services.Dokan.Dokan.ParseFileName(String fileName) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\Dokan\Dokan.cs:Zeile 188.
bei MediaPortal.Common.Services.Dokan.Dokan.CreateFile(String filename, FileAccess access, FileShare share, FileMode mode, FileOptions options, DokanFileInfo info) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\Dokan\Dokan.cs:Zeile 228.
bei Dokan.Proxy.CreateFileProxy(IntPtr rawFileName, UInt32 rawAccessMode, UInt32 rawShare, UInt32 rawCreationDisposition, UInt32 rawFlagsAndAttributes, DOKAN_FILE_INFO& rawFileInfo) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\Dokan\Proxy.cs:Zeile 217.

2. GetFiles():

bei System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
bei System.Environment.get_StackTrace()
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetFiles()
bei MediaPortal.Common.Services.Dokan.VirtualDirectory.get_ChildResources() in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\Dokan\VirtualDirectory.cs:Zeile 47.
bei MediaPortal.Common.Services.Dokan.Dokan.ParseFileName(String fileName) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\Dokan\Dokan.cs:Zeile 188.
bei MediaPortal.Common.Services.Dokan.Dokan.CreateFile(String filename, FileAccess access, FileShare share, FileMode mode, FileOptions options, DokanFileInfo info) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\Dokan\Dokan.cs:Zeile 228.
bei Dokan.Proxy.CreateFileProxy(IntPtr rawFileName, UInt32 rawAccessMode, UInt32 rawShare, UInt32 rawCreationDisposition, UInt32 rawFlagsAndAttributes, DOKAN_FILE_INFO& rawFileInfo) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\Dokan\Proxy.cs:Zeile 217.

3. GetFiles():

bei System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
bei System.Environment.get_StackTrace()
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetFiles()
bei MediaPortal.Common.ResourceAccess.FileSystemResourceNavigator.GetFiles(IResourceAccessor directoryAccessor) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\ResourceAccess\FileSystemResourceNavigator.cs:Zeile 110.
bei MediaPortal.Common.Services.MediaManagement.ImporterWorker.ImportDirectory(ImportJob importJob, Guid parentDirectoryId, IFileSystemResourceAccessor directoryAccessor, ICollection`1 metadataExtractors, IMediaBrowsing mediaBrowsing, IImportResultHandler resultHandler, IMediaAccessor mediaAccessor) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\MediaManagement\ImporterWorker.cs:Zeile 410.
bei MediaPortal.Common.Services.MediaManagement.ImporterWorker.Process(ImportJob importJob) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\MediaManagement\ImporterWorker.cs:Zeile 562.
bei MediaPortal.Common.Services.MediaManagement.ImporterWorker.ImporterLoop() in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\MediaManagement\ImporterWorker.cs:Zeile 228.
bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
bei System.Threading.ExecutionContext.runTryCode(Object userData)
bei System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
bei System.Threading.ThreadHelper.ThreadStart()

3. GetChildDirectories():

bei System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
bei System.Environment.get_StackTrace()
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetChildDirectories()
bei MediaPortal.Common.ResourceAccess.FileSystemResourceNavigator.GetChildDirectories(IResourceAccessor directoryAccessor) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\ResourceAccess\FileSystemResourceNavigator.cs:Zeile 56.
bei MediaPortal.Common.Services.MediaManagement.ImporterWorker.Process(ImportJob importJob) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\MediaManagement\ImporterWorker.cs:Zeile 569.
bei MediaPortal.Common.Services.MediaManagement.ImporterWorker.ImporterLoop() in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\MediaManagement\ImporterWorker.cs:Zeile 228.
bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
bei System.Threading.ExecutionContext.runTryCode(Object userData)
bei System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
bei System.Threading.ThreadHelper.ThreadStart()

4. GetFiles():

bei System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
bei System.Environment.get_StackTrace()
bei MediaPortal.Extensions.ResourceProviders.IsoResourceProvider.IsoResourceAccessor.GetFiles()
bei MediaPortal.Common.ResourceAccess.FileSystemResourceNavigator.GetChildDirectories(IResourceAccessor directoryAccessor) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\ResourceAccess\FileSystemResourceNavigator.cs:Zeile 59.
bei MediaPortal.Common.Services.MediaManagement.ImporterWorker.Process(ImportJob importJob) in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\MediaManagement\ImporterWorker.cs:Zeile 569.
bei MediaPortal.Common.Services.MediaManagement.ImporterWorker.ImporterLoop() in C:\_MP\Quellcode\MediaPortal-2\MediaPortal\Source\Core\MediaPortal.Common\Services\MediaManagement\ImporterWorker.cs:Zeile 228.
bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
bei System.Threading.ExecutionContext.runTryCode(Object userData)
bei System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
bei System.Threading.ThreadHelper.ThreadStart()

I hope this is what you needed. But once again, I don't think, it's an error in your code. It's just the fact, that my iso's iso9660-directory is not compliant. When accessing the same iso with my patch above, it uses udf in the first place (and iso9660 only if there is more information) and it works.

Michael

 

[MJGraf]

-------------------------------------

One last note about my patch: It seems to work for bluray-isos as well. At least they are imported into the database. When I try to play a bluray-iso it says: "there is no player registered to play media item [...]". Is the player not (yet) aware that blurays may also be accessed by the ISOResourceAccessor?!?