Suggested Patch to Filter Out Sample Files Faster (1 Viewer)

Bernard

Portal Pro
March 5, 2012
103
25
35
Centurion
South Africa South Africa
Hi Moving Pictures Devs,

I have made a small change to Moving Pictures source and would like to post it to the Developers Group on Google, but I can't figure out how to join since it complains that I don't have the correct Permissions to Join.

Problem: I have a bunch (about 80) sample videos between all my movies on a network drive. Every time the import of Moving Pictures run, it collects this list of samples and tries to process them like "new" video files, only to spit them out just before they are queued for online lookup. Part of this process is to build a Media object in memory and load it's current setting from the db.

Because my client PC/Laptop is working over a slow wifi connection, with a slow CPU. It takes a while to work through these files every time. And all the progress indicators "act" as if there are about 80 "new" video files to process.

Moral of the story: A lot of high level processing is involved on the samples to determine if the video files is new and need to be looked up online that can be skipped.

Suggested Solution: My suggested solution is that the Importer should disregards sample files the same way it disregard any other file type that is not a video file, like .nfo or .txt. In essence a sample video is like an invalid file format: It will never be in the DB, so no point in "just checking" every time.

In the source I have "copied" the code that spit out the samples higher up in the call stack so that it ignores those files in the same process as it ignores "unsupported" files.

Implementation:
MovingPictures\Database\DBImportPath.cs, line 237, add:
Code:
// exclude samplefiles and ignore them from any processing
if (VideoUtility.isSampleFile(videoFile))
{
	logger.Info("IGNORED: File='{0}', Bytes={1}, Reason='Sample detected'",videoFile.Name, videoFile.Length);
	continue;
}
That is it. This code comes from MovingPictures\LocalMediaManagement\MovieImporter.cs, line 1090. I would have suggested that the code be removed from ScanFiles(...) but then the importer will not spit out sample files that was in fact imported previously and is not "resent to importer" (this will happen if the setting was off during an initial import run).

I have attached a "patch" file as well.
 

Attachments

Last edited:

spacemanc

MP Donator
  • Premium Supporter
  • October 20, 2011
    89
    14
    44
    Country flag
    I also waste quite a bit of time with sample files. The other similar problem I have is that a lot of my movie folders have a separate "2 channel audio" file, which also get picked up on a scan. I suppose one way around it is to just manually delete all the unneeded files via windows, but I've never got around to it and I don't scan often enough to make it a big problem!
     

    Bernard

    Portal Pro
    March 5, 2012
    103
    25
    35
    Centurion
    South Africa South Africa
    I also waste quite a bit of time with sample files. The other similar problem I have is that a lot of my movie folders have a separate "2 channel audio" file, which also get picked up on a scan. I suppose one way around it is to just manually delete all the unneeded files via windows, but I've never got around to it and I don't scan often enough to make it a big problem!

    If you would like to try this "alternative code" you can copy this dll to your C:\Program Files\Team MediaPortal\MediaPortal\plugins\Windows folder.

    For my 'slow' laptop this actually help the GUI get faster through the initial import run when I'm waiting for new movie files to show up in the GUI importer, and also when there is no new movies and it is just running in the background.

    Full Disclaimer: Please just backup (rename) your existing MovingPictures.dll file, before using this one, if this somehow break your MePo you can just delete it and rename the old file back. I did not extensively test the code, I don't have any "
     

    Attachments

    Last edited:

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,348
    1,828
    Country flag
    You might also be able to modify the 'noise filter' and ignore the samples at a higher level, the same way MP-TVSeries does.
     

    Bernard

    Portal Pro
    March 5, 2012
    103
    25
    35
    Centurion
    South Africa South Africa
    You might also be able to modify the 'noise filter' and ignore the samples at a higher level, the same way MP-TVSeries does.
    I will see if I can find this then. I did not notice any part in MovingPictures that pointed to a "noise filter" it only does 'n simple:

    Get Files in Watched Folders > Loop and Keep Only Video Extentions

    then at a very later stage (part of the movie online details processing code) it removes entries in this queue if they match the samples RegEx. Important to note that "Send to Importer" starts in this same function, so the code should be here if you want the importer to remove samples even if they have been imported already as movies.
     

    Users Who Are Viewing This Thread (Users: 0, Guests: 1)

    OP Similar threads Forum Replies Date
    R Suggestions for reliable PCIe DVB-S2 tuner MediaPortal 1 Talk 8
    G Remote control suggestions? MediaPortal 1 Talk 16
    J Suggestion on fanart folder organization. Feature Requests 4
    K Suggest me how to install server on Mac Newcomers Forum 2
    SpudR Suggested Plugin MediaPortal 1 Plugins 20
    N [Approved] [MP2-198-200] Playlist handling patches Submit: Code Patches 4
    BlueMax1916 [no Bug] Windows 10 1803 January Patch -> H.265 videos without video output MediaPortal 2 6
    O General Merging various patches into TVE3 TV Engine 21
    T MadVR + LAV Filters, Perfect in MPC-BE, Bad in MP1 MediaPortal 1 Talk 13
    rsbrux 1.23.0 MPE Manager Problem with LAV Filters Bugreports 2
    A RTL2832U (REALTEK DTV Filter) re-appears with new card ID in TV server after something? General Support 3
    azzuro [fixed] [MP2-813] Album artist filter wrong display Submit: Bug Reports 75
    cbr954rr Videos flickering with lav Filter General 2
    HTPCSourcer [fixed] [MP2-794] Genre filtering always displays English strings MediaPortal 2 9
    HTPCSourcer [Rejected] Replace "Filter by" by "Group by" MediaPortal 2 14
    N Option to "Filter" Images by folder name Feature Requests 8
    N [Rejected] Rename "Filter" to "Group by" Feature Requests 3
    S [solved] Library view and filters General 5
    A Audio sync problems on LAV filters 1.21 Pre Release 31
    D [Rejected] LAV filter setup wizard Feature Requests 1
    B [MP2-745] Allow direct navigation to specified media item/filter Archive 7
    SpudR Custom filter for partially watched movies Moving Pictures 5
    Top Bottom