Reply to thread

Message

<blockquote data-quote="RoChess" data-source="post: 727904" data-attributes="member: 18896">Re: IMDb+ Scraper (short/long summary, imdb/RT score, US/UK rating, and more)Sorry for the quick updates the last few days, but now that I can finally test the scraper on my full collection I find rare bugs a lot faster.Since speed is still a main concern with some users, I tried to add as much optimalisation as possible. The regular expression code for covers is super fast now, so that should help, and I improved the speed of the search node.The main delay remains the RottenTomatoes rating, because it adds additional delays in obtaining the RT webpage. I actually considered making the imdb.com rating default, but then today I found out that RottenTomatoes sometimes has summaries (synopsis as they call them) for some movies that lack a summary on the imdb.com website. Unfortunatly RottenTomatoes does not have short synopsis, so this does conflict with the default short summary setting of the scraper, but I thought it was better to have a long synopsis then an empty short summary.I also did a couple of test imports to measure the speeds of the RT and imdb.com rating settings.Default RT rating:30-Mar-2011 00:45:31&nbsp; Info [&nbsp; &nbsp; &nbsp;&nbsp; MovieImporter]: Watcher queued The Matrix (tt0133093).avi for processing.
....
30-Mar-2011 00:45:35&nbsp; Info [&nbsp;&nbsp; FileBasedResource]: Added cover art for &quot;The Matrix&quot; from: <a href="http://." target="_blank">http://.</a>.. 
30-Mar-2011 00:45:36&nbsp; Info [&nbsp;&nbsp; FileBasedResource]: Added backdrop for &quot;The Matrix&quot; from: <a href="http://." target="_blank">http://.</a>..
....
30-Mar-2011 00:45:36&nbsp; Info [&nbsp; &nbsp; &nbsp;&nbsp; MovieImporter]: Added &quot;The Matrix&quot; (1999).So that took 5 seconds total for details, cover and backdrop. Then I removed movie from import folder so it was removed and deleted the thumbs folder, adjusted scraper settings and reimported the scraper with imdb.com ratings enabled.30-Mar-2011 00:49:36&nbsp; Info [&nbsp; &nbsp; &nbsp;&nbsp; MovieImporter]: Watcher queued The Matrix (tt0133093).avi for processing.
....
30-Mar-2011 00:49:38&nbsp; Info [&nbsp;&nbsp; FileBasedResource]: Added cover art for &quot;The Matrix&quot; from: <a href="http://." target="_blank">http://.</a>..
30-Mar-2011 00:49:39&nbsp; Info [&nbsp;&nbsp; FileBasedResource]: Added backdrop for &quot;The Matrix&quot; from: <a href="http://." target="_blank">http://.</a>..
....
30-Mar-2011 00:49:39&nbsp; Info [&nbsp; &nbsp; &nbsp;&nbsp; MovieImporter]: Added &quot;The Matrix&quot; (1999).So that time it took 3 seconds.Now this is only a single movie test, so not very accurate, but you get a rough idea on the difference. For me personally it makes no difference how long each import takes, because they happen in the background for me while I watch something else. For those of you who do care this might make you want to use the imdb.com rating, so adjust settings accordingly.Ohh and incase it wasn't clear, v3.1.1 is now available to download from first post <img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite8" alt=":D" title="Big Grin&nbsp; &nbsp; :D" loading="lazy" data-shortname=":D" /></blockquote>

[QUOTE="RoChess, post: 727904, member: 18896"] [b]Re: IMDb+ Scraper (short/long summary, imdb/RT score, US/UK rating, and more)[/b] Sorry for the quick updates the last few days, but now that I can finally test the scraper on my full collection I find rare bugs a lot faster. Since speed is still a main concern with some users, I tried to add as much optimalisation as possible. The regular expression code for covers is super fast now, so that should help, and I improved the speed of the search node. The main delay remains the RottenTomatoes rating, because it adds additional delays in obtaining the RT webpage. I actually considered making the imdb.com rating default, but then today I found out that RottenTomatoes sometimes has summaries (synopsis as they call them) for some movies that lack a summary on the imdb.com website. Unfortunatly RottenTomatoes does not have short synopsis, so this does conflict with the default short summary setting of the scraper, but I thought it was better to have a long synopsis then an empty short summary. I also did a couple of test imports to measure the speeds of the RT and imdb.com rating settings. Default RT rating: [INDENT]30-Mar-2011 00:45:31 Info [ MovieImporter]: Watcher queued The Matrix (tt0133093).avi for processing. .... 30-Mar-2011 00:45:35 Info [ FileBasedResource]: Added cover art for "The Matrix" from: [url]http://.[/url].. 30-Mar-2011 00:45:36 Info [ FileBasedResource]: Added backdrop for "The Matrix" from: [url]http://.[/url].. .... 30-Mar-2011 00:45:36 Info [ MovieImporter]: Added "The Matrix" (1999).[/INDENT] So that took 5 seconds total for details, cover and backdrop. Then I removed movie from import folder so it was removed and deleted the thumbs folder, adjusted scraper settings and reimported the scraper with imdb.com ratings enabled. [INDENT]30-Mar-2011 00:49:36 Info [ MovieImporter]: Watcher queued The Matrix (tt0133093).avi for processing. .... 30-Mar-2011 00:49:38 Info [ FileBasedResource]: Added cover art for "The Matrix" from: [url]http://.[/url].. 30-Mar-2011 00:49:39 Info [ FileBasedResource]: Added backdrop for "The Matrix" from: [url]http://.[/url].. .... 30-Mar-2011 00:49:39 Info [ MovieImporter]: Added "The Matrix" (1999).[/INDENT] So that time it took 3 seconds. Now this is only a single movie test, so not very accurate, but you get a rough idea on the difference. For me personally it makes no difference how long each import takes, because they happen in the background for me while I watch something else. For those of you who do care this might make you want to use the imdb.com rating, so adjust settings accordingly. Ohh and incase it wasn't clear, v3.1.1 is now available to download from first post :D [/QUOTE]

Verification