Overhaul IMDb+ rating system (1 Viewer)

[biship]

Would it be possible to change the way the single rating is calculated by letting us include or exclude ratings in the single (averaged) score?
ie, a way to select via the GUI:
[X] IMDB
[ ] Metacritic
[ ] RT Top Critics
[X ] RT All Critics
[ ] RT Audience Percentage
[X ] RT Audience Rating

Currently I seem to only be able to use IMDB or an average of all of them and I can't exclude any.
For example, google Bohemian Rhapsody, and most ratings put it around 80/100, however Metacritic and RT critics lower it to 6.4 in IMDB+.
This is exclude the "97% liked this movie - Google users" - which seems like a nice score to include.

Being greedy, I'd also love a way to weight the ratings individually for a single score:
50% IMDB
10% Metacritic
10% RT Top Critics
13% RT All Critics
12% RT Audience Percentage
5% RT Audience Rating

 

[RoChess]

Anything is possible, and I plan to look into this when I undertake the massive rewrite effort.

Until then all I can work on are small baby-step adjustments.

The main problem is that RT killed my API key as I lacked activity for a while when I retired my HTPC with a 3TB drive failure. By the time I resolved that they had expired my key and now made it super difficult to obtain a new one. That means I cannot do any debugging on the IMDb+ ratings with RT which must be broken by now.

Is your API key still working?

Otherwise it is down to just IMDb and their MetaCritics score, but the existing IMDb+ options should allow you to use either by itself, or merge them via average. Are you saying that is broken as well? What I'm working on whenever I have a moment is a fix for the 'actors' as I wanted to start to import 10 names into Moving-Pictures now after some skin changes, and realized that new RegExp code looks at JSON values that max out around 3 or 4 names. So, have to adjust that to be flexible enough to also look at HTML entries so I can get many more.

Got the RegExp finished last night:

Cast\sOverview[^<]+</td></tr>\s*(?:<tr[^>].+?>\s*(?<actor>[^<]+?)\s*</a>\s*</td>\s*<[^"]+"ellipsis"[^>]*>.+?</tr>\s*){3,10}

For the new actors_block, but I have to merge it properly where the JSON would act as a backup in case the HTML fails, without duplicating entries or causing other issues. That requires a bit more scripting that I might not have time for this weekend.

 

[biship]

The key I created 3 months ago on developer.fandango.com is still not approved, and fails when attempting to use it. I doubt it will ever be approved.
I think the single score is just IMDb and MetaCritic, which typically does not align with users ratings.

 

[RoChess]

Nope that should work.

IMDb only:

• global_options_imdb_score = true
• global_options_imdb_metascore = false
• global_options_single_score = true

MetaCritics only:

• global_options_imdb_score = true
• global_options_imdb_metascore = true
• global_options_single_score = true

The average of IMDb and MetaCritics scores with votes combined:

• global_options_imdb_score = true
• global_options_single_score = false

If that fails, then I need to see scraper-debug active log files (as per IMDb+ wiki) to find out why it is failing.

PS: Maybe it is confusing that IMDb and MetaScore need to be set to "true" to obtain those values, but that is because both are obtained from imdb.com website which is how that logic works.

 

[biship]

No longer working for me.

I set these options:
IMDb only:

• global_options_imdb_score = true
• global_options_imdb_metascore = false
• global_options_single_score = true

<?xml version="1.0" encoding="utf-8"?>
<imdbplus>
  <set id="01" name="global_options_original_title" value="false" />
  <set id="02" name="global_options_foreign_title" value="false" />
  <set id="03" name="global_options_foreign_first" value="false" />
  <set id="04" name="global_options_uk_rating" value="false" />
  <set id="05" name="global_options_imdb_score" value="true" />
  <set id="06" name="global_options_imdb_metascore" value="false" />
  <set id="07" name="global_options_long_summary" value="true" />
  <set id="08" name="global_options_rotten_meter" value="false" />
  <set id="09" name="global_options_rotten_average" value="false" />
  <set id="10" name="global_options_rotten_top_critics" value="false" />
  <set id="11" name="global_options_special_edition" value="true" />
  <set id="12" name="global_options_rename_titles" value="false" />
  <set id="13" name="global_options_single_score" value="true" />
  <set id="14" name="global_options_min_imdb_votes" value="false" />
  <set id="15" name="global_options_refresh_all_fields" value="true" />
  <set id="16" name="global_options_one_writer_director" value="false" />
  <set id="17" name="global_options_remove_first_roman" value="false" />
  <set id="18" name="global_options_fix_missing_summary" value="false" />
  <set id="94" name="global_options_english_movie_rating" value="01" />
  <set id="95" name="global_options_secondary_en_title" value="false" />
  <set id="96" name="global_options_secondary_summary" value="false" />
  <set id="97" name="global_options_secondary_details" value="01" />
  <set id="98" name="global_options_country_filter" value="us|ca|gb|ie|au|nz" />
  <set id="99" name="global_options_language_filter" value="en" />
</imdbplus>

Debug log: https://drive.google.com/file/d/1gkimvMBbP2DkigTMYgsx9BHKTjtg7_Jg/view?usp=sharing

 

[RoChess]

EDIT: Never-mind, something else is going on for you, it appears you got blocked from accessing imdb.com when it was time to get the main-details page:

29-Jun-2020 12:15:00 Debug [         ScraperNode]: executing retrieve: <retrieve name="details_page" url="${site}" encoding="UTF-8" /> 
29-Jun-2020 12:15:00 Debug [         ScraperNode]: Retrieving URL: https://www.imdb.com/title/tt0463854/ 
29-Jun-2020 12:15:11  Warn [          WebGrabber]: Connection failed: Reached retry limit of 5. URL= 
29-Jun-2020 12:15:11 Debug [         ScraperNode]: Assigned variable: details_page =

Can you think of anything that might cause that? Security software, firewall, etc.?

I'm going to have to add extra code that looks for this and log a big fat error message.

PS: Keep in mind that you can be on the HTPC in a browser and it works fine, but that the HTTPClient engine that C# uses for Moving-Pictures is denied. Still if there is a firewall/security issue going on the browser should quickly tell you. It could be that your system for example does not have the November 2019 SSL root-certificate update, so it fails to verify the SSL/TLS-credentials and then aborts the connection.

Original reply:

@biship there's something weird going on. The vote RegEx to obtain all the scores is failing for you, so it keeps the old data you had, but when I try it on "28 Weeks Later (2007)" I get:

Whereas your logs show:

29-Jun-2020 12:15:12 Debug [         ScraperNode]: executing parse: <parse name="imdb_votes" input="${details_page}" regex="${rx_imdb_votes}" />
29-Jun-2020 12:15:12 Debug [         ScraperNode]: name: imdb_votes ||| pattern: title="(?<imdbScore>\d+(?:.\d+)?) based on (?<imdbPopularity>[^\s]+) user ratings"(?:.+?<div class="metacriticScore[^"]+">\s*<span>(?<metaScore>\d+)</span>.+?>(?<metaPopularity>[^\s]+) crit[^<]+</a>)? ||| input: 
29-Jun-2020 12:15:12 Debug [         ScraperNode]: Assigned variable: imdb_votes.count = 0
29-Jun-2020 12:15:12 Debug [         ScraperNode]: Parse node returned no results...

However you did not add the HTML source option, so I cannot tell if IMDb is messing with you and giving you different HTML-data than me.

Please edit your IMDb+ config file, and add:

<set id="19" name="global_options_debug_imdb_sources" value="true" />

And then recreate the log file exactly like you did.

That should hopefully tell me what on earth IMDb.com is doing different for you.