IMDb+ Scraper (Force English title, Auto-Rename titles to group, and more) v3.1.7 | Page 15

Discussion in 'Moving Pictures' started by RoChess, February 23, 2011.

?

Should this be the default imdb scraper?

Poll closed March 25, 2011.
  1. Yes, I do not want to re-import

    19 vote(s)
    95.0%
  2. No, keep this one seperate

    0 vote(s)
    0.0%
  3. Who cares, I got movies to watch

    1 vote(s)
    5.0%
  1. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,153
    Likes Received:
    1,294
    Ratings:
    +1,659 / 2
    Re: AW: IMDb+ Scraper (Force English title, Auto-Rename titles to group, and more) v3

    Thank you, I've put a lot of work into it so far, but the results on my own HTPC are worth it.

    As far a integrating my code into a different system, if the 'other' scraper also uses IMDb tt-IDs as a reference, then it's almost as easy as copy and paste. But if you adjusted it from IMDb tt-ID into OFDb IDs then more changes are needed and I guess that is where things went wrong. What I did with the IMDb+ scraper goes way outside of the scope of scrapers, so a lot of additional conditions have to be met. For example the rename database file has to be in the location defined by the scraper script with the same filename, which I used file="C:\Rename dBase IMDb+ Scraper.xml" for. This file has to be proper XML syntax, so perhaps your rename system messed that up. Use the XML synax checker inside the comments section to verify. You can copy and paste the content of the XML file into the online form and check.

    The rest is then pretty logical, I load the contents of the XML file into an array called rename_array. And then via the @nodes I compare the @id (which I use to store the IMDb tt-ID with inside XML file) to the movie.imdb_id, and on a match rename the title and/or sortby field via their respective @nodes. To allow for a change on title, sortby or title+sortby, I added the extra empty string check.

     
    • Like Like x 1
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. maximm

    maximm Portal Member

    Joined:
    July 24, 2011
    Messages:
    21
    Likes Received:
    3
    Ratings:
    +3 / 0
    Re: IMDb+ Scraper (Force English title, Auto-Rename titles to group, and more) v3.1.6

    Hi, I tried the scraper, but when getting IMDB scores, i still get alot of zeros, and most scores are incorrect. Am i doing something wrong?

    My bad, checked the script and im getting the rottentomatoes.
     
  4. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,153
    Likes Received:
    1,294
    Ratings:
    +1,659 / 2
    Re: IMDb+ Scraper (Force English title, Auto-Rename titles to group, and more) v3.1.6

    I was still working on some major improvements to the scraper, so when I caught the rating issue on a few movies I figured I could just combine it all into a single release. But the time to test the new features is taking longer then expected, and since it affects more movies then I thought it did, I've released v3.1.7 to solve that for the time being.

    With the new version that I'm working on, you will be able to adjust the global_options on the fly via an XML file, so that re-importing of the script in scraper-debug mode is no longer needed. It will also support conditional updates, meaning that on refresh of movie data you can configure the scraper to only update missing info and force update the scores+votes. This way a refresh will not only go faster, but will retain any custom changes you made to title, summary, etc.
     
  5. ltfearme
    • Premium Supporter

    ltfearme Community Plugin Dev

    Joined:
    June 10, 2007
    Messages:
    6,451
    Likes Received:
    4,231
    Gender:
    Male
    Occupation:
    Software Test Engineer
    Location:
    Sydney
    Ratings:
    +5,371 / 0
    Home Country:
    Australia Australia
    Typo in rename DB: Basic Instinc => Basic Instinct
     
    • Like Like x 1
  6. Merlyn

    Merlyn Portal Pro

    Joined:
    July 8, 2011
    Messages:
    250
    Likes Received:
    161
    Ratings:
    +162 / 0
    Home Country:
    Germany Germany
    Show System Specs
    AW: IMDb+ Scraper (Force English title, Auto-Rename titles to group, and more) v3.1.7

    Thanks or your reply, RoChess :)

    I havent found, what was causing the issues first, but after a good nights sleep and a restart of mp config it was working fine...

    However, since I was in my first attempts relying a lot on the rename xml file ( had added the ofdb id and the summary from ofdb as fields to the file), I did some more changes.

    I successfully managed to translate the imdb id to the ofdb id and read the movie title and summary from ofdb. And while I am writing this on one monitor and have the code on the other, I noticed, that after the correct title was set from ofdb it is again set to the title from the rename xml in the rename section... need to fix this... Does work atm cause I changed all the titles in the rename xml to the correct german title...

    Anyway, the point is, that I now can get the german titles and summaries if available on ofdb.org.

    Next thing, that bothers me and that I plan to change, is that for correct movie sorting a valid rename xml is required. I noticed, that imdb has the necessary info to correctly sort movies already on the movieconnections subpage. So I'll try and see, if I can get correct sorting and make the rename db optional. So that if a movie is not in the rename xml (it only has some 700+ movies right now) the info is parsed from imdb directly...

    Need to learn and understand regex first, though...

    Note on attachments: These are rather dirty hacks! Use on your own risk! That they work quite well for me does not mean, they will not mess up your database! Make a backup before using! Changed IMDB+ scraper version and date to 3.1.8 on 07/24/2011... Also not for public. only eval for RoChess.
     

    Attached Files:

  7. Furetto

    Furetto Moderator - Dutch Forums

    Joined:
    April 11, 2005
    Messages:
    664
    Likes Received:
    60
    Gender:
    Male
    Occupation:
    Network Admin
    Location:
    Brussels
    Ratings:
    +60 / 0
    Home Country:
    Belgium Belgium
    Show System Specs
    Found a small error, the IMDB id for Art of War II is wrong:
    <rename id="tt0123357" title="The Art Of War II: Betrayal" />
    Should be
    <rename id="tt1233571" title="The Art Of War II: Betrayal" />
    Otherwise you get "The people's court"
     
  8. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,153
    Likes Received:
    1,294
    Ratings:
    +1,659 / 2
    Thank you, hate it when typos sneak in.

    It will be part of v0.9 of the rename XML, which I'll release when v3.2.x is done. The users that are freaking out now can fix it themselves :D

    That is going to be a feature of v3.2.x is ability to have a custom rename XML file, so that any update to the default rename file can be added without loosing any of your custom edits. And a ton more exciting new things, so stay tuned :cool:
     
  9. DMember 49125

    DMember 49125 Guest

    Ratings:
    +0 / 0
    Hi RoChess,

    Is it possible to include an option that does not remove "The" from the beginning in the sortby value? (Exactly as the 3.0.7 version worked)
    :D
     
  10. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,153
    Likes Received:
    1,294
    Ratings:
    +1,659 / 2
    The way the scraper is now, is that it relies on the MovingPictures settings. So disable the article removal setting in advanced settings and it should work. If not then let me know and I'll retest that scenario in more detail, I'm currently improving the regular expression codes to make things run faster, and my screen is full with windows to test that, but it should work.

    Or are you saying you want the title to remove the article prefix and not remove it on the sortby?
     
  11. DMember 49125

    DMember 49125 Guest

    Ratings:
    +0 / 0
    Thank you very much! Everything is OK now.
    :D
     
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!