UK IMDb Scraper | Page 3

Discussion in 'Moving Pictures' started by Mew, December 13, 2009.

  1. Dodgy Bob
    • Premium Supporter

    Dodgy Bob MP Donator

    Joined:
    July 10, 2008
    Messages:
    117
    Likes Received:
    3
    Gender:
    Male
    Ratings:
    +3 / 0
    Home Country:
    Wales Wales
    Show System Specs
    I have got it as "top priority" :)

    I've also done the old "re-read internet info." thing and specifically chosen this one. But I'll try disabling the others anyway.

    Cheers.

    Hmmm. Can't remember exactly what I did now but i got an error while in the Data Source dialog box, and now the UK source doesn't show in the list. But when I try and add it, it says it already exists.



    Found out what I did! :)

    If you select a source and then pressing the up arrow to move it's priority to the top too many times (i.e. "above" the top) causes the error.

    I know this is an error with MovingPictures and not with this script but it's preventing me testing this script now :(
     
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. Dodgy Bob
    • Premium Supporter

    Dodgy Bob MP Donator

    Joined:
    July 10, 2008
    Messages:
    117
    Likes Received:
    3
    Gender:
    Male
    Ratings:
    +3 / 0
    Home Country:
    Wales Wales
    Show System Specs
    Righto. I've found the new official scraper from the MovingPictures issue tracker.

    All I've done is changed the certification reg. exp. line, the name and modified the ID (no idea how this will affect the other scrapers). I've tested it and it seems to work fine.

    I hope the original authors don't mind the change and obviously use at your own risk, etc. I think it's being re-versioned to 2.0.0 now.
     

    Attached Files:

    • Like Like x 3
  4. funkstar
    • Premium Supporter

    funkstar Retired Team Member

    Joined:
    August 9, 2005
    Messages:
    771
    Likes Received:
    27
    Location:
    Home
    Ratings:
    +27 / 0
    Home Country:
    Scotland Scotland
    I certainly don't mind. I didn't realise there had been a version bump until now :)
     
  5. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,239
    Likes Received:
    1,336
    Ratings:
    +1,745 / 2
    I posted it on the forum last week :)

    https://forum.team-mediaportal.com/moving-pictures-284/imdb-scraper-new-layout-need-testers-86960/

    Just needed testers first, as new layout hasn't been visible to a lot of users. It's still mainly in the labs of IMDb, but they are deploying it to small regions, such as Australia. I guess IMDb wants to make sure it works first.

    And it will be revisioned to v2.0.0 due to the massive amount of changes made. While adding the changes to support the new layout, I simply decided to convert all the XML expressions into CDATA so that they are much easier to edit in the future. In the end the file changed much more, hence the reversioning.

    It will get bundled with next update, so you can just wait for that.
     
  6. Dodgy Bob
    • Premium Supporter

    Dodgy Bob MP Donator

    Joined:
    July 10, 2008
    Messages:
    117
    Likes Received:
    3
    Gender:
    Male
    Ratings:
    +3 / 0
    Home Country:
    Wales Wales
    Show System Specs
    RoChess : Sorry, I hadn't seen your post. For the record though, I spent most of yesterday rescanning loads of films using my modified version of 1.6.1 and it worked fine (for UK certs). Where a UK certification didn't exist (e.g. Defendor) it picked up the US one.
     
  7. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,239
    Likes Received:
    1,336
    Ratings:
    +1,745 / 2
    Since you can build filters based on certification results, it is usually preferred to have consistent results. So to prevent mixing UK and US certs, you could modify the scraper to make a replacement call based on US cert. It won't be pretty (certification usually isn't a direct conversion), but it's a rare case anyway when UK certs are not present and it's usually close enough.

    You get code such as:

    Code (Text):
    1.  
    2.       <!-- If UK certification failed, use converted US one -->
    3.       <if test="${movie.certification}=">
    4.             <parse name="certification_usa" input="${details_page}" regex="${rx_certification_usa}" />
    5.             <!-- Fix localized rating, forced to swap order, as 'G' gets replaced inside 'PG' and 'PG-13' otherwise -->
    6.             <replace name='certification1' input='${certification_usa[0][0]:htmldecode}' pattern='X' with='R18'/>
    7.             <replace name='certification2' input='${certification1}' pattern='NC-17' with='18'/>
    8.             <replace name='certification3' input='${certification2}' pattern='R' with='15'/>
    9.             <replace name='certification4' input='${certification3}' pattern='PG-13' with='12'/>
    10.             <replace name='certification5' input='${certification4}' pattern='PG' with='PG'/>
    11.             <replace name='certification6' input='${certification5}' pattern='G' with='U'/>
    12.             <set name='movie.certification' value='${certification6}'/>
    13.       </if>
    14.  
    You place this after you first try to obtain the UK certification, and be sure to add the CDATA expression, which i renamed to '_usa' postfix.

    Enjoy.
     
  8. Dodgy Bob
    • Premium Supporter

    Dodgy Bob MP Donator

    Joined:
    July 10, 2008
    Messages:
    117
    Likes Received:
    3
    Gender:
    Male
    Ratings:
    +3 / 0
    Home Country:
    Wales Wales
    Show System Specs
  9. funkstar
    • Premium Supporter

    funkstar Retired Team Member

    Joined:
    August 9, 2005
    Messages:
    771
    Likes Received:
    27
    Location:
    Home
    Ratings:
    +27 / 0
    Home Country:
    Scotland Scotland

    Attached Files:

  10. youngie10

    youngie10 Portal Member

    Joined:
    November 19, 2010
    Messages:
    7
    Likes Received:
    1
    Ratings:
    +1 / 0
    Home Country:
    England England
    Hey thanks alot for the uk scraper, i'm using the 2.0.2 version and i'm having a problem, every time i add it as a source in the moving pictures config it works fine first but then when i add films afterwards it doesn't use the new uk one and all the info is not there, i think it's using themoviedb.org cos i go into config and the uk imdb scraper is not there i have to keep adding it and rescanning my films. Anyone else having this problem or know how to solve it? Thank you very much in advance.

    edit:here's a copy of the log, i get this error at the end "Priority and List.IndexOf out of sync... " i don't understand a word of it :sorry: wonder if that has anything to do with it?
     
  11. Mew

    Mew Portal Pro

    Joined:
    January 11, 2007
    Messages:
    353
    Likes Received:
    55
    Gender:
    Male
    Occupation:
    COBOL/DB2 IBM z/Series Mainframe Developer
    Location:
    Hertfordshire
    Ratings:
    +56 / 0
    Home Country:
    England England
    Show System Specs
    Hi youngie10

    I have noticed a similar issue in the current version of Moving Pictures (1.0.8) that it does not seem to save the importer in the list.
    I think if you close the Moving Pictures configuration and then go back in it is still there but if you close MediaPortal configuration and then try to go back in, it is gone and the grabber defaults to a different importer. You can add the importer again and it works (so does not complain about duplicate IDs) for that session.

    Mew
     
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!