Polish and Spanish language scraper support (merged)

Discussion in 'IMDb+' started by piernik, November 19, 2011.

  1. piernik

    piernik Portal Pro

    Joined:
    October 22, 2008
    Messages:
    141
    Likes Received:
    26
    Ratings:
    +26 / 3
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. NightOwl69

    NightOwl69 Portal Member

    Joined:
    September 4, 2007
    Messages:
    18
    Likes Received:
    2
    Ratings:
    +2 / 0
    Re: IMDb+ Scraper - Movie Scraping on Steroids v3.3.9

    Thanks this script is great. I modified an old version slightly so when it's configured to use UK ratings it gets Australian ratings as follows.

    <if test="${global_options_uk_rating}=true">
    <set name="rx_certification">
    <![CDATA[
    (?:>\s*Australia:|Rated\s*?)(?<movieCertification>(?:C)|(?:G)|(?:pG)|(?:MA)|(?:M)|(?:R)|(?:X))(?:</a>|\s*?for)
    ]]>
    </set>

    It would be great if you could extend its capabilities to select a country of choice for the ratings?

    I'm just trying to figure out where your new extension installer puts the script at the moment so I can make the same mod to the latest version.
     
  4. pantav

    pantav Portal Pro

    Joined:
    January 5, 2009
    Messages:
    68
    Likes Received:
    5
    Ratings:
    +5 / 0
    Home Country:
    Spain Spain
    Re: IMDb+ Scraper - Movie Scraping on Steroids v3.3.9

    Hello,

    I decide not to wait to Spanish full suport for your scrapper and installed it. First: tanks.

    The scrapper is not showing in MovinPictures movie details scrappers list. I searched in the C: unit for all the files named with imdb an I can't not find the scrapper.

    Finally I browsed to google code and dowloaded and added the scrapper, but maybe there are an error in the installer. I never used it before this time (and i unistalled and reinstalled three times).

    The Imdb+ plugin works. I configured the scrapper (switching to wide skin, because black glass doesn't support the new features).

    The log says:

    Code (Text):
    1. 19/11/2011 18:36:47[MPMain][1][Info] Starting IMDb+ v1.3.0.188
    2. 19/11/2011 18:36:47[MPMain][1][Info] Using language es
    3. 19/11/2011 18:36:47[MPMain][1][Warning] Cannot find translation file C:\ProgramData\Team MediaPortal\MediaPortal\Language\IMDb+\es.xml. Falling back to English
    4. 19/11/2011 18:36:47[MPMain][1][Debug] Loading IMDb+ options from file.
    5. 19/11/2011 18:36:47[MPMain][1][Debug] Saving IMDb+ options to file.
    6. 19/11/2011 18:36:47[Extension Settings][12][Info] Adding hooks to MPEI Settings
    7. 19/11/2011 18:36:47[Extension Settings][12][Warning] Unable to add hooks into MPEI Settings, Extensions plugin not installed or out of date!
    8. 19/11/2011 18:36:47[MPMain][1][Debug] Set property "#IMDb.Replacements.Count" to "903" successful
    9. 19/11/2011 18:36:47[MPMain][1][Debug] Set property "#IMDb.Replacements.Custom.Count" to "0" successful
    10. 19/11/2011 18:36:47[MPMain][1][Debug] Set property "#IMDb.Replacements.Version" to "1.0.6" successful
    11. 19/11/2011 18:36:47[MPMain][1][Debug] Set property "#IMDb.Replacements.Published" to "26/10/2011" successful
    12. 19/11/2011 18:36:47[MPMain][1][Info] Last script update time: 18/11/2011 20:41:37
    13. 19/11/2011 18:36:47[MPMain][1][Info] Plugin initialization complete.
    14. 19/11/2011 18:38:25[MPMain][1][Debug] IMDb+ source not installed!
    15. 19/11/2011 18:45:46[MPMain][1][Debug] Saving IMDb+ options to file.
    16. 19/11/2011 18:45:56[MPMain][1][Debug] IMDb+ source not installed!
    17. 19/11/2011 18:46:25[MPMain][1][Debug] Saving IMDb+ options to file.
    18. 19/11/2011 18:46:25[MPMain][1][Debug] Saving IMDb+ options to file.
    19. 19/11/2011 18:46:25[MPMain][1][Info] Goodbye
    20.  
    I openend and searched for the file also in the installer, but there are no file similar to the scrapper.
     
  5. RoChess
    • Premium Supporter

    RoChess Extension Developer

    Joined:
    March 10, 2006
    Messages:
    4,209
    Likes Received:
    1,324
    Ratings:
    +1,717 / 2
    Re: IMDb+ Scraper - Movie Scraping on Steroids v3.3.9

    I can add support for any other movie website that allows me to get a direct match when I provide it with an IMDb tt-ID. The multi-language support is done at details node only, so imdb.com is used to do the search-node to locate the correct movie, and then the correct IMDb tt-ID is available, which can then be used at other website that supports it.

    So far this includes imdb.de (German), imdb.fr (French), imdb.es (Spanish), imdb.pt (Portugese), imdb.it (Italian), filmtipset.se (Swedish), kvikmyndir.is (Icelandic) and finally moviemeter.nl (Dutch).

    Find me a Polish movie site that gets me to "Avatar" via tt0499549 and I can add support for it.

    This is one of the enhancements already planned for next version, but because it also required an update to IMDb+ plugin to expose the new scraper-script option, I could not release a new version yet. If you want to use your modifications now, you need to use an SQLite tool to edit the 'scripts' table, because that is where scraper-scripts have always been stored by MovingPictures. The last row should be the current v3.3.9 edition of IMDb+.

    The IMDb+ plugin downloads and installs the IMDb+ scraper-script and inserts it directly into the MovingPictures database once you start MediaPortal GUI and the plugin is run for the first time. We're trying to expand it so that you can do the download+install of the scraper-script while inside MediaPortal config, but that is easier said then done.

    When you open MovingPictures config, and manually manage the scraper-scripts, what do you all see then? It should look something like:

    [​IMG]

    Look for "IMDb+" in there, if it is grey then use the [+] icon to enable it, and move it up to the top position to ensure that it gets used. If you do not see it, try re-installing the IMDb+ plugin and launch MediaPortal GUI to see if it will then try to install it for you. You should see:

    [​IMG]

    Which should show v3.3.9 got installed (current version of IMDb+ scraper-script).

    Then you can open IMDb+ plugin and adjust all the options to your liking (such as Spanish for language), and then use the hidden menu option "Force IMDb+" to switch all your existing movies over and use the refresh method to update all your existing movies via the IMDb+ scraper.
     
    • Like Like x 1
  6. pantav

    pantav Portal Pro

    Joined:
    January 5, 2009
    Messages:
    68
    Likes Received:
    5
    Ratings:
    +5 / 0
    Home Country:
    Spain Spain
    Re: IMDb+ Scraper - Movie Scraping on Steroids v3.3.9

    Yes, this is the problem. The first time, I started MP and a similar message (about install the scrapper) showed in the screen (my skin shows only the first part, so I can't read the entire message).
    After this, in MovinPictures there are no IMDB+ entry, nor enabled nor disabled.

    I uninstalled and reinstalled the plugin 3 times (with a computer restart for safety). But now, when I start MP no message appears (and the log is the attached log). And in Moving Picture no IMDB+ entry appears.

    So, I installed manually the script from Google Code (and configured it modifiyng the xml). My problem is resolved, but maybe there are a problem in the plugin (it thinks its installed but no).
     
  7. ltfearme
    • Premium Supporter

    ltfearme Community Plugin Dev

    Joined:
    June 10, 2007
    Messages:
    6,562
    Likes Received:
    4,437
    Gender:
    Male
    Occupation:
    Software Test Engineer
    Location:
    Sydney
    Ratings:
    +5,659 / 0
    Home Country:
    Australia Australia
    Re: IMDb+ Scraper - Movie Scraping on Steroids v3.3.9

    Right now, the IMDb+ plugin only installs\upgrades the scraper script when you are in the MediaPortal GUI. In the next version it will also install it during the installation of the MPEI package (but it will only install it if NOT installed as the upgrade over the wire is done in GUI).
     
    • Like Like x 2
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!