Filmtipset.se - Swedish grabber

Discussion in 'Moving Pictures' started by vuego, August 16, 2009.

  1. vuego
    • Team MediaPortal

    vuego Documentation Group

    Joined:
    August 5, 2006
    Messages:
    1,565
    Likes Received:
    651
    Occupation:
    Alchemist, Astrobiologist, Exotic dancer
    Location:
    Göteborg
    Ratings:
    +857 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    I've completed a first draft of a Swedish scraper. I did some more comparisons between filmdelta, moviezine and filmtipset.se and the site with most information about movies seems to be filmtipset.se.

    The scraper currently grab all available information. Filmtipset.se is only missing certification, language and tagline. It currently grabs the movie review if it's available. If not the normal summary is grabbed. I'm not really sure which is preferred. Some people might not want too much text and others find the summary too generic.

    I'll show one example, from the movie Mirrors.
    Filmtipset.ses summary:
    Filmtipset.ses review:
    Perhaps we should have two versions of the script.

    I tried the scraper with 50 movies and it auto approved 45 of them, all correct. For the other 5 the correct movie was in the possible matches list. I might add a function that retrieves the details page of every possible match to grab the imdb number for better matches but it'll probably be 10 extra web requests per movie which will slow down the possible matches list population...

    Any volunteers to try out the scraper?

     
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. vuego
    • Team MediaPortal

    vuego Documentation Group

    Joined:
    August 5, 2006
    Messages:
    1,565
    Likes Received:
    651
    Occupation:
    Alchemist, Astrobiologist, Exotic dancer
    Location:
    Göteborg
    Ratings:
    +857 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    Here's the scraper. Load it up in Importer Settings / Movie Details Data Sources and raise the priority to put it first in the list.
    If you already got your movies approved in the Movie Manager tab, you can select all, click "Remove Movie and Ignore Files" and then "Unignore All Files" at the Movie Importer tab to rescan using the new scraper.

    And please report any troubles here :)
     
    • Like Like x 3
  4. TheWho

    TheWho Portal Pro

    Joined:
    December 26, 2008
    Messages:
    67
    Likes Received:
    1
    Ratings:
    +1 / 0
    Home Country:
    Sweden Sweden
    i cant use it, it won't scrape..... it stops at around 50% on the process bar...
     
  5. vuego
    • Team MediaPortal

    vuego Documentation Group

    Joined:
    August 5, 2006
    Messages:
    1,565
    Likes Received:
    651
    Occupation:
    Alchemist, Astrobiologist, Exotic dancer
    Location:
    Göteborg
    Ratings:
    +857 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    I've also noticed that the importer stalls sometimes. I think it's caused by themoviedb.org which is very slow. Just let it run for some minutes and it'll finish.
     
  6. mr viggo
    • Premium Supporter

    mr viggo MP Donator

    Joined:
    March 1, 2008
    Messages:
    500
    Likes Received:
    58
    Ratings:
    +67 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    The scraper works pretty well over here. But I think the review is a bit to mush, especially when it's written i first person, looks a bit silly sometimes if you ask me. Here's the review from ARN 2:
    "Jo, nu är vi framme vid den andra och avslutande del av Arn-filmatiseringen av Jan Guillous tre böcker om Arn.

    Historien
    Historien fortsätter där den första filmen slutade..."

    And the summary:
    "Arn återvänder hem för att skapa enighet och fred i sitt eget land och äntligen gifta sig med Cecilia. Väl hemma upptäcker han..."

    The summary looks way better!
     
  7. vuego
    • Team MediaPortal

    vuego Documentation Group

    Joined:
    August 5, 2006
    Messages:
    1,565
    Likes Received:
    651
    Occupation:
    Alchemist, Astrobiologist, Exotic dancer
    Location:
    Göteborg
    Ratings:
    +857 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    I think you're right. I first included the review for moviezine.se. They publish more professional reviews but lack many movies. I then moved over to filmtipset which got all movies ever made (?) but I see now that the reviews often are written by illiterate fans.

    Here's a new version which only grabs the plot.
     
    • Like Like x 3
  8. Cazz

    Cazz Portal Pro

    Joined:
    May 30, 2008
    Messages:
    146
    Likes Received:
    1
    Ratings:
    +1 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    Very very nice.
    It works great.
    When MP 0.8.0 release I going to make a new database and use this as the primary information scrape :)
     
  9. mr viggo
    • Premium Supporter

    mr viggo MP Donator

    Joined:
    March 1, 2008
    Messages:
    500
    Likes Received:
    58
    Ratings:
    +67 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    Already the primary scraper here :p

    Just found a little odd summary for "Amelie från montmarte". It even looks wierd in the movie manager tab in m-pics configuration. Perhaps the issue lies with filmtipset.se?
     
    • Like Like x 1
  10. vuego
    • Team MediaPortal

    vuego Documentation Group

    Joined:
    August 5, 2006
    Messages:
    1,565
    Likes Received:
    651
    Occupation:
    Alchemist, Astrobiologist, Exotic dancer
    Location:
    Göteborg
    Ratings:
    +857 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    I remember seeing this problem and tried to fix it by removing newlines. Turns out that it isn't newlines but lots of spaces in the HTML code.

    I fixed it so here's version 1.1.1 :)
     
    • Like Like x 1
  11. emphatic
    • Team MediaPortal

    emphatic Design Group

    Joined:
    August 25, 2006
    Messages:
    3,579
    Likes Received:
    828
    Gender:
    Male
    Occupation:
    Surface treatment
    Location:
    Alingsås
    Ratings:
    +1,082 / 2
    Home Country:
    Sweden Sweden
    Show System Specs
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!