[Evaluate] - The Movie Database... first script, first trouble | Page 2

Discussion in 'Submit: Movie Info Grabbers' started by lopez.tuparles, March 17, 2013.

  1. megahorst
    • Super User

    megahorst Super User

    Joined:
    July 8, 2006
    Messages:
    877
    Likes Received:
    173
    Ratings:
    +271 / 0
    Home Country:
    Germany Germany
    Show System Specs
    Thx for this .

    I have tested this version.



    Much more details are scraped. The age limitations are also there.
    The only thing that I have seen is that movie details with a " do not work. The " is replaced by a \ and the rest is ignored.

    Example:
    http://www.themoviedb.org/movie/9657-otto-der-film
     
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. Fred777

    Fred777 Portal Member

    Joined:
    March 30, 2013
    Messages:
    9
    Likes Received:
    2
    Gender:
    Male
    Ratings:
    +3 / 0
    Home Country:
    Switzerland Switzerland
    Hi @lopez.tuparles,

    I had a look at your script as @megahorst told me about it.
    First point is that I didn't noticed the problem with " do not work. The " that you reported @megahorst. The movie Otto - Der film seems to work fine for me, everything (except empty fields of course) are well retrieved. But perhaps I missed something, could you provide a screenshot on configuration tool?

    Second point regarding code is that it could be clearly optimized:
    1. You should not retrieve movie details in FindFilm() (call to GetTMDBMovieDetails()) this can make search very long on some movies, try with "Cars" you will understand
    2. You can really simplify your code using deserialization functionalities of JavaScriptSerializer object to parse JSON. This allow to create a dictionnary in which you can easily navigate to retrieve data. Here is a small code example:

    JavaScriptSerializer deserializer = new JavaScriptSerializer();
    Dictionary<string, object> deserializedDictionary = deserializer.Deserialize<Dictionary<string, object>>(reader.ReadToEnd());
    Dictionary<string, object> movie = deserializedDictionary["movie"] as Dictionary<string, object>;

    if (movie.ContainsKey("title") == true)
    movieDetails.Title = (string)movie["title"];
    else
    movieDetails.Title = (string)movie["originalTitle"];

    (where "movie" is the name of the root object, I don't know its name in TMDB as I have not tested. I used this script for Allocine API data retrieval before they improved their security on the API few days ago)
    This will avoid you to use regular expression and thus avoid data extraction problems. (I hate regular expressions)
    Let me know if you need help on this (you can PM me in French)
    Fred
     
    Last edited: April 27, 2013
  4. megahorst
    • Super User

    megahorst Super User

    Joined:
    July 8, 2006
    Messages:
    877
    Likes Received:
    173
    Ratings:
    +271 / 0
    Home Country:
    Germany Germany
    Show System Specs
    Hello,

    the issue with Otto -Der Film is still there for me.
    @Fred777: Have you switched the script to german language?

    The same is for Elling:
    In tmdb.org i see:
    The script fetches the following into the database:
    What I have also mentioned:
    I have added some movies to the database. The following movies where not found by the script although they are avaliable at tmdb.org:

    "In guten Händen" (the ä?)
    "Nur für Personal!" (the !?)
    "Adams Äpfel" (the Ä?)
    "Fliegende Fische müssen ins Meer" (the ü?)
     
    Last edited: April 28, 2013
  5. lopez.tuparles
    • Team MediaPortal

    lopez.tuparles Retired Team Member

    Joined:
    July 20, 2006
    Messages:
    396
    Likes Received:
    34
    Location:
    Mimet
    Ratings:
    +46 / 0
    your right about these caracters. Try this new version.;)
     
    • Like Like x 1
  6. megahorst
    • Super User

    megahorst Super User

    Joined:
    July 8, 2006
    Messages:
    877
    Likes Received:
    173
    Ratings:
    +271 / 0
    Home Country:
    Germany Germany
    Show System Specs
    I have tested:
    "In guten Händen"
    "Nur für Personal!"
    "Adams Äpfel"

    Are still not there.
    "Fliegende Fische müssen ins Meer" is found now.

    Movies with " in the description now deliver a better result. The description is there now. Instead of " a \" is scraped.
     
  7. Wbunaarf
    • Team MediaPortal

    Wbunaarf Test Group

    Joined:
    December 9, 2005
    Messages:
    534
    Likes Received:
    89
    Location:
    Sweden
    Ratings:
    +98 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    I have all movies in individual folders. While many of the folders for various reasons have crappy names they are all properly tagged with their IMDB-IDs, e.g. "crappy.name.xyz.xvid.blabla (2013) (tt1234567)". MP recognises the ID, tries to do the search using it and fails.

    I had a quick look at the code and it seems to me as if gettitles(), using Grabber.API_SearchMovie + "?api_key=" + Grabber.API_KEY; where API_SearchMovie = API_BaseUrl + "search/movie";, is always called no matter what it's searching for. I think that in order to search using IMDB IDs you need to use something like API_BaseURL + "movie/" + IMDBID + "?api_key=" + Grabber.API_KEY;. If I'm right, perhaps this could be added?

    //W
     
  8. lopez.tuparles
    • Team MediaPortal

    lopez.tuparles Retired Team Member

    Joined:
    July 20, 2006
    Messages:
    396
    Likes Received:
    34
    Location:
    Mimet
    Ratings:
    +46 / 0
    Well, it seems your right. it might be possible in TMDB API to add Imdb Id instead of TMDB Id.
    Have a look on first page and set API_IMDB_LOOKUP to true

    Code (C):
    1.  
    2.         #region<<TMDB_PARAMETERS>>
    3.         public const bool API_IMDB_LOOKUP = false; //IMDB LookUp
    4.         public const string API_KEY = "......................."; //Insert API Key
    5.         public const string API_LANG = "fr"; //Change it to change Language
    6.         private const string API_BaseUrl = "http://api.themoviedb.org/3/";
    7.         #endregion<<TMDB_PARAMETERS>>
    8.  
     
    Last edited: May 13, 2013
  9. Wbunaarf
    • Team MediaPortal

    Wbunaarf Test Group

    Joined:
    December 9, 2005
    Messages:
    534
    Likes Received:
    89
    Location:
    Sweden
    Ratings:
    +98 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    I just did a Quick test.

    Code (Text):
    1.         public const bool API_IMDB_LOOKUP = true; //IMDB LookUp
    2.         public const string API_KEY = "..."; //Inset API Key
    3.         public const string API_LANG = "sv"; //Change it to change Language
    On my first run I got no hits. The title passed to the script is on the form tt1234567 which the regexp didn't seem to catch. Since I don't really speak regexpish and didn't fully understand what you where doing I simplified it to

    Code (Text):
    1. System.Text.RegularExpressions.Regex rx = new System.Text.RegularExpressions.Regex(@"tt(?<jocker>\d*)");
    Using this modified regexp I tested on a subset of my movies and it seems to work just fine. I'm currently doing a full scan which should be complete in the morning.

    //W
     
  10. Wbunaarf
    • Team MediaPortal

    Wbunaarf Test Group

    Joined:
    December 9, 2005
    Messages:
    534
    Likes Received:
    89
    Location:
    Sweden
    Ratings:
    +98 / 0
    Home Country:
    Sweden Sweden
    Show System Specs
    With the above change most movies seems to have been correctly added. However, 16 of them are named 'unknown'.

    Some, like tt0095226 and tt0095260, simply doesn't seem to exist at tmdb, Should these really be added?

    Some, like tt0268531 and tt0795421, have title, sort title, tagline, studio and country set to unknown while plot, director, writer, rating, actors and genres have proper information.

    //W
     
  11. azzuro
    • Team MediaPortal

    azzuro Test Group

    Joined:
    May 10, 2007
    Messages:
    8,214
    Likes Received:
    2,294
    Occupation:
    technician HVAC
    Location:
    France - IDF
    Ratings:
    +3,920 / 13
    Home Country:
    France France
    Show System Specs
    i'm sorry guys : we are 2 work for scrapping TMDB what is the good thread ?
    one by @Fred777 and the second by @lopez.tuparles
    Fred work is already ONLINE and no lopez
    but i think we need have only one TMDB for French. and i hope compatible with other country.

    eh les mecs vous bossez ensemble ?
     
Loading...

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!