TMDB import problems (1 Viewer)

ponchick

Portal Member
May 9, 2013
6
3
Home Country
Russian Federation Russian Federation
MP cannot import some titles using TMDB. Network sniffer shows correct API answers, all necessary data received, but movie importer doesn't even show possible matches from this scrapper.
GET http://api.themoviedb.org/3/movie/tt0395479?api_key=cc25933c4094ca50635f94574491f320&language=ru HTTP/1.1
User-Agent: MovingPictures/1.5.1.1487
Accept: application/json
Host: api.themoviedb.org
Proxy-Connection: Keep-Alive

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Age: 0
Cache-Control: public, max-age=86402
Content-Type: application/json;charset=utf-8
Date: Thu, 09 May 2013 14:55:45 GMT
ETag: "8a300dd149e787e3b0326855c56f2156"
Server: nginx
Status: 200 OK
Via: 1.1 varnish
X-Cache: MISS
X-Memc: HIT
X-Memc-Age: 50495
X-Memc-Expires: 35907
X-Memc-Key: 1847c7f43230a607a02d0da26b510de7
X-Varnish: 2249901869
X-Varnish-Sid: tm-v-p-1200
Content-Length: 1017
X-Cache: MISS from atom.home
X-Cache-Lookup: HIT from atom.home:3128
Connection: keep-alive

{"adult":false,"backdrop_path":"/wFfHC9aQBDMc0a44XHTpld1VdjX.jpg","belongs_to_collection":{"id":189257,"name":"Короткометражные фильмы Pixar","poster_path":"/1WHtPF7LXQGl6PkqPxRby2488SN.jpg","backdrop_path":null},"budget":0,"genres":[{"id":16,"name":"мультфильм"},{"id":10751,"name":"семейный"},{"id":null,"name":"короткометражка"}],"homepage":"","id":13062,"imdb_id":"tt0395479","original_title":"Boundin'","overview":"Веселый мультфильм о приключениях, беззаботного горного барашка.","popularity":3.62820868,"poster_path":"/35jJseJi3BiQ2ucCxKpnUoJhal3.jpg","production_companies":[{"name":"Pixar Animation Studios","id":3}],"production_countries":[{"iso_3166_1":"US","name":"United States of America"}],"release_date":"2003-11-04","revenue":0,"runtime":0,"spoken_languages":[{"iso_639_1":"en","name":"English"}],"status":"Released","tagline":"","title":"Барашек","vote_average":8.0,"vote_count":8}

GET http://api.themoviedb.org/3/search/...e=1&language=ru&include_adult=False&year=2003 HTTP/1.1
User-Agent: MovingPictures/1.5.1.1487
Accept: application/json
Host: api.themoviedb.org

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=3600
Content-Type: application/json;charset=utf-8
Date: Thu, 09 May 2013 14:43:17 GMT
ETag: "3ac11112417c74072905982f2d5b4cb3"
Server: nginx
Status: 200 OK
Via: 1.1 varnish
X-Cache: MISS
X-Memc: MISS, STORE
X-Memc-Age: 0
X-Memc-Expires: 3600
X-Memc-Key: 2e115fee0228de21bbfd17358a2e577b
X-Varnish: 2249464322
X-Varnish-Sid: tm-v-p-1200
Content-Length: 323
Age: 748
X-Cache: HIT from atom.home
X-Cache-Lookup: HIT from atom.home:3128
Connection: keep-alive

{"page":1,"results":[{"adult":false,"backdrop_path":"/kcHvzJuHXsMNXwR9bjjsf7RX1lQ.jpg","id":13062,"original_title":"Boundin'","release_date":"2003-11-04","poster_path":"/35jJseJi3BiQ2ucCxKpnUoJhal3.jpg","popularity":3.288462604,"title":"Барашек","vote_average":8.0,"vote_count":8}],"total_pages":1,"total_results":1}

GET http://api.themoviedb.org/3/movie/13062?api_key=cc25933c4094ca50635f94574491f320&language=ru HTTP/1.1
User-Agent: MovingPictures/1.5.1.1487
Accept: application/json
Host: api.themoviedb.org

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Age: 0
Cache-Control: public, max-age=86402
Content-Type: application/json;charset=utf-8
Date: Thu, 09 May 2013 14:55:45 GMT
ETag: "8a300dd149e787e3b0326855c56f2156"
Server: nginx
Status: 200 OK
Via: 1.1 varnish
X-Cache: MISS
X-Memc: HIT
X-Memc-Age: 50495
X-Memc-Expires: 35907
X-Memc-Key: 1847c7f43230a607a02d0da26b510de7
X-Varnish: 2249902018
X-Varnish-Sid: tm-v-p-1200
Content-Length: 1017
X-Cache: MISS from atom.home
X-Cache-Lookup: HIT from atom.home:3128
Connection: keep-alive

{"adult":false,"backdrop_path":"/wFfHC9aQBDMc0a44XHTpld1VdjX.jpg","belongs_to_collection":{"id":189257,"name":"Короткометражные фильмы Pixar","poster_path":"/1WHtPF7LXQGl6PkqPxRby2488SN.jpg","backdrop_path":null},"budget":0,"genres":[{"id":16,"name":"мультфильм"},{"id":10751,"name":"семейный"},{"id":null,"name":"короткометражка"}],"homepage":"","id":13062,"imdb_id":"tt0395479","original_title":"Boundin'","overview":"Веселый мультфильм о приключениях, беззаботного горного барашка.","popularity":3.62820868,"poster_path":"/35jJseJi3BiQ2ucCxKpnUoJhal3.jpg","production_companies":[{"name":"Pixar Animation Studios","id":3}],"production_countries":[{"iso_3166_1":"US","name":"United States of America"}],"release_date":"2003-11-04","revenue":0,"runtime":0,"spoken_languages":[{"iso_639_1":"en","name":"English"}],"status":"Released","tagline":"","title":"Барашек","vote_average":8.0,"vote_count":8}

GET http://api.themoviedb.org/3/search/...&page=1&language=ru&include_adult=False&year= HTTP/1.1
User-Agent: MovingPictures/1.5.1.1487
Accept: application/json
Host: api.themoviedb.org

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=3600
Content-Type: application/json;charset=utf-8
Date: Thu, 09 May 2013 14:43:17 GMT
ETag: "3ac11112417c74072905982f2d5b4cb3"
Server: nginx
Status: 200 OK
Via: 1.1 varnish
X-Cache: MISS
X-Memc: MISS, STORE
X-Memc-Age: 0
X-Memc-Expires: 3600
X-Memc-Key: 4ccab2cf3e7a1e1d5b5b40ae66e910a8
X-Varnish: 2249464456
X-Varnish-Sid: tm-v-p-1200
Content-Length: 323
Age: 749
X-Cache: HIT from atom.home
X-Cache-Lookup: HIT from atom.home:3128
Connection: keep-alive

{"page":1,"results":[{"adult":false,"backdrop_path":"/kcHvzJuHXsMNXwR9bjjsf7RX1lQ.jpg","id":13062,"original_title":"Boundin'","release_date":"2003-11-04","poster_path":"/35jJseJi3BiQ2ucCxKpnUoJhal3.jpg","popularity":3.288462604,"title":"Барашек","vote_average":8.0,"vote_count":8}],"total_pages":1,"total_results":1}

GET http://api.themoviedb.org/3/movie/13062?api_key=cc25933c4094ca50635f94574491f320&language=ru HTTP/1.1
User-Agent: MovingPictures/1.5.1.1487
Accept: application/json
Host: api.themoviedb.org

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=86402
Content-Type: application/json;charset=utf-8
Date: Thu, 09 May 2013 14:55:45 GMT
ETag: "8a300dd149e787e3b0326855c56f2156"
Server: nginx
Status: 200 OK
Via: 1.1 varnish
X-Cache: MISS
X-Memc: HIT
X-Memc-Age: 50495
X-Memc-Expires: 35907
X-Memc-Key: 1847c7f43230a607a02d0da26b510de7
X-Varnish: 2249902018
X-Varnish-Sid: tm-v-p-1200
Content-Length: 1017
Age: 2
X-Cache: HIT from atom.home
X-Cache-Lookup: HIT from atom.home:3128
Connection: keep-alive

{"adult":false,"backdrop_path":"/wFfHC9aQBDMc0a44XHTpld1VdjX.jpg","belongs_to_collection":{"id":189257,"name":"Короткометражные фильмы Pixar","poster_path":"/1WHtPF7LXQGl6PkqPxRby2488SN.jpg","backdrop_path":null},"budget":0,"genres":[{"id":16,"name":"мультфильм"},{"id":10751,"name":"семейный"},{"id":null,"name":"короткометражка"}],"homepage":"","id":13062,"imdb_id":"tt0395479","original_title":"Boundin'","overview":"Веселый мультфильм о приключениях, беззаботного горного барашка.","popularity":3.62820868,"poster_path":"/35jJseJi3BiQ2ucCxKpnUoJhal3.jpg","production_companies":[{"name":"Pixar Animation Studios","id":3}],"production_countries":[{"iso_3166_1":"US","name":"United States of America"}],"release_date":"2003-11-04","revenue":0,"runtime":0,"spoken_languages":[{"iso_639_1":"en","name":"English"}],"status":"Released","tagline":"","title":"Барашек","vote_average":8.0,"vote_count":8}

Debug mode doesn't help much:

Code:
09-May-2013 22:43:12  Info [	  MovieImporter]: User reprocessing Барашек (Boundin') (2003) [tt0395479].mkv
09-May-2013 22:43:20  Info [	  MovieImporter]: No exact match for Барашек (Boundin') (2003) [tt0395479].mkv
Here some broken movies (not complete list):
https://www.themoviedb.org/movie/13062-boundin
https://www.themoviedb.org/movie/158-knockin-on-heaven-s-door
https://www.themoviedb.org/movie/13928-knick-knack
https://www.themoviedb.org/movie/17963-gone-nutty
https://www.themoviedb.org/movie/53985-mutant-pumpkins-from-outer-space
https://www.themoviedb.org/movie/13926-red-s-dream
 

RoChess

Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    For these situations, MovPic has what is called scraper-script debug mode. If you enable that, it will allow you to see exactly why the movie failed as you can see every step a scraper-script goes through. Well this works for external scraper-scripts such as IMDb+ anyway, never really tested it much for internal scraper-scripts such as TMDb, but should work the same.

    Steps on how to enable scraper-script debugging = http://code.google.com/p/imdbplus/wiki/WikiInstallScraper

    And be sure to include *full* MovPic log file, not just the parts you think are needed. And you will have to of course repeat the steps so that the log file contains the info needed. Easy way would be to send one of the failing movies back to the importer. Due to asynchrone nature and large log files in scraper-script debug mode, keep it down to just one movie that fails.

    Just tested and all those movies import fine with IMDb+ for me, so might be something specific to your setup or indeed something wrong with TMDb.
     

    ponchick

    Portal Member
    May 9, 2013
    6
    3
    Home Country
    Russian Federation Russian Federation
    As I mentioned before debug mode is on. Log are attached.
     

    Attachments

    • movingpictures.zip
      1.8 KB

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    As I mentioned before debug mode is on. Log are attached.

    If you had enabled scraper-script debug mode, alongside MediaPortal debug verbosity, then the log file would have shown lines containing the following:

    Debug [ ScraperNode]:

    This lacks from your logs, so there is nothing really in them that is of any help to properly diagnose the issue.

    Based on previous experiences the most likely answer is that TMDb has trouble with the Russian filename you use, and your MovPic noise filter ends up removing the English title or confuses TMDb even more in the search. If TMDb scraper is able to make use of the IMDb tt-ID, then more then likely this tt-ID is missing at TMDb website for these movies, causing it to fail when searching.

    IMDb+ would solve this for you, as it supports multiple methods to ensure IMDb tt-ID lookups result in an instant match. Or you can user contribute the missing data to TMDb, or adjust your noise filter to create a workable title, or adjust your filenames.

    If you wish to keep TMDb then adding the missing data would be the best way, as it will fix it not just for your collection, but the next user as well that might run into this problem.
     

    ponchick

    Portal Member
    May 9, 2013
    6
    3
    Home Country
    Russian Federation Russian Federation
    I forgot change MediaPortal log verbosity to debug. Updated log is attached but it does not explain the problem. Sniffer shows same network activity, scrapper get all data about that movie 3 times.
    I've 300+ movies and only 20 have problems with this scrapper. All folders named by pattern, all of them contain translated title and themoviedb.org scrapper works fine with them. Again, scrapper fetch all data about movie but cannot parse it.
    IMDB is not a option for russian - a lot of movie data still untranslated.
     

    Attachments

    • movingpictures.zip
      7.5 KB

    Edalex

    Community Plugin Dev
  • Premium Supporter
  • January 3, 2008
    2,955
    1,264
    Saratov
    Home Country
    Russian Federation Russian Federation
    Нет практически ни слова про themoviedb в логе.
    -----
    No word of tmbd in log
     

    ponchick

    Portal Member
    May 9, 2013
    6
    3
    Home Country
    Russian Federation Russian Federation
    themoviedb.org scrapper written in c#, not xml. Seems it has no logging capabilities at all.
     

    RoChess

    Extension Developer
  • Premium Supporter
  • March 10, 2006
    4,434
    1,897
    Guess it has to be step-traced inside Visual Studio then.

    @Edalex perfect job for you since you know C# and Russian.
     
    Last edited:

    Edalex

    Community Plugin Dev
  • Premium Supporter
  • January 3, 2008
    2,955
    1,264
    Saratov
    Home Country
    Russian Federation Russian Federation
    Fine, @RoChess ;)
    I found the problem
    There was json deserialization error
    Looks like, int& is needed somewhere instead of int, @ltfearme
    Failing here https://code.google.com/p/moving-pi...ingPictures/DataProviders/TheMovieDbAPI.cs#49
    So something in MovieInfo is wrong
    System.Runtime.Serialization.SerializationException перехвачено
    Message=ValueType "System.Int32" не может иметь значение null.
    Source=System.ServiceModel.Web
    StackTrace:
    в ReadMovieInfo.GenreFromJson(XmlReaderDelegator , XmlObjectSerializerReadContextComplexJson , XmlDictionaryString , XmlDictionaryString[] )
    в System.Runtime.Serialization.Json.JsonClassDataContract.ReadJsonValueCore(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
    в System.Runtime.Serialization.Json.JsonDataContract.ReadJsonValue(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
    в System.Runtime.Serialization.Json.XmlObjectSerializerReadContextComplexJson.ReadDataContractValue(DataContract dataContract, XmlReaderDelegator reader)
    в System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, DataContract& dataContract)
    в System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 id, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
    в System.Runtime.Serialization.XmlObjectSerializerReadContextComplex.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
    в ReadArrayOfMovieInfo.GenreFromJson(XmlReaderDelegator , XmlObjectSerializerReadContextComplexJson , XmlDictionaryString , XmlDictionaryString , CollectionDataContract )
    в System.Runtime.Serialization.Json.JsonCollectionDataContract.ReadJsonValueCore(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
    в System.Runtime.Serialization.Json.JsonDataContract.ReadJsonValue(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
    в System.Runtime.Serialization.Json.XmlObjectSerializerReadContextComplexJson.ReadDataContractValue(DataContract dataContract, XmlReaderDelegator reader)
    в System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, DataContract& dataContract)
    в System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 id, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
    в System.Runtime.Serialization.XmlObjectSerializerReadContextComplex.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
    в ReadMovieInfoFromJson(XmlReaderDelegator , XmlObjectSerializerReadContextComplexJson , XmlDictionaryString , XmlDictionaryString[] )
    в System.Runtime.Serialization.Json.JsonClassDataContract.ReadJsonValueCore(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
    в System.Runtime.Serialization.Json.JsonDataContract.ReadJsonValue(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
    в System.Runtime.Serialization.Json.XmlObjectSerializerReadContextComplexJson.ReadDataContractValue(DataContract dataContract, XmlReaderDelegator reader)
    в System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, DataContract& dataContract)
    в System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Type declaredType, DataContract dataContract, String name, String ns)
    в System.Runtime.Serialization.XmlObjectSerializerReadContextComplex.InternalDeserialize(XmlReaderDelegator xmlReader, Type declaredType, DataContract dataContract, String name, String ns)
    в System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName)
    в System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName)
    в System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(XmlDictionaryReader reader)
    в System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(Stream stream)
    в Cornerstone.Extensions.JsonExtensions.FromJson[T](String json) в D:\Documents and Settings\Дмитрий\Мои документы\Visual Studio 2010\Projects\moving-pictures\Cornerstone\Extensions\JsonExtensions.cs:строка 45
    InnerException:
     
    Last edited:

    Users who are viewing this thread

    Top Bottom