Automated Artist / Album Scraping

Discussion in 'Listen Music' started by jameson_uk, August 29, 2011.

Thread Status:
Not open for further replies.
  1. jameson_uk
    • Premium Supporter

    jameson_uk Retired Team Member

    Joined:
    January 27, 2005
    Messages:
    7,258
    Likes Received:
    2,228
    Gender:
    Male
    Occupation:
    Data Warehousing Consultant
    Location:
    Birmingham
    Ratings:
    +2,405 / 0
    Home Country:
    United Kingdom United Kingdom
    Show System Specs
    Now in MPEI
    Please install via MPEI installer, MPEI plugin or direct from here
    ============================================================
    This has moved on a bit now so this includes downloading of
    • Artist Thumbs
    • Album Thumbs
    • CD Art
    • Artist Clear Logo
    • Artist Bio - choice of allmusic.com or theaudiodb.com (including multi-language on the audiodb.com)
    • Album reviews - choice of allmusic.com or theaudiodb.com (including multi-language on the audiodb.com)
    This still need fanart handler to make the images appear inside MP and will not overwrite existing fanart (so you might want to remove existing fanart)
    ============================================================
    This is a very very rough attempt at scraping artist and album info automatically.

    I can not stress enough that this is a very early version of some code so has many many issues.

    basically place the attached file in

    C:\Program Files\Team MediaPortal\MediaPortal\plugins\process
    C:\Program Files (x86)\Team MediaPortal\MediaPortal\plugins\process

    depending on whether you are running 32 or 64 bit.

    Then when you open MP (this is currently the only time scraping is triggered) it will go off and start looking for info.

    This is designed for 1.2 RC (I don't think it will work with 1.2 beta).

    I am looking for some feedback on what success rates people get. Any areas that are obviously missing?

    Currently it will only scrape main albums so no compilations, EPs, singles etc. It also appears to have an issue with some special characters. Some times it is simply a case of allmusic.com not having the right data to be able to make a decision. Eg. if you search for something like Michael Jackson you actually get a surprising number of hits hence the automated scraped is unable to figure out which one to choose.

    I plan to update this during the life of 1.2 so that people have working systems but I am looking to implement something much better in 1.3 :)

    Please let me have feedback


     
    Last edited: September 28, 2014
    • Like Like x 17
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. mbuzina
    • Premium Supporter

    mbuzina Retired Team Member

    Joined:
    April 11, 2005
    Messages:
    2,839
    Likes Received:
    627
    Location:
    Germany
    Ratings:
    +723 / 4
    Home Country:
    Germany Germany
    Show System Specs
    Sounds cool, will check it out later today. Will it download thumbs or "just" the allmusic.com description of album / artist?
     
  4. jameson_uk
    • Premium Supporter

    jameson_uk Retired Team Member

    Joined:
    January 27, 2005
    Messages:
    7,258
    Likes Received:
    2,228
    Gender:
    Male
    Occupation:
    Data Warehousing Consultant
    Location:
    Birmingham
    Ratings:
    +2,405 / 0
    Home Country:
    United Kingdom United Kingdom
    Show System Specs
    It will not download any thumbs only the info
     
    • Like Like x 1
  5. ysmp
    • Team MediaPortal

    ysmp Design Group

    Joined:
    May 17, 2008
    Messages:
    1,861
    Likes Received:
    701
    Gender:
    Male
    Location:
    Seoul.
    Ratings:
    +790 / 1
    Home Country:
    South Korea South Korea
    Great news ...:)
    will give feedback tonight ..

    :D
     
  6. 1stdead

    1stdead Portal Pro

    Joined:
    February 1, 2008
    Messages:
    1,089
    Likes Received:
    139
    Location:
    Copenhagen
    Ratings:
    +139 / 0
    Home Country:
    Denmark Denmark
    What's the difference from fanarthandler? Only fanarts?

    Anyways if there is any difference, why not combine them :)?
     
  7. ysmp
    • Team MediaPortal

    ysmp Design Group

    Joined:
    May 17, 2008
    Messages:
    1,861
    Likes Received:
    701
    Gender:
    Male
    Location:
    Seoul.
    Ratings:
    +790 / 1
    Home Country:
    South Korea South Korea
    Hi 1stdead !
    FH plugin scrap artist fanart and artist Thumbnail only , this Automated scraping is for info abut the artis / album to save and view in the skin ,this option alrady in mp but the way it's work now have to scrap 1 by 1 , wich for a big music collection rally pain ..:) many of us ask for option to to it Auto .. and this what jameson_uk doing ....

    i ask for this in fanart handler long time ago ... but as i know cul8er he worry abut mamory problams with fanart handler ,so he verry carfull abut new add-on for FH plugin ,and noting bin down abut it ...
     
  8. emphatic
    • Team MediaPortal

    emphatic Design Group

    Joined:
    August 25, 2006
    Messages:
    3,586
    Likes Received:
    828
    Gender:
    Male
    Occupation:
    Surface treatment
    Location:
    Alingsås
    Ratings:
    +1,083 / 2
    Home Country:
    Sweden Sweden
    Show System Specs
    Does this have the power to overwrite null values? When you've used the MediaPortal built in info tool (Album/Artist info) and it has come up with a blank you can't erase, it would be very nice if this would just disregard it. I've downloaded it and will try it ASAP.

    Emph
     
  9. mbuzina
    • Premium Supporter

    mbuzina Retired Team Member

    Joined:
    April 11, 2005
    Messages:
    2,839
    Likes Received:
    627
    Location:
    Germany
    Ratings:
    +723 / 4
    Home Country:
    Germany Germany
    Show System Specs
    OK, have installed it and after a short wait I pressed info on a track (which was not (yet) scraped by your plugin). The result was a hanged MP. Will the plugin generate logs? I have not seen it in my quick log scan.

    As my Music DB is >1000 Artists large I am not sure on which album I can get the info while the process runs. I will restart MP and let it run for a while.
     
  10. ysmp
    • Team MediaPortal

    ysmp Design Group

    Joined:
    May 17, 2008
    Messages:
    1,861
    Likes Received:
    701
    Gender:
    Male
    Location:
    Seoul.
    Ratings:
    +790 / 1
    Home Country:
    South Korea South Korea
  11. Pog
    • Premium Supporter

    Pog Retired Team Member

    Joined:
    September 7, 2009
    Messages:
    401
    Likes Received:
    220
    Location:
    Wicklow
    Ratings:
    +315 / 1
    Home Country:
    Ireland Ireland
    Show System Specs
    So happy the music section of MP is getting some love! Bring it on!
     
Loading...
Thread Status:
Not open for further replies.

Users Viewing Thread (Users: 0, Guests: 0)

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
  • About The Project

    The vision of the MediaPortal project is to create a free open source media centre application, which supports all advanced media centre functions, and is accessible to all Windows users.

    In reaching this goal we are working every day to make sure our software is one of the best.

             

  • Support MediaPortal!

    The team works very hard to make sure the community is running the best HTPC-software. We give away MediaPortal for free but hosting and software is not for us.

    Care to support our work with a few bucks? We'd really appreciate it!