Automated Artist / Album Scraping (8 Viewers)

Status
Not open for further replies.

jameson_uk

Retired Team Member
  • Premium Supporter
  • January 27, 2005
    7,258
    2,528
    Birmingham
    Home Country
    United Kingdom United Kingdom
    Updated download in first post. This may take a while longer to scan but wondering if this picks up any more artists / albums ?
     

    Lbr_Lion

    Extension Designer
    July 19, 2008
    243
    372
    Home Country
    Netherlands Netherlands
    Hi jameson_uk,

    I did a fresh scan with the new version, but it does not find any artist info :( , but it picks up a little more albums.

    albumartists: 0 out of 176 (0%)
    albums: 168 out of 331 (50%)

    Please find attached the logs.
     

    jameson_uk

    Retired Team Member
  • Premium Supporter
  • January 27, 2005
    7,258
    2,528
    Birmingham
    Home Country
    United Kingdom United Kingdom
    I did a fresh scan with the new version, but it does not find any artist info :( , but it picks up a little more albums.
    Whoops.... unless you happened to be running a proxy server at 127.0.0.1:8888 this would not work :oops:

    New plugin in first post.

    There was an issue with album info in that the album artist field does not actually get added to the database for some reason :confused: This meant that it was checking for album info every time and would add a new record so you will end up with lots of duplicates.
    To solve this you either need to delete the whole database (then you would need to re-import your music) or connect to the database and delete records from artistInfo and albumInfo tables
     

    Lbr_Lion

    Extension Designer
    July 19, 2008
    243
    372
    Home Country
    Netherlands Netherlands
    I tested v3 and now only one error occur :) It looks like that Sinead O'Connor - Am I Not Your Girl? causes issues (I checked also the previous logs and every time the same error occur.

    the artist results are the same as in the first version (115 out of 176) and album info is the same as version 2 (168 out of 331)
     

    jameson_uk

    Retired Team Member
  • Premium Supporter
  • January 27, 2005
    7,258
    2,528
    Birmingham
    Home Country
    United Kingdom United Kingdom
    OK further version in the first post.

    This should sort out where the strip prefix option was ticked.
    It also adds a little more logic to sort out artist lookups so should pick up some more artists and albums.

    There are a couple of issues I know of. First off if the artist or album name ends in a "." this will not work. This is actually a .NET bug that MS do not seem to want to fix so not sure what I can do about this.
    AC/DC does not work as it always treats the "/" as a directory separator and again this appears to be something .NET can not actually cope with.

    Would be interested in what people are missing now. Looking at allmusic.com things like sinead o'connor quite simply appear to issues with allmusic.com (if you search for her it only brings up an odd classical page even though there is a real page. If you can figure out what you are missing then look up the data on allmusic.com to see if there is a reason why it is not there that would be helpful
     

    Pog

    Retired Team Member
  • Premium Supporter
  • September 7, 2009
    401
    315
    Wicklow
    Home Country
    Ireland Ireland
    Artists using strip prefix are still not been picked up.

    Big improvement with artists containing accents and punctuation in their names, though "Björk" is still being missed.

    Request: remove anything between brackets in album lookup. eg: I use brackets for specific album info like this "Abbey Road (remastered)" or "Abbey Road (vinyl 1967 issue)", others may have albums with disc info etc.

    I've been trying to see what's happening with some missed artists and allmusic.com...

    For "sinead o'connor", her name is actually spelt "Sinéad O'Connor" and having the accent seems to help as it brings up the usual page of results with to top result being the correct one.

    For artist "Air" which does not get picked up right now the returned search page has several entries all with the same relevance and the top result is for a jazz artist while I want the electronic one. I wonder if it would be possible to use the albums in the database listed for that artist to help drill down the results?

    For artist "Al Green" the search returns a list with top result being correct but has the same relevance as the second result. Actually both results with the same relevance are for the same artist. Maybe always take the top result?

    For artist "Björk" the top result and highest relevance is the correct one. Don't see why the plugin doesn't use it.
     

    Lbr_Lion

    Extension Designer
    July 19, 2008
    243
    372
    Home Country
    Netherlands Netherlands
    I used v4 and the results for artist and albums are better than the previous versions:

    Artists: 70% and Ablbums: 54%

    I checked some missing artists and it is related to multiple entries with a relevance of 100%.

    As Pog already mentioned it would be good to check as second step for related album(s) to decide what entry to use?

    For the missing albums it is simular as for artists, but maybe it is possible to check for a review and use this as selection option
     

    jameson_uk

    Retired Team Member
  • Premium Supporter
  • January 27, 2005
    7,258
    2,528
    Birmingham
    Home Country
    United Kingdom United Kingdom
    Artists using strip prefix are still not been picked up.
    Did not actually test this... will test over the weekend.

    For artist "Björk" the top result and highest relevance is the correct one. Don't see why the plugin doesn't use it.
    Prior to version 4 any search for an artist that went to a results page rather than an artist page was not picked up. In v4 I have added a check if there are only two entries and one is a classical entry with no years active.

    Will check out Björk but for me, sinead O'Connor (with or without accent) goes to classical page with no discography. I can only seem to access the real page by going there and then choosing the pop link.

    Keep letting me know what is not found and I will see if I can try a few more strategies to pickup the data
     

    Pog

    Retired Team Member
  • Premium Supporter
  • September 7, 2009
    401
    315
    Wicklow
    Home Country
    Ireland Ireland
    Will check out Björk but for me, sinead O'Connor (with or without accent) goes to classical page with no discography. I can only seem to access the real page by going there and then choosing the pop link.

    Ah yes I see what you mean... I was just looking at the returned search page and never clicked the link, indeed it goes to the same page accent or not. Darn!

    There is a tab on the search results page for pop music and that lists Sinéad O'Connor as the top result. Maybe prioritise the results for pop music? if that is possible as most likely for most people their search will be for pop music.
     

    jameson_uk

    Retired Team Member
  • Premium Supporter
  • January 27, 2005
    7,258
    2,528
    Birmingham
    Home Country
    United Kingdom United Kingdom
    There is a tab on the search results page for pop music and that lists Sinéad O'Connor as the top result. Maybe prioritise the results for pop music? if that is possible as most likely for most people their search will be for pop music.
    Yes could easily add that as a config option.

    Does not make a difference though ....

    Code:
    http://allmusic.com/search/artist/Sin%C3%A9ad%20O%27Connor/filter:pop
    Would be the link but that does not take you to the artist page but still goes to the listing. I can work around that but it is still a pain...

    Lets try v5 soon then :)
     
    Status
    Not open for further replies.

    Users who are viewing this thread

    Top Bottom