Recommendations experiment results thread

The Heart of the Prog/Power Movement

Moderators: TROY or maybe DEBBIE, Stevie WOnder, Digital Man

User avatar
elendil
Posts: 54
Joined: Wed Mar 08, 2017 11:41 pm
Location: Free State of New Hampshire

Recommendations experiment results thread

Postby elendil » Mon Aug 14, 2017 12:02 am

I've been working away at this project, and there are a few interesting results to start off with, though also some new reasons to make me think the whole concept is a lot more difficult than I initially thought.

I got 6 sets of album ratings, including my own. Of those, 4 were large files with 1000+ albums. Two rated just the 48 albums I recommended everyone rate. And ironically, the other three large files did not include the majority of those recommended! In fact, there were only 14 albums that all four people who sent 1000+ albums rated - and two of these people sent ratings for more than 3000 albums! It just goes to show how diverse tastes are among metal fans.

If that weren't enough, the correlations between ratings were low. Correlation is a statistical estimate of how similar two sets of numbers are - do they tend to "go together"? High correlations would make it easier for multiple imputation to get accurate estimates of "missing" ratings.

The two ratings that had the highest correlation were my file and that of Sir Exar Kun. But that correlation was a mere 0.16. Here's what a 0.16 correlation looks like in a scatter plot.

Image

(In case this doesn't show up well, here's the direct link: https://www.dropbox.com/s/6gz2pjyz843w8 ... r.png?dl=0 .)

Yes, that dotted line slopes upward, showing that in general, the higher SEK rates things, the higher I do too. But it's a weak relationship. There are lots of albums I rate high that he doesn't so much, and vice versa. (The scatter plot uses artist names rather than album names for marker labels, as they're shorter and a bit more visible. And in case you're curious, I added a random normal decimal to SEK's ratings so that they don't overprint so much in this graph.)

And even SEK and I rated only 294 of the same albums. That was by far the highest number of commonly rated albums among all pairs of the six submissions. Interestingly, the higher the correlation in ratings between any two submissions, the greater the number of albums they had in common. There was actually a slightly negative correlation between my ratings and that of Ryan Bowers (don't know his forum name): -0.04. And we only rated 133 of the same albums. OTOH, Ryan Bowers and GTog (Metal-Archives user) had a not-bad correlation of 0.14 and rated 139 of the same albums. Ryan and Sir Exar Kun rated 205 of the same albums and had a ratings correlation of 0.08. All this tells me that if you share similar tastes, you listen to more of the same albums. (Duh.)

Looking at the two submissions that only rated the 48 albums I recommended everyone rate, the correlations between my ratings and those of jimbies and kytokinesis were -0.17 and 0.12, respectively, while they correlated with each other at -0.04. These somewhat stronger correlations tell me that when you "force" everyone to rate albums they wouldn't otherwise listen to, you get more informative results about people's tastes.

So... long story short, de gustibus non est disputandum. Among six metal & prog fans, tastes were sufficiently diverse that *no one* had tastes extremely similar to anyone else. And that means it will be hard for other people's ratings of albums to tell you much about what your own will be. The only ways to solve this problem are to: 1) get more people involved with the ratings project and 2) have everyone rate more of the same albums, including ones you would never otherwise want to listen to. Given how hard the latter is, I'm thinking about alternatives. You could scrape Metal-Archives to get genre tags and build a selection model of which albums people have listened to... But now we're getting beyond ourselves. I'm going to soldier on under the heroic assumption that the data I have will tell us *something* informative about what we would like (and not like), notwithstanding all the problems the data are already showing up. Stay tuned...

User avatar
elendil
Posts: 54
Joined: Wed Mar 08, 2017 11:41 pm
Location: Free State of New Hampshire

Re: Recommendations experiment results thread

Postby elendil » Thu Aug 17, 2017 11:32 pm

Finished the recommendations and emailed them out. Thanks to all who participated! For gory statistical details & discussion, see here: https://www.metal-archives.com/board/vi ... 1&t=119066.


Return to “PM: Core”

Who is online

Users browsing this forum: No registered users and 2 guests