I am going to respond to "buffy fan's" questions in a new comment rather than burying it deep in the thread where her or his questions occur, because the questions resonate with things a number of people seem to be curious about.

With respect to our micro-genres, buffy fan says, "would it not be more logical to emphasize peer behavior, like Amazon does?"

I think this is a great question and for those people who are interested in such things, I am going to address it in this comment.

First, I think it might be useful to give some history, so that readers may be a bit less likely to assume that if we aren't doing things the way that appears most logical upon a quick examination of the problem, it's not because of ignorance, laziness, etc. There are OTHER reasons.

For those who aren't interested in that aspect and are only interested in a direct response to buffy fan's questions, please scroll down to "DIRECT RESPONSE TO BUFFY FAN'S QUESTIONS".

I have noticed that in the discussions here, there seems to be a fundamental assumption on the part of some intelligent people that we are being... how do I say it... stupid in our approach. That we haven't even considered certain obvious issues and problems and strategies. That the first thing an intelligent reader of these threads might think of is obviously a better strategy than ours.

I think that such assumptions are made because intelligent people can get jaded by the amount of crap out there. There ARE many crappy web sites out there, created by people who think that good marketing, rather than good technology, is the way to create a successful company.

But this company is honestly technology-driven. We don't even have a marketing person aboard at this point. It happens that I was the first person to create a commercial system that does what buffy fan suggests, in the mid-80's. It was a voice-mail-based dating service that monitored people's keystrokes in response to voice ads of the opposite sex... you could fast-forward over them, reply, store them away for later listening etc. These were considered to be "passive ratings" and we formed a cluster of similarly-minded peers around every user; these peers were used to recommend people of the opposite sex.

I know that it was the first such system because I've talked to people like the CEO and top tech guys at Net Perceptions (NP), which supplied Amazon.com's technology, and everyone so far seems to agree that mine was the first commercial application (or maybe the first application period; that's harder to establish) of the principles that a decade later came to be known as "Collaborative Filtering" (CF).

It also happened that later CF technology produced by my company beat the published technology of NP's people (they were at the U of Minnesota or something like that, I'm not sure I'm remembering the exact school as I write this) before they started NP. At the time they were academics and published their work in academic papers.

We tested ours against theirs on a database of actual movie ratings and ours more reliably found peers that made recommendations that actually turned out to be liked by the person receiving the recommendations. Again, this was the core task to the exact solution that buffy fan proposes (if I understand him or her correctly).

You may have heard of another company, Firefly, that came out of the MIT Media Lab with similar technology. We also tested ours against their published academic work and ours again proved superior at finding peers who could make reliable movie recommendations. They were subsequently purchased by Microsoft.

And I know exactly why ours beat theirs in both cases (and in fact in the case of every peer-finding technology we tried). It was because they didn't put much effort into theirs. They used the most obvious things as their solutions. That was enough to get a paper published or to get funding (at least if you're a personal friend of a top-5 software industry VC, as one of the founders of NP was ). It was shocking to me. But that's how it was. Whereas we DID seek, from the beginning, to do things as well as we could from a technological perspective, not because we are morally superior or geniuses, but because we are engineers instead of marketers.

In fact, when I criticized Firefly's technology to their CEO, his response was not, as I expected, "We're working on it, here's what we're doing..." Rather it was, and these are his EXACT words, "You can make up for that with marketing."

So I don't blame people here for assuming the worst about us -- that we aren't serious about technology, that any smart person who thinks about it for 10 minutes can come up with a better solution or figure out why our site will never work, etc. I do think that is often the case with new companies, particularly in the Internet space, so it is rational to initially assume we would be in that group. But there are other kinds of companies too, driven by quality of engineering, which we seek to emulate: 3M and Google are among them.

DIRECT RESPONSE TO BUFFY FAN'S QUESTIONS:

Going back to buffy fan's question: I agree, You are right, it DOES make sense to "to emphasize peer behavior" as you suggest which is why I have been responsible for the creation of a fair amount of technology based on that principle.

However, I believe that the evolutionary micro-genre principle has the potential to be more exciting, which is why we are going to bring it onto the site before the more traditional peer technology you suggest. (BTW, we have already created such technology, based on information theoretic principles, and written the code for it, and it's waiting in the wings.)

buffy fan: "Who actually has such well defined musical tastes?" I agree that that question makes a lot of sense with respect to the kinds of rigid, superficial genres that the music industry has used so far in defining genres. Personally, I have never had much use for those kinds of genres. For instance, if I get on ANY Internet radio station, no matter how many HUNDREDS of streams it may have each depicting a specific small genre, I have NEVER consistently liked the music.

So, if we were talking about those kinds of genres, I would agree with you, they would have no use to me either. I consider it proven that those kinds of genres do not work.

But we are envisioning genres that evolve directly out of community need. The genres that best serve the actual community's need for focused taste-centers will emerge, we posit. This is totally different than the top-down approach where some industry analyst decides how to group music and what to call the groups. This is a bottom-up approach. I believe the results will be very different.

In fact, if I called them emergent-evolving-micro-taste-centers rather than micro-genres, I suspect that this whole question might not have even emerged. But I just think that micro-genres is a more succinct term.

The taste-centers that emerge should be clusters where people with ACTUAL similar tastes want to congregate in order to most effectively share music with like-minded people... that is, peers.

I posit that these taste-centers will not be far from the clusters that are created by more traditional nearest-neighbor (peer) based clustering techniques, but will have some advantages, like having distinct identities, the ability to be reached quickly through navigating a hierarchy rather than by inputting hundreds of ratings and mathematically comparing yours to those of other people, and others.

buffy fan: " Surely people tend to like a variety of styles?" Of course. I, for one, like Beethoven, jazz guitarist Ralph Towner, folk songwriter Leonard Cohen, and music of other types. I'd say my tastes are quite varied.

I see this as a nice thing about the micro-genre approach. When I am in the mood for songwriting with mystical, poetic lyrics, I will go to one micro-genre. When I'm in the mood for lively jazz sax playing, I will go to another. Having this kind of micro-genre will make it easy for me to do that.

Whereas, if I am grouped with peers who happen to also like Beethoven, John Coltrane, and Leonard Cohen, as our earlier CF technology, and Net Perceptions, and Firefly's all did, I will get finely tuned suggestions, but they won't necessarily be what I am looking for at the time, and I will have had to enter hundreds of ratings (or made many purchases) to get those recommendations.

But you're right... that peer approach does capture something special. I currently think that the main common thread in music that I like is that it is honest. So, if peer-based recommendation technology considers all the artists I like, it will find peers who also like honest music instead of manipulative, saccharine music, EVEN THOUGH THERE ARE NO TRADITIONAL GENRES FOR "HONEST" MUSIC. IMHO that's the great strength of the peer approach, and that's why I spent years focused on developing a sound theoretical framework for doing that.

However... I posit that the evolutionary, emergent micro-genre approach WILL DISCOVER THAT SAME INFORMATION. I think there will be micro-genres that capture honest folk songwriting as opposed to manipulative, saccharine folk songwriting, which would be captured in other micro-genres. The names of those genres will come from the fans of artists in those genres, so I assume that the "manipulative, saccharine" folk songwriting genre will be called "Romantic Folk Music" or something like that. ;)

I think that such genres will emerge because they are being created to suit the needs of individual music lovers rather than a corporation's need to categorize for mass-marketing purposes. They are a totally different animal.

NOW, that being said, I am quite aware that I may be wrong about what will happen with our evolving, emergent micro-genres. I don't think I am, but I may be. It is, after all, an experiment -- one that can only be meaningfully conducted in public on a large scale with a many, many users. That's why it's going to go directly into our Web site rather than proven in a lab somewhere first.

SO, we also have a more traditional recommendation technology waiting in the wings. It finds peers by means of information-theoretic calculations. It is, I believe, a step up from our earlier peer-finding technology. It finds peers who have similar tastes without restriction to traditional genres and groups them together for purposes of recommendation. We have already written the code for this. We just aren't integrating it into the site now because we think the emergent micro-genre technology is a more interesting and exciting. Unless the emergent micro-genre (I guess I'll call it EMG) technology works out incredibly will, we plan on introducing the peer-finding technology subsequently. We don't know how much value it will add over EMG because EMG hasn't been tested yet.