Creepiness
I've been contemplating and discussing the idea of a pervasive recommender system. I'm find most people find it creepy moreso than intriguing. I don't know if it would always have been like this, but particularly in this modern age of corporate malfeasance and secret government torture facilities people just don't trust organizations to act responsibly.
I'll be honest that I don't necessarily like it all that much myself. It kinda makes me skirmish to guess how much Google knows about me.
Privacy has been called the ability to pretend to be like everyone else. As a researcher I want as much raw data as possible. What steps can be taken to make people feel comfortable?
Two main things come to mind:
- Allow a complete separation of a person's recreational and professional personas. If I'm a respected writer by day and dress up in a bunny suit for sexual pleasure at night, privacy is the capacity to keep those two lives separate.
- Give the user complete control over their data. At any time if I can see the specific information I have entered into the system from any source and I should be able to modify or at least delete it. If there are derived attributes that is the organizations discretion to disclose. Some dating sites create an emotional stability score. They don't tell members because having a computer tell you you're unstable is slightly insulting to many.

In terms of users owning their own data, that is related to the typically European approach to privacy(I "own" data about me), but it does get sticky in this area, since recommender systems are about "lots" of people, and often derived from "side channel" information(what do I click on, how long do I listen to/read something).
Unfortunately, its a complicated issue with few good answers. Greg Linden has written about this too. Also, remember the clickstream/search data AOL released awhile back? And how they found the identities of many people in the anonymized data?
I also have to worry about this in the analysis I do of power usage data.
Ownership
The recent opening of data from Facebook seemed to indicate that people aren't all that concerned. Though do you remember a while back a company made a site that allowed you to search through all your livejournal posts? It took a person's username and password and simply scraped the whole thing. The problem was it also scraped the private entries from their friends and then made those public. There was a bit of a fuss about that.
I think that at the least letting people know what the computer knows would be nice. Giving people control over that is something I don't know of another system doing.