Wednesday, January 24, 2007

Revolution Health and the Wisdom of Crowds

A couple of weeks ago I was invited to preview the new RevolutionHealth.com health portal site prior to its official launch on 22 January 2007 (see the press release).

Revolution Health was founded in March 2005 by Steve Case, best known as the co-founder and former chief executive officer and chairman of America Online (AOL). According to the press release, Revolution Health's goal is "to create products and services that empower people by putting them at the center of the health system." A year and a half ago, when Revolution Health was pitching its services to pharma companies, some executives thought it would have a huge impact on healthcare, but others pooh-poohed the idea.

Now comes the RevolutionHealth.com portal site. I am particularly interested in the "social networking" aspect of the site where users can rate products and post comments, specifically in the Conditions & Treatments (shown below).


In this area of the site, visitors can rate Rx drugs and other treatments. According to the site, "Revolution Health users have rated more than 4,000 remedies. Check out the most preferred treatments—or add your own rating to help everyone else." I decided to explore more about ED and insomnia treatments.

Add Your Own Rating
In the Erectile Dysfunction area several treatments were listed including the well-advertised Rx drugs (Viagra, Cialis, and Levitra) as well as some lesser advertised ones (Caverject) and some more exotic treatments and devices (vacuum pumps, MACA, and Horny Goat Weed). Visitors can rate these in terms of effectiveness, side effects, ease of use, and cost effectiveness. The rating for Viagra is shown on the left. It was rated better than Horny Goat Weed, which received only one rating (overall: 3.0). The guy who rated it gave it a zero for effectiveness (duh!). However, he did give it a 5.0 for cost effectiveness, which beat out Viagra. Levitra, with the lowest US market share among the top 3 Rx's, beat all with an overall rating of 8.3 (based on only 9 votes). Does this mean that Levitra has potential if only it were marketed better? I don't think we have enough data here to make that conclusion.

Where's the Crowd?
Revolution Health's treatment rating system is intriguing although flawed. I could find no guidelines for what to consider when voting. What does cost effectiveness or ease of use mean, for example? These can mean different things to different people. There is the problem of statistical significance as well. Twenty or 36 votes is hardly a large enough sample to achieve any "wisdom of crowds" effect.

Revolution Health acknowledges this problem:
IMPORTANT MESSAGE: This service is in its infancy, but our philosophy is that if millions of people participate, it will emerge as a very useful tool for people as they consider their treatment options.
RevolutionHealth.com also allows visitors to post comments about treatments. One comment I saw right off the bat was negative about Viagra:
TOPIC: Too awkward
"Yes, it gave me an erection but only if I didn't have ANY fat or oil in my meals that day - it apparently is negated by fat and this can be a strong effect in some individuals. It also gave me a constant runny nose, flushing, and an anxious feeling. I was disappointed with the shortness of its effectiveness - the window period for me was one or two hours - after that, it didn't help. I consider Viagra a good beginning to solving this problem. Cialis was a big improvement."
Good for Cialis but not for Viagra. Who knows? Maybe this was a post made by a Cilais brand team member or an ad agency working for Lilly?

I am a great proponent for the use and support of social networking tools by pharmaceutical marketers. But this kind of thing might scare them off as advertisers. Or maybe not.

Also, as a user, I would be wary of the possibility that the system can be "gamed" by people with an agenda. I've seen this happen in my own surveys, for example. Maybe it happens also on non-health Web sites that sell products like flat screen TVs. But there, the buying decision is different (you don't need a prescription -- another person's permission -- to buy one) and the consequences of a bad decision is much less critical to your health.

Podcast Interview
Anyway, I haven't learned enough about RevolutionHealth.com to really be too critical. That's why I plan to interview Jay Silverstein, the COO of Revolution Health, in my next Pharma Marketing Talk live! podcast (more info about that here).

5 comments:

  1. Anonymous12:05 PM

    "It also gave me a constant runny nose, flushing, and an anxious feeling."

    As required by law, I'm assuming all Pfizer employees and their agents who read this blog and/or use this new site will report this and other adverse events to the FDA.

    The law's pretty clear - "Each applicant having an approved application under 314.50 or, in the case of a 505(b)(2) application, an effective approved application, shall promptly review all adverse drug experience information obtained or otherwise received by the applicant from any source, foreign or domestic, including information derived from commercial marketing experience, postmarketing clinical investigations, postmarketing epidemiological/surveillance studies, reports in the scientific literature, and unpublished scientific papers."

    See the problem with unfiltered consumer-generated content, John? Am I missing something?

    ReplyDelete
  2. You are expanding the law to cover "agents", which is incorrect, IMHO. Only employees of pharma companies are required to do whatever it is that is required.

    And what exactly is required? The law is NOT clear -- at least the part that you quote. Does "review" mean "report" to the FDA? I always thought that was the requirement, but now that I see the actual language, I am not sure. Maybe I missed something.

    Anyway, we all know that pharma companies and their agents read user generated content (UGC) on the Internet and I'm sure none of what they access anonymously is ever reported, much less reviewed. Who's to know? Is BIG BROTHER watching us?

    Anyway, the pharmaceutical industry is always ready to use the AE card whenever it talks about the Internet and especially when considering whether or not to get involved with UGC or social networking. I believe, however, that this is just an excuse or unfounded fear and not a real deterrent.

    Pharma marketers don't have to -- and probably should not -- insert themselves into the conversation or attempt to manipulate or even monitor the conversations. They should, however, make sure that their messages (ads) are served on sites where this content can be found. As long as there is separation between what is considered editorial content and what is considered advertising, it's all good. Why not take advantage of the growing popularity of these sites?

    ReplyDelete
  3. Anonymous12:53 PM

    And the consumer ratings are nearly useless if the site doesn't micro-classify them. This means collecting extensive health information -- a massive privacy problem -- and using it to categorize the ratings. What about comorbid conditions? Previous treatment failure? Age group? Genetic issues? Drug interactions? Was the regimen optimal, and did the patient adhere? Given all the givens, what degree of success would we have expected for this patient?

    Also: Is the patient necessarily capable of measuring success? Maybe s/he had a miserable time with side effects and hated the treatment, but the alternative would have been far worse. Or maybe, as often happens, positive changes took place that the patient wasn't capable of perceiving.

    A better way would be break down responses into consumer vs. HCP, and then to make outcomes sortable/filterable by very specific health criteria -- or to measure outcomes against expectations. After that, we could carefully define the regimens used. Then we could try some stat analyses...

    ...and so on, until the site became a giant set of clinical trials.

    I'd love this to be useful, but it sounds like a return to the patient-testimonial era in drug advertising. The trial process was developed to resolve exactly these reliability issues.

    ReplyDelete
  4. You make some interesting comments and I must say that I have some of the same concerns. However, just to be clear, but this is not advertising, although it is a form of patient testimonial.

    It would be good to be able to rate the raters so to speak and even segment the audience in ways that you suggest. Yahoo! Answers, for example, lets visitors rate the answers that they get to their questions. The person answering gets a score based on these ratings and others can see who provides the most valuable answers in the eyes of the community.

    This might work on RevolutionHealth.com if the raters of treatments also participated in discussions and were rated by readers on their overall knowledge of the subject matter (eg, ED). I don't know, however, if this is how RevolutionHealth.com does things.

    One of the tenets of the book "The Wisdom of Crowds" is that often crowds of non-experts make more accurate decisions than one or two experts. Also, while each person in the crowd has an incomplete set of information to work with, somehow the crowd makes a decision as if had all the information it needed.

    You'll have to read the book to see if you agree or not with its conclusions and how it may be applicable here.

    ReplyDelete
  5. Anonymous11:37 PM

    Make sure you ask Jay about the company's health background too, like how many employees have a health background as a percentage of total employees... 5%? 20% 50%? If you're going to dole out health information and advice, even with the "wisdom of the crowds" in the background, you'd better have a decent medical staff backing you up.

    These ratings seem to have basic statistical problems in that they are in no way representative of the population -- sampling bias. Any statistician sees this, but this important differentiation seems to get lost in people clamoring for the "wisdom of the crowds." If the crowd is wrong, so is the wisdom. E.g., earth = flat.

    ReplyDelete