Cautionary Tale About a Web-Based Tool Gone Awry

I just returned from the Social Media, Mobile & Gaming for Pharma conference in New York City where I participated in a panel discussion focusing on the future of social, mobile, & gaming within the pharma industry. I was joined by Julie Wittes Schlack, Communispace Corporation, and Heather Guidone, Center for Endometrosis.

We discussed a number of issues including my "pet peeve," which is health web and mobile apps that use a formula or algorithm to generate a number to help a physician make a diagnosis. One example of such an app is Janssen's "Psoriasis" app for the iPhone and iPad. This app includes a "PASI calculator." PASI stands for "Psoriasis Area and Severity Index," which is a tool for the measurement of severity of psoriasis.

The problem with this app is that there is no documentation that tells the physician the source of the algorithm. Also, there is no guarantee that the algorithm was programmed correctly without any "bugs." You can read more about this here: "Some Unregulated Physician Smartphone Apps May Be Buggy."

Imagine my surprise upon opening up my email just now to find a press release from The JAMA Network titled "Cautionary Findings in Study of Web-Based Tool for Children with Flu-Like Illness" (here)!

Here are some excerpts from the press release:

"While parents and caregivers reported that a web-based triage tool for children with influenza-like illness was easy to understand and use, the tool’s underlying mathematical formula needs to be improved so it does not result in more, rather than fewer, children seeking emergency department care, according to a study published Online First by Archives of Pediatrics & Adolescent Medicine, a JAMA Network publication. The study is being published early because its public health importance."

"Emergency department (ED) crowding is an issue, especially when demand for ED care is high, such as during an influenza epidemic. One potential strategy is to develop automated algorithms so patients can self-triage to determine if they need to visit the ED, according to the study background."

"Rebecca Anhang Price, Ph.D., of the RAND Corporation, Arlington, Va., and colleagues conducted a pilot validation study during the 2012 influenza season to get feedback on the Strategy for Off-site Rapid Triage (SORT) for Kids, a web-based tool intended to triage patients affected by the flu."

"An adult version of the triage tool was made available to the public during the 2009 influenza pandemic, but the Centers for Disease Control and Prevention (CDC) and the American Academy of Pediatrics (AAP), which jointly devised the pediatric algorithm, would not endorse the effort to make such a tool available for pediatric patients without evidence of its safety [my emphasis].

Although this concerned a Web-based algorithm, it could have just as easily been developed as a mobile app, which leads me to ask the question: Where is the "evidence" that mobile pharma apps such as the PASI calculator is safe, in the sense that it can lead to false positive diagnoses and over prescribing? But safety aside, should these apps be certified as accurate? Shouldn't they cite references for the source of the algorithms used?

These are just a few of the questions I am asking readers to give me feedback on in my Regulation of Pharma Mobile Medical Apps Survey. Please take 5 minutes to respond to this survey. When you complete the survey, you'll be able to see a de-identified summary of results to date. Thanks.