Need to find a good, clean hotel room? There’s Hotels.com. How about help in choosing a decent restaurant? Try OpenTable.com. Both websites offer plenty of user reviews of lots of outlets.

What about finding a good physician? Maybe… Certainly a large number of physicians are coming under scrutiny on physician-review websites. One problem, though, is that many of those sites rely on few patient reviews.

In fact, according to a study of 500 urologists by Loyola University Medical Center, ratings are based on scores of only 2.4 patients on average. This study found that 79.6 percent of physicians were rated by at least one of 10 free physician-review websites researchers examined. Eighty-six percent of physicians had positive ratings, with 36 percent receiving highly positive ratings. Healthgrades.com had the most physician ratings overall. Results were published online ahead of print in the Journal of Urology.

Chandy Ellimoottil, M.D., first author of the study, said consumers should be cautious about physician ratings. “Our findings suggest that consumers should take these ratings with a grain of salt,” he said. In his view, the sites have the potential to help inform consumers, but they need more reviews to make them reliable.

Nonetheless, many people do rely on websites to some extent. According to Loyola, half of Americans who go online for health information look up their providers and 40 percent use physician review websites.

Ellimoottil, senior author Ahmer Farooq, D.O., and their colleagues randomly selected 500 of the nation’s 9,940 urologists for inclusion in the study, including 471 male and 29 female urologists from 39 states. On each website, the number of reviews per physician ranged from zero to 64, with an average of 2.4. The researchers found no statistically significant difference in the median numbers of reviews when gender, region and city size were compared.

Healthgrades.com posted reviews on 54 percent of physicians, followed in order by Vials.com (45 percent), Avvo.com (39 percent), RateMDs.com (25 percent), Drscore.com (13 percent), Revolutionhealth.com (5 percent), Kudzu.com and Healthcarereviews.com (1 percent), and Zocdoc.com and Yelp.com (less than 1 percent).

The researchers also conducted a qualitative analysis of written comments posted on a single website, Vitals.com. The comments were rated extremely negative (“He needs to retire as he can barely walk”); negative; neutral, positive; or extremely positive (“One of the best checkups in a long time!!”). Three percent of the comments were extremely negative, 22 percent negative, 22 percent neutral, 39 percent positive, and 14 percent extremely positive.

My own view of physician ratings is mixed, but overall I’m skeptical. Surely physician review websites are part of the social media trend, and are here to stay. On the other hand, these are all subjective opinions, and who’s to say that a bad review is from a consumer who simply has an ax to grind? That’s a potential problem—especially given the overall lack of reviews per physician—which can be a disservice to both the provider and to consumers who rely on these sites. There’s really no substitute for face-to-face consultations, and hopefully the data-driven changes now transforming healthcare will give consumers better tools to use in evaluating their health providers.

Incidentally, neither Dr. Ellimoottil nor Dr. Farooq, respectively a resident and an instructor in the Department of Urology of Loyola University Chicago Stritch School of Medicine, have yet gotten reviewed on Healthgrades.com.