Skip to content Skip to navigation

Survey: Variety, Not Volume, the Biggest Challenge to Analyzing Big Data

July 9, 2014
by Rajiv Leventhal
| Reprints

While the huge volume of big data in healthcare continues to generate headlines, a new report finds that it’s the diverse types of data that’s a bigger challenge to data scientists and is causing them to “leave data on the table.”

According to a new survey by the Waltham, Mass.-based computational database company Paradigm4, nearly three-quarters of data scientists—71 percent—said big data had made their analytics more difficult, and data variety, not volume of data from electronic health records (EHRs), connected mobile devices, genomic sequencers and sensors, was to blame.

The survey also showed that 36 percent of data scientists say it takes too long to get insights because the data is too big to move to their analytics software. These issues cause data scientists to omit data from analyses and prevent them from maximizing the value of their work.

Additional results of the survey, which were generated from 111 data scientists in the U.S., found that 91 percent of data scientists said they're using complex analytics on their big data now or plan to within the next two years. Forty-nine percent of respondents said they're finding it more difficult to fit their data into relational database tables. And 40 percent said their biggest problem they face in gaining insights from their big data is managing new types and sources of data.

 “The increasing variety of data sources is forcing data scientists into shortcuts that leave data and money on the table,” Marilyn Matz, CEO of Paradigm4, said in a statement. “The focus on the volume of data hides the real challenge of data analytics today. Only by addressing the challenge of utilizing diverse types of data will we be able to unlock the enormous potential of analytics.”



Insurers to CBO: Consider Private Insurers’ Data in Evaluations of Telemedicine

Eleven private insurers, including Aetna, Humana and Anthem, are urging the Congressional Budget Office (CBO) to consider the experience of commercial insurers when evaluating the impact of telemedicine coverage in Medicare.

AHRQ Developing New Patient Safety Surveillance Tool

With the aim of improving patient safety monitoring, the Agency for Healthcare Research and Quality (AHRQ) within the U.S. Department of Health and Human Services (HHS) is currently developing and testing an improved patient safety surveillance system.

Gates Foundation Awards $210M to UW's Population Health Initiative

The Bill and Melinda Gates Foundation is awarding $210 million to Seattle-based University of Washington’s Population Health Initiative, with the funds going toward the construction of a new building to serve as the initiative’s hub.

AHA Offers Interoperability Standards Recommendations to ONC

The American Hospital Association (AHA) has offered feedback to the ONC on the agency’s draft Interoperability Standards Advisory (ISA) that it issued in August.

Survey: Healthcare Orgs Not Taking Mobile Security Seriously Enough

More than half (56 percent) of healthcare professionals believe their organization could be doing more to educate employees on HIPAA compliance and the rules around sharing protected health information.

Mount Sinai’s Research Arm Using Data Analytics to Address Health Inequities

The Arnhold Institute for Global Health at the Icahn School of Medicine at Mount Sinai is partnering with DigitalGlobe to create the Health Equity Atlas Initiative (ATLAS), a platform that standardizes and maps population data in order to generate insights that address health inequities.