Survey: Variety, Not Volume, the Biggest Challenge to Analyzing Big Data | Healthcare Informatics Magazine | Health IT | Information Technology Skip to content Skip to navigation

Survey: Variety, Not Volume, the Biggest Challenge to Analyzing Big Data

July 9, 2014
by Rajiv Leventhal
| Reprints

While the huge volume of big data in healthcare continues to generate headlines, a new report finds that it’s the diverse types of data that’s a bigger challenge to data scientists and is causing them to “leave data on the table.”

According to a new survey by the Waltham, Mass.-based computational database company Paradigm4, nearly three-quarters of data scientists—71 percent—said big data had made their analytics more difficult, and data variety, not volume of data from electronic health records (EHRs), connected mobile devices, genomic sequencers and sensors, was to blame.

The survey also showed that 36 percent of data scientists say it takes too long to get insights because the data is too big to move to their analytics software. These issues cause data scientists to omit data from analyses and prevent them from maximizing the value of their work.

Additional results of the survey, which were generated from 111 data scientists in the U.S., found that 91 percent of data scientists said they're using complex analytics on their big data now or plan to within the next two years. Forty-nine percent of respondents said they're finding it more difficult to fit their data into relational database tables. And 40 percent said their biggest problem they face in gaining insights from their big data is managing new types and sources of data.

 “The increasing variety of data sources is forcing data scientists into shortcuts that leave data and money on the table,” Marilyn Matz, CEO of Paradigm4, said in a statement. “The focus on the volume of data hides the real challenge of data analytics today. Only by addressing the challenge of utilizing diverse types of data will we be able to unlock the enormous potential of analytics.”



Community Data Sharing: Eight Recommendations From San Diego

A learning guide focuses on San Diego’s experience in building a community health information exchange and the realities of embarking on a broad community collaboration to achieve better data sharing.

HealthlinkNY’s Galanis to Step Down as CEO

Christina Galanis, who has served as president and CEO of HealthlinkNY for the past 13 years, will leave her position at the end of the year.

Email-Related Cyber Attacks a Top Concern for Providers

U.S. healthcare providers overwhelmingly rank email as the top source of a potential data breach, according to new research from email and data security company Mimecast and conducted by HIMSS Analytics.

Former Health IT Head in San Diego County Charged with Defrauding Provider out of $800K

The ex-health IT director at North County Health Services, a San Diego County-based healthcare service provider, has been charged with spearheading fraudulent operations that cost the organization $800,000.

Allscripts Touts 1 Billion API Shares in 2017

Officials from Chicago-based health IT vendor Allscripts have attested that the company has reached a new milestone— one billion application programming interface (API) data exchange transactions in 2017.

Dignity Health, CHI Merging to Form New Catholic Health System

Catholic Health Initiatives (CHI), based in Englewood, Colorado, and San Francisco-based Dignity Health officially announced they are merging and have signed a definitive agreement to combine ministries and create a new, nonprofit Catholic health system.