Publications total: 16
  Feb 25th,2016

Big Data Analytics in The Healthcare Sector


The term "big data" is generating lot of interest in the last decade and it appears to be the new technological buzz word drawing attention from all industrial sectors. If we look at the scale of data that is being generated every day, thanks to massive improvements in technology and the promise that big data presents with regards to analytics, the reason behind the buzz becomes self-explanatory. To present a real scale of data that is generated everyday in the 21st century here is a very interesting fact. Reports from the industry state that the amount of data that is generated from the first day of the first human civilization till the year 2003 was produced in just 2 days in 2012 (Shah and Pathak, 2014).

A simple definition of big data would be the huge collection of data, both structured and unstructured, that is collected by age old data collection paradigms as modern digital units for the purpose of analysis, knowledge generation and new invention (Arthur, 2013). IBM states that an overwhelming majority of the data that is collected every day is highly unstructured and very difficult to analyse or make sense of (Zettaset, 2016). So it is not very difficult to understand that harnessing and analysing such volumes of data to draw knowledge and meaningful conclusions is a real challenge. Different big data analytic solutions available in the markets today have been extremely helpful in this regards in sorting and formatting the unstructured data for drawing meaningful information. One good example of big data analytics is the Hadoop platform that employs a distributed data processing architecture to manage and structure huge unorganized data volumes (Zettaset, 2016). Industrial units from different sectors have already started adopting the big data solutions for the data analysis and have even reaped rich dividends. However, when talking about the healthcare sector these are still very early days. In the recent years, the markets have witnessed the contribution of big data analytics in improving industrial output capabilities, increasing service and production efficiency and promoting new ideas and innovations.

In the healthcare sector the contribution of big data is not based in reducing care delivery costs but also on creating avenues that will improve diagnosis and treatment outcomes, prevent diseases and enrich the quality of life of people in general. Medical research today has made phenomenal progress and this had a very positive effect on the longevity of people. While this is a good thing, it has also substantially increased the load on the healthcare sector and it has become imperative for care providers to quickly adopt new paradigms of data analysis and healthcare delivery. As already mentioned earlier, the adoption of big data solutions in the healthcare sector, even though not dismal, have been slow compared to the other industrial sectors. One reason behind this is the resistance to give up on the traditional paradigms of decision making (Groves et al., 2013). Care providers would like to rely on their judgment more while making diagnostic or treatment decisions than on any big data analytics decision model. Another reason could very well be the structure of the healthcare sector that is composed of a myriad of different players. However, rapidly changing trends in the industry is creating an environment of change and innovation that will surely push the healthcare sector in adopting big data solutions in a more aggressive manner. Figure 1 below illustrates the aspects that will make big data solutions a vital necessity in the healthcare sector (Groves et al., 2013).

Factors driving the healthcare sector towards big data analytics
Fig 1: Factors driving the healthcare sector towards big data analytics.

Impact of Big Data on healthcare:

According to the industry experts, modern big data analytics will help caregivers and medical researchers to get the most out of their research data so that the best decisions could be made in a time-sensitive manner. This will reduce cost of care delivery, save lives and significantly improve the quality of service. Since the completion of the human genome project in 2003, biological research data is being generated at a phenomenal rate. Big data solutions can present the best data mining paradigms to the researchers and caregivers so that knowledge could be drawn from them in the shortest possible time. Another major aspect of big data analytics that is highly relevant to the healthcare sector is its ability to handle a variety of data, be it structured or unstructured. The healthcare sector is a conglomeration of different players and a as a result a very wide variety of data is produced every day ranging from insurance claims and prescriptions to blood profile reports and MRI scans. Big data solutions offer the means to organize such variety of data in a streamline manner to facilitate intelligent decision making from them.

According to Monappa (2015), 3 critical elements that will help the healthcare sector to reap the benefits of big data analytics are:

Data integration: Effective integration of data generated from different sources help care providers and decision makers obtain an overall picture of the situation and identify key areas that need attention or improvement. Successful data integration also creates avenues to implement new and innovative strategies in place of traditional and unproductive paradigms.

Knowledge generation: Big data solutions have the ability to retrieve knowledge and information from volumes of structured and unstructured data in the shortest possible time. The healthcare sector has been passive in adopting new data analysis paradigms compared to other industries and kept relying on the old regression-based models for too long. Techniques such as graphy analysis and machine learning helped other industries in getting much deeper insights into the market trends and improve their efficiency and productivity to a large extent while the healthcare sector lagged behind, thanks to its slow adoption rate. However, trends are changing now and the healthcare sector is opening up to big data analytics at a steady rate.

Turning knowledge into practice: Data analytics will present the knowledge and information in from the researchers in a structured manner but the real objective is to use that data in the practical world for the benefit of mankind. This is what is known as transformation of knowledge to real practice. For this reason it is important that researchers who carry big data analytics have a clear idea or vision on how they are going to implement the knowledge that they will gain into real practice. Apart from improving the care delivery practices, diagnostic paradigms and treatment outcomes, big data solutions can also help care providers understand the mind-set of the consumers and this may go a long way in developing care packages that perfectly complements consumer expectations.

There are already some good examples of big data adoption in the healthcare sector in the United States. Mayo clinic is collaborating with an IBM computer system called "Watson" to structure and analyse research data. The computer system is customized to analyse data so that it can match different clinical studies with their most appropriate cohort populations. It is believed that the approach can also help locate cohorts of patients for clinical trials that are focussing on very rare conditions. Similarly, a California-based managed care consortium called Kaiser Permanente developed a computing architecture called HealthConnect to streamline the sharing and dissemination of healthcare data across a wide range of platforms. Another California-based health cover provider called Blue Shield is collaborating with a healthcare company called NantHealth to develop a system that will allow doctors to provide diagnosis and treatment that will be more personalized or attuned to the individual patients. AstraZeneca is another prominent player in the healthcare sector that is looking to improve the management of chronic conditions by developing a system called HealthCore in collaboration with US health insurance provider WellPoint (Monnappa, 2015).


While applying big data solutions in the healthcare sector it is critically important that the highly sensitive nature of the data is taken into consideration. As the industry move towards centralized data storage paradigms there will be concerns regarding the safety and integrity of the sensitive healthcare data. Incidences of major data breach have become very common these days, thanks to the involvement of state players in the act, and the healthcare sector has been on the receiving end of such attacks on a number of occasions. One recent example is the compromise of the Anthem's system leading to the loss of around 80 million patient records. While no sensitive patient details could be stolen in the attack, industry insiders are apprehensive that the next attack could even cross that boundary. However hope is definitely there in the horizon as leaders in the big data sector continue to improve their systems, making them more robust and secure. When seen objectively, the benefits that can be derived by the health industry from big data analytics easily outweighs the anticipated risks.


Arthur, L. (2013). Forbes Welcome . [online]
Available at:

Groves, P., Kayyali, B., Knott, D. and Kuiken, S. (2013). The "big data" revolution in healthcare. Accelerating value and innovation . 1st ed. [ebook] Centre for US Health System Reform; Business Technology Office, pp.1-18.
Available at:

Marr, B. (2015). Forbes Welcome. [online]
Available at:

Monnappa, A. (2015). Big Data in Healthcare Sector? Revolutionizing the Management of Laborious Tasks | Simplilearn . [online]
Available at:

Shah, N. and Pathak, J. (2014). Why Health Care May Finally Be Ready for Big Data . [online] Harvard Business Review.
Available at:

Zettaset, (2016). What is Big Data? . [online]
Available at:

We use cookies to improve our services. If you continue browsing, you accept their use. More information