The Rising Rise of Big Data
Big data or large data sets, when pulled together in the right context can provide hidden insights to trends and patterns that can help dictate or guide certain decisions. But how did it happen?
Emerging technologies these past few years have really allowed large complex sets of data be used in everyday applications, especially in healthcare, businesses, academics as well as in the government. However, that is such an abstract concept though, it’s important to understand on the consumer/receiving side, just how it can impact us on a day2day basis. Big Data spans four dimensions, as mentioned in this IBM post.
- Volume, it is important that the infrastructure be built that can securely store and manage all of the incoming data, especially in large quantities at any time.
- Velocity, now with all of the data — it’s not like we can just let it travel bit by bit or even in a stream because it could take a CRAP ton of time to even transmit and exchange information between vendors, departments, you call it, it’ll take forever. Since most of the data is vital to certain business decisions, now not only does there have to be a physical infrastructure, but a logical infrastructure as well that can process the data in real-time.
- Variety, utilizing both forms of infrastructure. The data arriving will most likely be registered from a variety of sources as a result, the infrastructure must be able to handle the different forms of data as well as from a variety of sources. It must be analyzed correctly and placed in the correct data structure.
- Veracity, similar to the triad of IT security, veracity is synonymous to “integrity” in that the data has to be secure and there has to be a method in transmission that can ensure the “trust” between sources and data architecture.
The image above is one of many examples in how big data can be used and how it can be utilized. Data scientists, will often gather, aggregate, and analyze the data and the most beautiful part? They can create beautiful reports, infographics, or overlain images that depict their results. Primarily in health care though, there have been many rising projects such as one for Alzheimer’s treatment that will gather the genomic data from hundreds of participants that will be available through a cloud-based system that can allow researchers to aggregate and analyze the data to try and find more insightful clues on Alzheimer’s.
In the above picture, Dell is working with health organizations and hospitals to provide services that can utilize big data in bioinformatics and genomics for personalized medicine. As a result, not only will it help providers determine the proper diagnosis and treatment through analysis of lab results, but it can be a method to best treat a patient’s illness. The potential for big data is tremendous, especially with how it can improve various areas in healthcare. It can analyze large data sets to gather key insights to aid patient care and even with business decisions as well (more later on).
Not only can big data be used in research, but it can be used in the entire healthcare industry. Ever since HITECH and the Affordable Care Act was passed into the general population, the rise in medical technologies and EHRs has given rise to a large volume of data available, and most of it is unstructured but clinically relevant. (Refer to my rants about EHR and the poor design quality in UX/UI that led to the inconveniences for provider use). The data can reside in hospital wars, lab and imaging systems, notes, and even the CRM systems, however there is no central data structure with the necessary infrastructure that can gather all of this data and process it. Not in healthcare yet, but data warehouses have been a rising trend in healthcare despite its start back in the 80’s. We’ve discussed how healthcare is lagging behind, and it certainly is in this case. As organizations are trying to develop and implement data warehouses to aggregate and analyze the variety of information, it is a difficult struggle as some organizations are unwilling to try the risk for a slow ROI.
As stated by IBM Analytics on Healthcare Big Data, some of the goals that healthcare big data should be used is to help collaborate to improve care and outcomes while increasing access to healthcare and to help build sustainable medical systems. The first two are primarily linked to public health, which is, unfortunately, the overlooked child of health these days. Despite the many studies that indicate that horizontally-focused health interventions with public health in mind, provide greater health outcomes, many are still focused on the acute medical interventions such as expensive surgeries as opposed to improving water quality or increasing the number of community care clinics. Some of the major issues in healthcare are in fact the access and quality of care, especially in more rural areas (think of the Appalachians in the USA), that often lack primary care resources or simply cannot afford it.
With urbanization and large metros, why would providers travel out to the middle of nowhere to practice medicine and provide health in a place where there is likely going to be a low amount of return? As someone looking to go into the profession, even I realize that money is one of the leading factors in going into the profession. Sure, it’s no investment banking or day trading at wall street, but what it lacks in burst makes up for in job security. Many people graduate without hundreds of thousands in debt, so to have a decision to make a modest salary in the middle of nowhere, vs much more in a large city, it’s typically an easy decision. With the ever increasing health inequalities, many organizations are starting to provide loan repayment programs or even the opportunity to cancel their loans if they serve in underprivileged areas or in areas that lack medical professionals (coughcoughVAcough). By utilizing big data in healthcare, organizations can start too look at reducing healthcare expenditures while working to improve patient outcomes as well, by not only making medical care more efficient but also by improving the quality and quantity of public health solutions.