Big Data is a Big Deal

Big Data is a Big Deal by Dan Gatti, Big Data I/O Forum

The White House Office of Science and Technology announced the “Big Data Research and Development Initiative.”   To launch the initiative, six Federal departments and agencies will announce more than $200 million in new commitments that, together, promise to greatly improve the tools and techniques needed to access, organize, and glean discoveries from huge volumes of digital data. The focus of this inter-agency collaboration is, as the White House stated, on improving “the tools and techniques needed to access, organize and glean discoveries from huge volumes of data.” OSTP officials narrowed the policy action down to three bullets that define the inter-agency approach. These include:

• Advance state of the art core technologies needed to collect, store, preserve, manage, analyze and share huge quantities of data.
• Harness these technologies to accelerate the pace of discovery in science and engineering, strengthen our national security and transform teaching and learning.
• Expand the workforce needed to develop and use big data technologies.

While broad policy action items like this can often be tough to follow in terms of how the funding is filtered, there are some notable projects that are set to roll forward immediately from the agencies and departments slated as recipients for the funding.

Homeland Security, the Department of Defense (DoD), the Department of Energy (DOE), the National Institutes of Health, the Food and Drug Administration, and the U.S. Geological Survey are but a few divisions set to see their big data R&D capacity get a boost.

There are some solid projects on the table from the six agencies tasked to follow through with the initiative.

For instance, the DoD is placing what it calls a “big bet on big data” to the tune of $250 million to the table to find ways to make use of sensor data, sentiment, and decision support systems to make “truly autonomous systems that can maneuver and make decisions of their own.”

They also point to other defense projects to help soldiers on the field with sentiment, textual and predictive analytics operations that lead to better situational awareness.

In addition to the “big bet” within the main arm of the department, other DoD-fed agencies, including DARPA will receive resources to push ahead with projects like XDATA, which will receive around $25 million over the next four years to develop new techniques of analysis on diverse data across distributed stores.

Other government divisions, including the Department of Homeland Security, will explore the use of real-time data to better respond to emergency situations. As the department described this week, it will be working with researchers at Rutgers (home to its own big data investment news this week), Purdue and the Center of Excellence on Visualization and Data Analytics (CVADA) to advance techniques that will use real-time data to create instant decision-making capabilities.

The DOE will be focusing on computer science projects that focus on storage, including work on the High Performance Storage System (HPSS) , the Adaptable IO System and the Kepler Scientific Workflow System, among others.

The National Science Foundation is another key recipient of resources under the big data program. The NSF released a solicitation, “Core Techniques and Technologies for Advancing Big Data Science & Engineering,” or “Big Data,” jointly with NIH.  According to the NSF, the program will seek to tap large data sets in order to accelerate progress in science and engineering research. Specifically, it will fund research to develop and evaluate new algorithms, statistical methods, technologies, and tools for improved data collection and management, data analytics and e-science collaboration environments.

This includes a $10 million award under the Expeditions in Computing program to researchers at the University of California, Berkeley. The team will integrate algorithms, machines, and people to turn data into knowledge and insight. The objective is to develop new scalable machine-learning algorithms and data management tools that can handle large-scale and heterogeneous datasets, novel datacenter-friendly programming models, and an improved computational infrastructure.

According to the NSF’s director, Subra Suresh, “Data are motivating a profound transformation in the culture and conduct of scientific research in every field of science and engineering.” He called on American scientists to “rise to the challenges and seize the opportunities afforded by this new, data-driven revolution,” saying that “The work we do today will lay the groundwork for new enterprises and fortify the foundations for U.S. competitiveness for decades to come.”

Edward Lazowska, a computer scientist from University of Washington, summarized the sentiments of many in the academic community who spoke in the wake of the announcement when he pointed to the recognition of big data technologies as a critical first step in aiding missions related to everything from national defense to healthcare.

Category: Uncategorized