What’s Your Strategic Intent for Big Data?

Like many new information technologies, Big Data can bring about dramatic
cost reductions, substantial improvements in the time required to perform a
computing task, or new product and service offerings. Which of these benefits
are you seeking? The technologies and concepts behind Big Data allow
organizations to achieve a variety of objectives, but you are unlikely to
realize all the possible benefits. Deciding what your organization wants from
Big Data is a critical decision that you need to address. The answer has
implications for not only the outcome from Big Data, but also the process—who
leads the initiative, where it fits within your organization, and how you manage
the project.

If you are looking for cost reduction, you’re probably conscious of the fact
that MIPS and terabyte storage for structured data are now most cheaply
delivered through Big Data technologies like Hadoop clusters. One company’s cost
comparison, for example, estimated that the cost of storing one terabyte for a
year was $37,000 for a traditional relational database, $5,000 for a data
appliance, and only $2,000 for a Hadoop cluster. Of course, these figures are
not directly comparable, in that the more traditional technologies may be
somewhat more reliable and easily managed.

If you’re focusing primarily on cost reduction, then the decision to adopt
Big Data tools is relatively straightforward. It should be made primarily by the
IT organization on largely technical and economic criteria. You may want to
involve some of your users and sponsors in debating the data management
advantages and disadvantages of this kind of storage, but that’s about it.

The second key benefit of Big Data tools is time reduction. Macy’s Inc.’s
merchandise pricing optimization application provides a classic example of
reducing the cycle time for complex and large-scale analytical calculations from
hours or even days to minutes or seconds. The department store chain has been
able to reduce the time to optimize pricing of its 73 million items for sale
from over 27 hours to just over 1one hour. Described as “high
performance analytics,” or HPA, by the software vendor SAS, HPA makes it
possible for Macy’s to re-price items much more frequently to adapt to changing
conditions in the retail marketplace. This HPA application doesn’t employ a
Hadoop cluster, but it does take advantage of parallel computing and in-memory
software architectures. Macy’s also says it achieved 70% hardware cost
reductions.

If your company is primarily interested in time reduction, you need to work
much more closely with the owner of the relevant business process. A key
question is what you are going to do with all the time saved in the process.
Good answers include:

• We’re going to iterate and tune the model much more frequently to get a
  better solution;
• We’re going to use many more variables and more data to compute a real-time
  offer for our customers;
• We’re going to be able to respond much more rapidly to contingencies in our
  environment.

Bad answers (at least in strict business terms) include playing more golf,
drinking more coffee, or finally having enough time for that three-martini
lunch.

To my mind, the best thing an organization can do with Big Data is to develop
new products and services. The best organization at this may be LinkedIn
Corp. , which has
used Big Data and data scientists to develop a broad array of product offerings
and features, including People You May Know, Groups You May Like, Jobs You May
Be Interested In, Who’s Viewed My Profile, and several others General
Electric Co. is
primarily focused on Big Data for improving services—among other things, to
optimize the service contracts and maintenance intervals for industrial products
Google
Inc. , of course,
uses Big Data to refine its core search and ad-serving algorithms Zynga
Inc. uses Big Data
to target games and game-related products to customers Netflix
Inc. created the
well-known Netflix prize for the data science team that could optimize the
company’s movie recommendations for customers. The testing firm Kaplan uses Big
Data to begin advising customers on effective learning and test-preparation
strategies.  These companies’ Big Data efforts are directly focused on products,
services, and customers.

This has important implications, of course, for the organizational locus of
Big Data and the processes and pace of new product development. Obviously you
need to be working closely with the product development team, and perhaps
marketing as well. These projects should probably be sponsored by a business
leader, rather than a technician or data scientist. You may not save a lot of
money or time, but you may well add some big numbers to your company’s top
line.

I hope you now agree that deciding what you want to accomplish with Big Data
is one of the first and most important things you should do with this resource.
If you’re not discussing that yet within your organization, get busy!

Category: Uncategorized