Lisa Morgan's Official Site

Click-worthy Content Development

Tag: Business strategy

Big Data: The Interdisciplinary Vortex

vortexAs seen in InformationWeek

Getting the most from data requires information sharing across departmental boundaries. Even though information silos remain common, CIOs and business leaders in many organizations are cooperating to enable cross-functional data sharing to improve business process efficiencies, lower costs, reduce risks, and identify new opportunities.

Interdepartmental data sharing can take a company only so far, however, as evidenced by the number of companies using (or planning to use) external data. To get to the next level, some organizations are embracing interdisciplinary approaches to big data.

Why Interdisciplinary Problem-Solving May Be Overlooked

Breaking down departmental barriers isn’t easy. There are the technical challenges of accessing, cleansing, blending, and securing data, as well as very real cultural habits that are difficult to change.

 Today’s businesses are placing greater emphasis on data scientists, business analysts, and data-savvy staff members. Some of them also employ or retain mathematicians and statisticians, although they may not have considered tapping other forms of expertise that could help enable different and perhaps more accurate forms of data analysis and new innovations.

“Thinking of big data as one new research area is a misunderstanding of the entire impact that big data will have,” said Dr. Wolfgang Kliemann, associate VP for research at Iowa State University. “You can’t help but be interdisciplinary because big data is affecting all kinds of things including agriculture, engineering, and business.”

[Tear down the silos. See How Corporate Culture Impedes Data Innovation.]

Although interdisciplinary collaboration is mature in many scientific and academic circles, applying non-traditional talent to big data analysis is a stretch for most businesses.

But there are exceptions. For example, Ranker, a platform for lists and crowdsourced rankings, employs a chief data scientist who is also a moral psychologist.

“I think psychology is particularly useful because the interesting data today is generated by people’s opinions and behaviors,” said Ravi Iyer, chief data scientist at Ranker. “When you’re trying to look at the error that’s associated with any method of data connection, it usually has something to do with a cognitive bias.”

Ranker has been working with a UC Irvine professor in the cognitive sciences department who studies the wisdom of crowds.

“We measure things in different ways and understand the psychological biases each method of data creates. Diversity of opinion is the secret to both our algorithms and the philosophy behind the algorithms,” said Iyer. “Most of the problems you’re trying to solve involve people. You can’t just think of it as data, you have to understand the problem area you’re trying to solve.”

Why Interdisciplinary Problem-Solving Will Become More Common

Despite the availability of new research methods, online communities, and social media streams, products still fail and big-name companies continue to make high-profile mistakes. They have more data available than ever before, but there may be a problem with the data, the analysis, or both. Alternatively, the outcome may fall short of what is possible.

“A large retail chain is interested in figuring out how to optimize supply management, so they collect the data from sales, run it through a big program, and say, ‘this is what we need.’ This approach leads to improvements for many companies,” said Kliemann. “The question is, if you use this specific program and approach, what is your risk of not having the things you need at a given moment? The way we do business analytics these days, that question cannot be answered.”

One mistake is failing to understand the error structure of the data. With such information, it’s possible to identify missing pieces of data, what the possible courses of action are, and the risk associated with a particular strategy.

“You need new ideas under research, ideas of data models, [to] understand data errors and how they propagate through models,” said Kliemann. “If you don’t understand the error structure of your data, you make predictions that are totally worthless.”

Already, organizations are adapting their approaches to accommodate the growing volume, velocity, and variety of data. In the energy sector, cheap sensors, cheap data storage, and fast networks are enabling new data models that would have been impossible just a few years ago.

“Now we can ask ourselves questions such as if we have variability in wind, solar, and other alternative energies, how does it affect the stability of a power system? [We can also ask] how we can best continue building alternative energies that make the system better instead of jeopardizing it,” said Kleinman.

Many universities are developing interdisciplinary programs focused on big data to spur innovation and educate students entering the workforce about how big data can affect their chosen field. As the students enter the workforce, they will influence the direction and culture of the companies for which they work. Meanwhile, progressive companies are teaming up with universities with the goal of applying interdisciplinary approaches to real-world big data challenges.

In addition, the National Science Foundation (NSF) is trying to accelerate innovation through Big Data Regional Innovation Hubs. The initiative encourages federal agencies, private industry, academia, state and local governments, nonprofits, and foundations to develop and participate in big data research and innovation projects across the country. Iowa State University is one of about a dozen universities in the Midwestern region working on a proposal.

In short, interdisciplinary big data problem-solving will likely become more common in industry as organizations struggle to understand the expanding universe of data. Although interdisciplinary problem-solving is alive and well in academia and in many scientific research circles, most businesses are still trying to master interdepartmental collaboration when it comes to big data.

Big Data: The Interdisciplinary Vortex

As seen in  InformationWeek.

vortexGetting the most from data requires information sharing across departmental boundaries. Even though information silos remain common, CIOs and business leaders in many organizations are cooperating to enable cross-functional data sharing to improve business process efficiencies, lower costs, reduce risks, and identify new opportunities.

Interdepartmental data sharing can take a company only so far, however, as evidenced by the number of companies using (or planning to use) external data. To get to the next level, some organizations are embracing interdisciplinary approaches to big data.

Why Interdisciplinary Problem-Solving May Be Overlooked

Breaking down departmental barriers isn’t easy. There are the technical challenges of accessing, cleansing, blending, and securing data, as well as very real cultural habits that are difficult to change.

Today’s businesses are placing greater emphasis on data scientists, business analysts, and data-savvy staff members. Some of them also employ or retain mathematicians and statisticians, although they may not have considered tapping other forms of expertise that could help enable different and perhaps more accurate forms of data analysis and new innovations.

“Thinking of big data as one new research area is a misunderstanding of the entire impact that big data will have,” said Dr. Wolfgang Kliemann, associate VP for research at Iowa State University. “You can’t help but be interdisciplinary because big data is affecting all kinds of things including agriculture, engineering, and business.”

Although interdisciplinary collaboration is mature in many scientific and academic circles, applying non-traditional talent to big data analysis is a stretch for most businesses.

But there are exceptions. For example, Ranker, a platform for lists and crowdsourced rankings, employs a chief data scientist who is also a moral psychologist.

“I think psychology is particularly useful because the interesting data today is generated by people’s opinions and behaviors,” said Ravi Iyer, chief data scientist at Ranker. “When you’re trying to look at the error that’s associated with any method of data connection, it usually has something to do with a cognitive bias.”

Ranker has been working with a UC Irvine professor in the cognitive sciences department who studies the wisdom of crowds.

“We measure things in different ways and understand the psychological biases each method of data creates. Diversity of opinion is the secret to both our algorithms and the philosophy behind the algorithms,” said Iyer. “Most of the problems you’re trying to solve involve people. You can’t just think of it as data, you have to understand the problem area you’re trying to solve.”

Why Interdisciplinary Problem-Solving Will Become More Common

Despite the availability of new research methods, online communities, and social media streams, products still fail and big-name companies continue to make high-profile mistakes. They have more data available than ever before, but there may be a problem with the data, the analysis, or both. Alternatively, the outcome may fall short of what is possible.

“A large retail chain is interested in figuring out how to optimize supply management, so they collect the data from sales, run it through a big program, and say, ‘this is what we need.’ This approach leads to improvements for many companies,” said Kliemann. “The question is, if you use this specific program and approach, what is your risk of not having the things you need at a given moment? The way we do business analytics these days, that question cannot be answered.”

One mistake is failing to understand the error structure of the data. With such information, it’s possible to identify missing pieces of data, what the possible courses of action are, and the risk associated with a particular strategy.

“You need new ideas under research, ideas of data models, [to] understand data errors and how they propagate through models,” said Kliemann. “If you don’t understand the error structure of your data, you make predictions that are totally worthless.”

Already, organizations are adapting their approaches to accommodate the growing volume, velocity, and variety of data. In the energy sector, cheap sensors, cheap data storage, and fast networks are enabling new data models that would have been impossible just a few years ago.

“Now we can ask ourselves questions such as if we have variability in wind, solar, and other alternative energies, how does it affect the stability of a power system? [We can also ask] how we can best continue building alternative energies that make the system better instead of jeopardizing it,” said Kleinman.

Many universities are developing interdisciplinary programs focused on big data to spur innovation and educate students entering the workforce about how big data can affect their chosen field. As the students enter the workforce, they will influence the direction and culture of the companies for which they work. Meanwhile, progressive companies are teaming up with universities with the goal of applying interdisciplinary approaches to real-world big data challenges.

In addition, the National Science Foundation (NSF) is trying to accelerate innovation through Big Data Regional Innovation Hubs. The initiative encourages federal agencies, private industry, academia, state and local governments, nonprofits, and foundations to develop and participate in big data research and innovation projects across the country. Iowa State University is one of about a dozen universities in the Midwestern region working on a proposal.

In short, interdisciplinary big data problem-solving will likely become more common in industry as organizations struggle to understand the expanding universe of data. Although interdisciplinary problem-solving is alive and well in academia and in many scientific research circles, most businesses are still trying to master interdepartmental collaboration when it comes to big data.

How Corporate Culture Impedes Data Innovation

As seen in InformationWeek

Floppy disk

Corporate culture moves slower than tech

Competing in today’s data-intensive business environment requires unprecedented organizational agility and the ability to drive value from data. Although businesses have allocated significant resources to collecting and storing data, their abilities to analyze it, act upon it, and use it to unlock new opportunities are often stifled by cultural impediments.

While the need to update technology may be obvious, it may be less obvious that corporate cultures must also adapt to changing times. The necessary adjustments to business values, business practices, and leadership strategies can be uncomfortable and difficult to manage, especially when they conflict with the way the company operated in the past.

If your organization isn’t realizing the kind of value from its big data and analytics investments that it should be, the problem may have little to do with technology. Even with the most effective technologies in place, it’s possible to limit the value they provide by clinging to old habits.

Here are five ways that cultural issues can negatively affect data innovation:

1. The Vision And Culture Are At Odds

Data-driven aspirations and “business as usual” may well be at odds. What served a company well up to a certain point may not serve the company well going forward.

“You need to serve the customer as quickly as possible, and that may conflict with the way you measured labor efficiencies or productivity in the past,” explained Ken Gilbert, director of business analytics at the University of Tennessee Office of Research and Economic Development, in an interview with InformationWeek.

[ What matters more: Technology or people? Read Technology Is A Human Endeavor. ]

Companies able to realize the most benefit from their data are aligning their visions, corporate mindsets, performance measurement, and incentives to effect widespread cultural change. They are also more transparent than similar organizations, meaning that a wide range of personnel has visibility into the same data, and data is commonly shared among departments, or even across the entire enterprise.

“Transparency doesn’t come naturally,” Gilbert said. “Companies don’t tend to share information as much as they should.”

Encouraging exploration is also key. Companies that give data access to more executives, managers, and employees than they did in the past have to also remove limits that may be driven by old habits. For example, some businesses discourage employees from exploring the data and sharing their original observations.

2. Managers Need Analytics Training

Companies that are training their employees in ways to use analytical tools may not be reaching managers and executives who choose not to participate because they are busy or consider themselves exempt. In the most highly competitive companies, executives, managers, and employees are expected to be — or become — data savvy.

Getting the most from BI and big data analytics means understanding what the technology can do, and how it can be used to best achieve the desired business outcomes. There are many executive programs that teach business leaders how to compete with business analytics and big data, including the Harvard Business School Executive Education program.

3. Expectations Are Inconsistent

This problem is not always obvious. While it’s clear the value of BI and big data analytics is compromised when the systems are underutilized, less obvious are inconsistent expectations about how people within the organization should use data.

“Some businesses say they’re data-driven, but they’re not actually acting on that. People respond to what they see rather than what they hear,” said Gilbert. “The big picture should be made clear to everybody — including how you intend to grow the business and how analytics fits into the overall strategy.”

4. Fiefdoms Restrict Data Sharing

BI and analytics have moved out from the C-suite, marketing, and manufacturing to encompass more departments, but not all organizations are taking advantage of the intelligence that can be derived from cross-functional data sharing. An Economist Intelligence Unit survey of 530 executives around the world revealed that information-sharing issues represented the biggest obstacle to becoming a data-driven organization.

“Some organizations supply data on a need-to-know basis. There’s a belief that somebody in another area doesn’t need to know how my area is performing when they really do,” Gilbert said. “If you want to use data as the engine of business growth, you have to integrate data from internal and external sources across lines, across corporate boundaries.”

5. Little-Picture Implementations

Data is commonly used to improve the efficiency or control the costs of a particular business function. However, individual departmental goals may not align with the strategic goal of the organization, which is typically to increase revenue, Gilbert said.

“If the company can understand what the customer values, and build operational systems to better deliver, that is the company that’s going to win. If the company is being managed in pieces, you may save a dime in one department that costs the company a dollar in revenue.”

Tech Buying: 6 Reasons Why IT Still Matters

ErrorOriginally published in InformationWeek, and available as a slideshow here.

Making major tech purchases, especially big data analytics and business intelligence tools, without consulting IT may cause major problems. Here’s why.

Although shadow IT is not new, the percentage of business tech purchases made outside IT is significant and growing. When Bain & Company conducted in-depth interviews with 67 marketing, customer service, and supply chain professionals in February 2014, it found that nearly one-third of technology purchasing power had moved to executives outside of IT. Similarly, member-based advisory firm CEB has estimated that non-IT departments control 30% of enterprise IT spend. By 2020, Gartner estimates, 90% of tech spending will occur outside IT.

There are many justifications for leaving IT in the dark about departmental tech purchases. For one thing, departmental technology budgets seem to point to departmental decision making. Meanwhile, cloud-based solutions, including analytics services, have become more popular with business users because they are easy to set up. In addition, their relatively low subscription rates or pay-per-use models may be more attractive from a budgetary standpoint than their traditional on-premises counterparts, which require significant upfront investments and IT consideration. Since the cost and onboarding barriers to cloud service adoption are generally lower than for on-premises products, IT’s involvement may seem to be unnecessary.

Besides, IT is busy. Enterprise environments are increasingly complex, and IT budgets are not growing proportionally, so the IT department is resource-constrained. Rather than waiting for IT — or complicating decision-making by getting others involved — non-IT tech buyers anxious to deploy a solution may be tempted to act first and answer questions later.

However, making tech purchase without IT’s involvement may result in unforeseen problems. On the following pages, we reveal six risks associated with making business tech purchases without involving IT.

1. Tech Purchases Affect Everybody
Tech purchases made without IT’s involvement may affect IT and the IT ecosystem in ways that someone outside IT couldn’t anticipate. You might be introducing technical risk factors or tapping IT resources IT will have to troubleshoot after the fact. To minimize the potential of unforeseen risks, IT can perform an in-depth assessment of your department’s requirements, the technology options, their trade-offs, and the potential ripple effect that your tech purchase might have across the organization. This kind of risk/benefit analysis is important. Even if it seems like a barrier for your department to get what it wants, it’s better for the entire organization in the long run.
Also, you may need help connecting to data sources, integrating data sources, and ensuring the quality of data, all of which require specific expertise. IT can help you understand the scope of an implementation in greater detail than you might readily see.

2. Sensitive Information May Be Compromised
Information security policies need to be defined, monitored, and enforced. While it’s common for businesses to have security policies in place, education about those policies, and the enforcement of those policies, sometimes fall short. Without appropriate precautions, security leaks can happen innocently, or you could be opening the door to intentional bad actors.
Cloud-based services can expose organizations to risks that users haven’t considered, especially when the service’s terms of use are not understood. Asurvey of 4,140 business and IT managers, conducted in July 2012 by The Ponemon Institute and sponsored by Thales e-Security, revealed that 63% of respondents did not know what cloud providers are doing to protect their sensitive or confidential data.

3. Faulty Data = Erroneous Conclusions
There is no shortage of data to analyze. However, inadequate data quality and access to only a subset of information can negatively impact the accuracy of analytics and, ultimately, decision making.
In an interview with InformationWeek, Jim Sterne, founder of the eMetrics Summit and the Digital Analytics Association, warned that the relative reliability of sources needs to be considered since CRM system data, onsite user behavior data, and social media sentiment analysis data are not equally trustworthy.
“If I’m looking at a dashboard as a senior executive and I know where the data came from and how it was cleansed and blended, I’m looking at the numbers as if they have equal weight,” he said. “It’s like opening up a spice cabinet and assuming each spice is as spicy as any other. I will make bad decisions because I don’t know how the information was derived.”

4. Not Getting What You Bought
Similar products often sound alike, but their actual capabilities can vary greatly. IT can help identify important differences.
While it may be tempting to purchase a product based on its exhaustive feature set or its latest enhancements, feature-based buying often proves to be a mistake because it omits or minimizes strategic thinking. To reduce the risk of buyer’s remorse, consulting with IT can help you assess your current and future requirements and help you choose a solution that aligns with your needs.

5. Scope Creep
Business users typically want immediate benefits from big data, analytics packages, and BI systems. But, if the project has a lot of technological complexity — and particularly if it involves tech dependencies that are outside the control of your department — it’s often best to implement in phases. Approaching large initiatives as one big project may prove to be more complicated, time-consuming, and costly than anticipated.
IT can help you break a large, difficult-to-manage project into several smaller projects, each of which has its own timeline and goals. That way, you can set realistic end-user and C-suite expectations and effectively control risks. Phasing large projects can also provide you with the flexibility you need to adjust your implementation as business requires.

6. Missing Out On Prior Experience
IT professionals and outsourced IT resources often have prior experience with BI and analytics implementations that are specific or relevant to your department. Some of them have implemented solutions in other companies, departments, or industries and have gained valuable insight from those experiences. When armed with such knowledge, they can help you understand potential opportunities, challenges, and pitfalls you may not have considered which can affect planning, implementation, and the choice of solutions.