Lisa Morgan's Official Site

Click-worthy Content Development

Category: Analytics strategy

Why Surveys Should Be Structured Differently

keyboard-417093_1280If you’re anything like me, you’re often asked to participate in surveys.  Some of them are short and simple.  Others are very long, very complicated, or both.

You may also design and implement surveys from time to time like I do.   If you want some insight into the effectiveness of your survey designs and their outcomes, pay attention to the responses you get.

Notice the Drop-off Points

Complicated surveys that take 15 or 20 minutes to complete tend to reflect drop off points at which the respondents decided that the time investment required wasn’t worth whatever incentive was offered.  After all, not everyone actually cares about survey results or a  1-in-1,000 chance of winning the latest iPad, for example.  If there’s no incentive whatsoever, long and complicated surveys may  be even less successful, even if you’re pinging your own  database.

A magazine publisher recently ran such a survey, and boy, was it hairy.  It started out like similar surveys, asking questions about the respondent’s title, affiliation, company revenue and size.  It also asked about purchasing habits – who approves, who specifies, who recommends, etc. for different kinds of technologies.  Then, what the respondent’s content preferences are for learning about tech (several drop-down menus), using tech (several drop-down menus), purchasing tech (several drop-down menus), and I can’t remember what else.  At that point, one was about 6% done with the survey.  So much for “10 – 15 minutes.”  It took about 10 or 15 minutes just to wade through the first single-digit percent of it.  One would really want a slim chance of winning the incentive to complete that survey.

In short, the quest to learn everything about everything in one very long and complex survey may end in more knowledge about who took the survey than how how people feel about important issues.

On the flip side are very simple surveys that take a minute or two to answer.  Those types of surveys tend to focus on whether a customer is satisfied or dissatisfied with customer service, rather than delving into the details of opinions about several complicated matters.

Survey design is really important.  Complex fishing expeditions can and often do reflect a lack of focus on the survey designer’s part.

Complex Surveys May Skew Results

Overly complicated surveys may also yield spurious results.  For example, let’s say 500 people agree to take a survey we just launched that happens to be very long and very complex.  Not all of the respondents will get past the who-are-you questions because those too are complicated.  Then, as the survey goes on, more people drop, then more.

The result is that  X% of of the survey responses at the end of the survey are not the same as X% earlier in the survey.  What I mean by that is 500 people started, maybe 400 get past the qualification portion, and the numbers continue to fall as yet more complicated questions arise but  the “progress bar” shows little forward movement.  By the end of the survey, far less than 500 have participated, maybe 200  or 100.

Of course, no one outside the survey team knows this, including the people in the company who are presented with the survey results.  They only know that 500 people participated in the survey and X% said this or that.

However, had all 500 people answered all the questions, the results of some of the questions would likely look slightly or considerably different, which may be very important.

Let’s say 150 people completed our  survey and the last question asked whether they planned to purchase an iPhone 7 within the next three months.  40% of them or 60 respondents said yes.  If all 500 survey respondents answered that same question, I can almost guarantee you the answer would not be 40% .  It might be close to 40% or it might not be even close to 40%.

So, if you genuinely care about divining some sort of “truth” from surveys, you need to be mindful about how to define and structure the survey and that the data you see may not be telling you the entire story, or even an accurate story.

The point about accuracy is very important and one that people without some kind of statistical background likely haven’t even considered because they’re viewing all aggregate numbers as having equal weight and equal accuracy.

I, for one, think that survey “best practices” are going to evolve in the coming years with the help of data science.  While the average business person knows little about data science now, in the future it will likely seem cavalier not to consider the quality of the data you’re getting and what you can do to improve the quality of that data.  Your credibility and perhaps your job may depend on it.

In the meantime, try not to shift the burden of thinking entirely to your survey audience because it won’t do either of you much good.  Think about what you want to achieve, structure your questions in a way that gives you insight into your audience and their motivations (avoid leading questions!), and be mindful that not all aggregate answers are equally accurate or representative, even within the same survey.

How Corporate Culture Impedes Data Innovation

As seen in InformationWeek

Floppy disk

Corporate culture moves slower than tech

Competing in today’s data-intensive business environment requires unprecedented organizational agility and the ability to drive value from data. Although businesses have allocated significant resources to collecting and storing data, their abilities to analyze it, act upon it, and use it to unlock new opportunities are often stifled by cultural impediments.

While the need to update technology may be obvious, it may be less obvious that corporate cultures must also adapt to changing times. The necessary adjustments to business values, business practices, and leadership strategies can be uncomfortable and difficult to manage, especially when they conflict with the way the company operated in the past.

If your organization isn’t realizing the kind of value from its big data and analytics investments that it should be, the problem may have little to do with technology. Even with the most effective technologies in place, it’s possible to limit the value they provide by clinging to old habits.

Here are five ways that cultural issues can negatively affect data innovation:

1. The Vision And Culture Are At Odds

Data-driven aspirations and “business as usual” may well be at odds. What served a company well up to a certain point may not serve the company well going forward.

“You need to serve the customer as quickly as possible, and that may conflict with the way you measured labor efficiencies or productivity in the past,” explained Ken Gilbert, director of business analytics at the University of Tennessee Office of Research and Economic Development, in an interview with InformationWeek.

[ What matters more: Technology or people? Read Technology Is A Human Endeavor. ]

Companies able to realize the most benefit from their data are aligning their visions, corporate mindsets, performance measurement, and incentives to effect widespread cultural change. They are also more transparent than similar organizations, meaning that a wide range of personnel has visibility into the same data, and data is commonly shared among departments, or even across the entire enterprise.

“Transparency doesn’t come naturally,” Gilbert said. “Companies don’t tend to share information as much as they should.”

Encouraging exploration is also key. Companies that give data access to more executives, managers, and employees than they did in the past have to also remove limits that may be driven by old habits. For example, some businesses discourage employees from exploring the data and sharing their original observations.

2. Managers Need Analytics Training

Companies that are training their employees in ways to use analytical tools may not be reaching managers and executives who choose not to participate because they are busy or consider themselves exempt. In the most highly competitive companies, executives, managers, and employees are expected to be — or become — data savvy.

Getting the most from BI and big data analytics means understanding what the technology can do, and how it can be used to best achieve the desired business outcomes. There are many executive programs that teach business leaders how to compete with business analytics and big data, including the Harvard Business School Executive Education program.

3. Expectations Are Inconsistent

This problem is not always obvious. While it’s clear the value of BI and big data analytics is compromised when the systems are underutilized, less obvious are inconsistent expectations about how people within the organization should use data.

“Some businesses say they’re data-driven, but they’re not actually acting on that. People respond to what they see rather than what they hear,” said Gilbert. “The big picture should be made clear to everybody — including how you intend to grow the business and how analytics fits into the overall strategy.”

4. Fiefdoms Restrict Data Sharing

BI and analytics have moved out from the C-suite, marketing, and manufacturing to encompass more departments, but not all organizations are taking advantage of the intelligence that can be derived from cross-functional data sharing. An Economist Intelligence Unit survey of 530 executives around the world revealed that information-sharing issues represented the biggest obstacle to becoming a data-driven organization.

“Some organizations supply data on a need-to-know basis. There’s a belief that somebody in another area doesn’t need to know how my area is performing when they really do,” Gilbert said. “If you want to use data as the engine of business growth, you have to integrate data from internal and external sources across lines, across corporate boundaries.”

5. Little-Picture Implementations

Data is commonly used to improve the efficiency or control the costs of a particular business function. However, individual departmental goals may not align with the strategic goal of the organization, which is typically to increase revenue, Gilbert said.

“If the company can understand what the customer values, and build operational systems to better deliver, that is the company that’s going to win. If the company is being managed in pieces, you may save a dime in one department that costs the company a dollar in revenue.”

Six Ways to Master the Data-Driven Enterprise

As seen in InformationWeek.

StatisticsBig data is changing the way companies and industries operate. Although virtually all businesses acknowledge the trend, not all of them are equally prepared to meet the challenge. The companies in the best position to compete have transformed themselves into “data-driven” organizations.

Data-driven organizations routinely use data to inform strategy and decision-making. Although other businesses share the same goal, many of them are still struggling to build the necessary technological capabilities, or otherwise their culture is interfering with their ability to use data, or both.

Becoming a data-driven organization isn’t easy, however. In fact, it’s very difficult. While all organizations have a glut of data, their abilities to collect it, cleanse it, integrate it, manage it, access it, secure it, govern it, and analyze it vary significantly from company to company. Even though each of these factors helps ensure that data can be used with higher levels of confidence, it’s difficult for a business to realize the value of its data if its corporate culture lags behind its technological capabilities.

Data-driven organizations have extended the use of data across everyday business functions, from the C-suite to the front lines. Rather than hoping that executives, managers, and employees will use business intelligence (BI) and other analytical tools, companies that are serious about the use of data are training employees, making the systems easier to use, making it mandatory to use the systems, and monitoring the use of the systems. Because their ability to compete effectively depends on their ability to leverage data, such data-driven organizations make a point of aligning their values, goals, and strategies with their ability to execute.

On the following pages we reveal the six traits common to data-driven organizations that make them stand out from their competitors.

Forward Thinkers

Data-driven enterprises consider where they are, where they want to go, and how they want to get there. To ensure progress, they establish KPIs to monitor the success of business operations, departments, projects, employees, and initiatives. Quite often, these organizations have also established one or more cross-functional committees of decision-makers who collectively ensure that business goals, company practices, and technology implementations are in sync.

“The companies that have integrated data into their business strategies see it as a means of growing their businesses. They use it to differentiate themselves by providing customers with better service, quicker turnaround, and other things that the competition can’t meet,” said Ken Gilbert, director of business analytics at the University of Tennessee’s Office of Research and Economic Development, in an interview with InformationWeek. “They’re focused on the long-term and big-picture objectives, rather than tactical objectives.”

Uncovering Opportunities

Enterprises have been embracing BI and big data analytics with the goal of making better decisions faster. While that goal remains important to data-driven enterprises, they also are trying to uncover risks and opportunities that may not have been discoverable previously, either because they didn’t know what questions to ask or because previously used technology lacked the capability.

According to Gartner research VP Frank Buytendijk, fewer than half of big data projects focus on direct decision-making. Other objectives include marketing and sales growth, operational and financial performance improvement, risk and compliance management, new product and service innovation, and direct or indirect data monetization.

Hypothesis Trumps Assumption

People have been querying databases for decades to get answers to known questions. The shortcoming of that approach is assuming that the question asked is the optimal question to ask.

Data-driven businesses aim to continuously improve the quality of the questions they ask. Some of them also try to discover, through machine learning or other means, what questions they should be asking that they have not yet asked.

The desire to explore data is also reflected in the high demand for interactive self-service capabilities that enable users to adjust their thinking and their approaches in an iterative fashion.

Pervasive Analytics

Data analytics has completely transformed the way marketing departments operate. More departments than ever are using BI and other forms of analytics to improve business process efficiencies, reduce costs, improve operational performance, and increase customer satisfaction. A person’s role in the company influences how the data is used.

Big data and analytics are now on the agendas of boards of directors, which means that executives not only have to accept and support the use of the technologies, they also have to use them — meaning they have to lead by example. Aberdeen’s 2014 Business Analytics survey indicated that data-driven organizations are 63% more likely than the average organization to have “strong” or “highly pervasive” adoption of advanced analytical capabilities among corporate management.

Failure Is Acceptable

Some companies encourage employees to experiment because they want to fuel innovation. With experimentation comes some level of failure, which progressive companies are willing to accept within a given range.

Encouraging exploration and accepting the risk of failure that accompanies it can be difficult cultural adjustments, since failure is generally considered the opposite of success. Many organizations have made significant investments in big data, analytics, and BI solutions. Yet, some hesitate to encourage data experimentation among those who are not data scientists or business analysts. This is often because, historically, the company’s culture has encouraged conformity rather than original thinking. Such a mindset not only discourages innovation, it fails to acknowledge that the failure to take risks may be more dangerous than risking failure.

Data Scientists And Machine Learning

Data-driven companies often hire data scientists and use machine learning so they can continuously improve their ability to compete. Microsoft, IBM, Accenture, Google, and Amazon ranked first through fifth, respectively, in a recent list of 7,500 companies hiring data scientists. Google, Netflix, Amazon, Pandora, and PayPal are a few examples of companies using machine learning with the goal of developing deeper, longer-lasting, and more profitable relationships with their customers than previously possible.