Lisa Morgan's Official Site

Strategic Insights and Clickworthy Content Development

Month: June 2018

Machine Learning’s Greatest Weakness Is Humans

June 29, 2018 /

Machine learning– deep learning and cognitive computing in particular– attempt to model the human brain. That seems logical because the most effective way to establish bilateral understanding with humans is to mimic them. As we have observed from everyday experiences, machine intelligence isn’t perfect and neither is human intelligence.

Still, understanding human behavior and emotion is critical if machines are going to mimic humans well. Technologists know this, so they’re working hard to improve natural language processing, computer vision, speech recognition, and other things that will enable machines to better understand humans behave more like humans

I imagine that machines will never emulate humans perfectly because they will be able to rapidly identify the flaws in our thinking and behavior and improve upon them. To behave exactly like us would

From an analytical perspective, I find all of this fascinating because human behavior is linear and non-linear, rational and irrational, logical and illogical. If you study us at various levels of aggregation, it’s possible to see patterns in the way humans behave as a species, why we fall into certain groups and why behave the way we do as individuals. I think it would be very interesting to compare what machines have to say about all of that with what psychologists, sociologists, and anthropologists have to say.

Right now we’re at the point where we believe that machines need to understand human intelligence. Conversely, humans need to understand machine intelligence.

Why AI is Flawed

Human brain function is not infallible. Our flaws present challenges for machine learning, namely, machines have the capacity to make the same mistakes we do and exhibit the same biases we do, only faster. Microsoft’s infamous twitter bot is a good example of that.

Then, when you model artificial emotional intelligence based on human emotion, the results can be entertaining, inciting or even dangerous.

Training machines, whether for supervised or unsupervised learning, begins with human input at least for now. In the future, the necessity for that will diminish because a lot of people will be teaching machines the same things. The redundancy will indicate patterns that are easily recognizable, repeatable and reusable. Open source machine learning libraries are already available, but there will be many more that approximate some aspect of human brain function, cognition, decision-making, reasoning, sensing and much more.

Slowly but surely, we’re creating machines in our own image.

The Trouble with Data About Data

June 28, 2018 /

Two people looking at the same analytical result can come to different conclusions. The same goes for the collection of data and its presentation. A couple of experiences underscore how the data about data — even from authoritative sources — may not be as accurate as the people working on the project or the audience believe. You guessed it: Bias can turn a well-meaning, “objective” exercise into a subjective one. In my experience, the most nefarious thing about bias is the lack of awareness or acknowledgement of it.

The Trouble with Research

I can’t speak for all types of research, but I’m very familiar with what happens in the high-tech industry. Some of it involves considerable primary and secondary research, and some of it involves one or the other.

Let’s say we’re doing research about analytics. The scope of our research will include a massive survey of a target audience (because higher numbers seem to indicate statistical significance). The target respondents will be a subset of subscribers to a mailing list or individuals chosen from multiple databases based on pre-defined criteria. Our errors here most likely will include sampling bias (a non-random sample) and selection bias (aka cherry-picking).

The survey respondents will receive a set of questions that someone has to define and structure. That someone may have a personal agenda (confirmation bias), may be privy to an employer’s agenda (funding bias), and/or may choose a subset of the original questions (potentially selection bias).

The survey will be supplemented with interviews of analytics professionals who represent the audience we survey, demographically speaking. However, they will have certain unique attributes — a high profile or they work for a high-profile company (selection bias). We likely won’t be able to use all of what a person says so we’ll omit some stuff — selection bias and confirmation bias combined.

We’ll also do some secondary research that bolsters our position — selection bias and confirmation bias, again.

Then, we’ll combine the results of the survey, the interviews, and the secondary research. Not all of it will be usable because it’s too voluminous, irrelevant, or contradicts our position. Rather than stating any of that as part of the research, we’ll just omit those pieces — selection bias and confirmation bias again. We can also structure the data visualizations in the report so they underscore our points (and misrepresent the data).

Bias is not something that happens to other people. It happens to everyone because it is natural, whether consciously or unconsciously. Rather than dismiss it, it’s prudent to acknowledge the tendency and attempt to identify what types of bias may be involved, why, and rectify them, if possible.

I recently worked on a project for which I did some interviews. Before I began, someone in power said, “This point is [this] and I doubt anyone will say different.” Really? I couldn’t believe my ears. Personally, I find assumptions to be a bad thing because unlike hypotheses, there’s no room for disproof or differing opinions.

Meanwhile, I received a research report. One takeaway was that vendors are failing to deliver “what end customers want most.” The accompanying infographic shows, on average, that 15.5% of end customers want what 59% of vendors don’t provide. The information raised more questions than it answered on several levels, at least for me, and I know I won’t get access to the raw data.

My overarching point is that bias is rampant and burying our heads in the sand only makes matters worse. Ethically speaking, I think as an industry, we need to do more.

 

Analytics Leaders and Laggards: Which Fits Your Company?

June 20, 2018 /

Different companies and industries are at different levels of analytical maturity. There are still businesses that don’t use analytics at all and businesses that are masters by today’s standards. Most organizations are somewhere in between.

So, who are the leaders and laggards anyway? The International Institute for Analytics (IIA) asked that question in 2016 and found that digital natives are the most mature and the insurance industry is the least mature.

How Industries and Sectors Stack Up

IIA’s research included 11 different industries and sectors, in addition to digital natives. The poster children included Google, Facebook, Amazon, and Netflix. From Day 1, data has been their business and analytics has been critical to their success.

The report shows the descending order of industries in terms of analytical maturity, with insurance falling behind because its IT and finance analytics are the weakest of all.

Another report, from business and technology consultants West Monroe Partners found that only 11% of the 122 insurance executives they surveyed think their companies are realizing the full benefits of advanced analytics. “Advanced analytics” in this report is defined as identifying new revenue opportunities, improving customer and agent experience, performing operational diagnostics, and improving control mechanisms.

Two of the reasons West Monroe cited for the immaturity of the insurance industry are the inability to quantify the ROI and poor data quality.

Maturity is a Journey

Different organizations and individuals have different opinions about what an analytics maturity model looks like. IIA defines five stages ranging from “analytically impaired” (organizations that make decisions by gut feel) to “analytical nirvana” (using enterprise analytics).

“Data-first companies haven’t had to invest in becoming data-driven since they are, but for the companies that aren’t data-first, understanding the multi-faceted nature of the journey is a good thing,” said Daniel Magestro, research director at IIA. “There’s no free lunch, no way to circumvent this. The C-suite can’t just say that we’re going to be data-driven in 2017.”

Others look at the types of analytics companies are doing: descriptive, predictive, and prescriptive. However, looking at the type of analytics doesn’t tell the entire story.

What’s interesting is that different companies at different stages of maturity are stumped by different questions: Do you think you need analytics? If the answer is no, then it’s going to be a long and winding road.

Why do you think you need analytics? What would you use analytics to improve? Those two related questions require serious thought. Scope and priorities are challenges here.

How would you define success? That can be a tough question because the answers have to be quantified and realistic to be effective. “Increase sales” doesn’t cut it. How much and when are missing.

One indicator of maturity is what companies are doing with their analytics. The first thing everyone says is, “make better business decisions,” which is always important. However, progressive companies are also using analytics to identify risks and opportunities that weren’t apparent before.

The degree to which analytics are siloed in an organization also impacts maturity as can the user experience. Dashboards can be so complicated they’re ineffective versus simple to prioritize and expedite decision-making.

Time is another element. IT-created reports have fallen out of favor. Self-service is where it’s at. At the same time, it makes no sense to pull the same information in the same format again and again, such as weekly sales reports. That should simply be automated and pushed to the user.

The other time element — timeliness whether real-time, near real-time, or batch — is not an indication of maturity in my mind because what’s timely depends on what’s actually necessary.

How Valuable Is Your Company’s Data?

June 14, 2018 /

Companies are amassing tremendous volumes of data, which they consider their greatest asset, or at least one of their greatest assets. Yet, few business leaders can articulate what their company’s data is worth.

Successful data-driven digital natives understand the value of their data and their valuations depend on sound applications of that data. Increasingly venture capitalists, financial analysts and board members will expect startup, public company and other organizational leaders to explain the value of their data in terms of opportunities, top-line growth, bottom line improvement and risks.

For example, venture capital firm Mercury Fund recently analyzed SaaS startup valuations based on market data that its team has observed. According to Managing Director Aziz Gilani, the team confirmed that SaaS company valuations, which range from 5x to 11x revenue, depend on the underlying metrics of the company. The variable that determines whether those companies land in the top or bottom half of the spectrum is the company’s annual recurring revenue (ARR) growth rate, which reflects how well a company understands its customers.

Mercury Fund’s most successful companies scrutinize their unit economics “under a microscope” to optimize customer interactions in a capital-efficient manner and maximize their revenue growth rates.

For other companies, the calculus is not so straightforward and, in fact, it’s very complicated.

Direct value

When business leaders and managers ponder the value of data, their first thought is direct monetization which means selling data they have.

“[I]t’s a question of the holy grail because we know we have a lot of data,” said David Schatsky, managing director at Deloitte. “[The first thought is] let’s go off and monetize it, but they have to ask themselves the fundamental questions right now of how they’re going to use it: How much data do they have? Can they get at it? And, can they use it in the way they have in mind?”

Data-driven digital natives have a better handle on the value of their data than the typical enterprise because their business models depend on collecting data, analyzing that data and then monetizing it. Usually, considerable testing is involved to understand the market’s perception of value, although a shortcut is to observe how similar companies are pricing their data.

“As best as I can tell, there’s no manual on how to value data but there are indirect methods. For example, if you’re doing deep learning and you need labeled training data, you might go to a company like CrowdFlower and they’d create the labeled dataset and then you’d get some idea of how much that type of data is worth,” said Ben Lorica, chief data officer at O’Reilly Media. “The other thing to look at is the valuation of startups that are valued highly because of their data.”

Observation can be especially misleading for those who fail to consider the differences between their organization and the organizations they’re observing. The business models may differ, the audiences may differ, and the amount of data the organization has and the usefulness of that data may differ. Yet, a common mistake is to assume that because Facebook or Amazon did something, what they did is a generally-applicable template for success.

However, there’s no one magic formula for valuing data because not all data is equally valuable, usable or available.

“The first thing I look at is the data [a client has] that could be turned into data-as-a-service and if they did that, what is the opportunity the value [offers] for that business,” said Sanjay Srivastava, chief digital officer at global professional services firm Genpact.

Automation value

More rote and repeatable tasks are being automated using chatbots, robotic process automation (RPA) and AI. The question is, what is the value of the work employees do in the absence of automation and what would the value of their work be if parts of their jobs were automated and they had more time to do higher-value tasks?

“That’s another that’s a shortcut to valuing that data that you already have,” said O’Reilly’s Lorica.

Recombinant value

Genpact also advances the concept of “derivative opportunity value” which means creating an opportunity or an entirely new business model by combining a company’s data with external data.

For example, weather data by zip code can be combined with data about prevalent weeds by zip code and the available core seed attributes by zip codes. Agri-food companies use such data to determine which pesticides to use and to optimize crops in a specific region.

“The idea is it’s not just selling weather data as a service, that’s a direct opportunity,” said Srivastava. “The derivative opportunity value is about enhancing the value of agriculture and what value we can drive.”

It is also possible to do an A/B test with and without a new dataset to determine the value before and after the new data was added to the mix.

Algorithmic value

Netflix and Amazon use recommendation engines to drive value. For example, Netflix increases its revenue and stickiness by matching content with a customer’s tastes and viewing habits. Similarly, Amazon recommends products, including those that others have also viewed or purchased. In doing so, Amazon successfully increases average order values through cross-selling and upselling.

“Algorithmic value modeling is the most exciting,” said Srivastava. “For example, the more labeled data I can provide on rooftops that have been damaged by Florida hurricanes, the more pictures I have of the damage caused by the hurricanes and the more information I have about claim settlements, the better my data engine will be.”

For that use case, the trained AI system can automatically provide an insurance claim value based on a photograph associated with a particular claim.

Risk-of-Loss value

If a company using an external data source were to lose access to that data source, what economic impact would it have? Further, given the very real possibility of cyberattacks and cyberterrorism, what would the value of lost or corrupted data be? Points to consider would be the financial impact which may include actual loss, opportunity cost, regulatory fines and litigation settlement values. If the company has cybersecurity insurance, there’s a coverage limit on the policy which may differ from the actual claim settlement value and the overall cost to the company.

A bigger risk than data loss is the failure to use data to drive value, according to Genpact’s Srivastava.

There’s no silver bullet

No single equation can accurately assess the value of a company’s data. The value of data depends on several factors, including the usability, accessibility and cleanliness of the data. Other considerations are how the data is applied to business problems and what the value of the data would be if it were directly monetized, combined with other data, or used in machine learning to improve outcomes.

Further, business leaders should consider not only what the value of their company’s data is today, but the potential value of new services, business models or businesses that could be created by aggregating data, using internal data or, more likely, using a combination of internal and external data. In addition, business leaders should contemplate the risk of data loss, corruption or misuse.

While there’s no standard playbook for valuing data, expect data valuation and the inability to value data to have a direct impact on startup, public company, and merger and acquisition target valuations.

Lisa Morgan Advances Digital Ethics

June 10, 2018 /

Lisa Morgan is on a mission to educate the high-tech industry about the importance of digital ethics.  In advancement of that goal, she has been appointed Program Manager, Content and Community of the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems Outreach Committee.  In that capacity, she is responsible for content and community development for the worldwide membership which now exceeds 1,050 members.  She is also a contributor to the group’s Ethically-Aligned Design document that is being cooperatively developed by technologists, business leaders, law makers, attorneys and others dedicated to advancing A/IS ethics.

Why Operationalizing Analytics is So Difficult

June 2, 2018 /

Today’s businesses are applying analytics to a growing number of use cases, but analytics for analytics’ sake has little, if any, value. The most analytically astute companies have operationalized analytics, but many of them, particularly the non-digital natives, have faced several challenges along the way getting the people, processes and technology aligned in a way that drives value for the business.

Here are some of the hurdles that an analytics initiative might encounter.

Analytics is considered a technology problem

Some organizations consider analytics a technology problem, and then they wonder why the ROI of their efforts is so poor. While having the right technology in place matters, successful initiatives require more.

“The first key challenge is designing how and in what way an analytics solution would affect the outcome of the business,” said Bill Waid, general manager of Decision Management at FICO. “We start by modeling the business problem and then filling in the analytic pieces that address that business problem. More often than not, there’s a business process or business decision that needs to be incorporated into the model as we build the solution.”

Framing the business problem is essential, because if the analytics don’t provide any business value, they won’t get used.

“Better than 80% of analytics never end up being used. A lot of that stems from the fact that an analysis gets built and it might make sense given the dataset but it’s not used to make something happen,” said Waid. “That’s probably the hardest element.”

Placing analytics in the hands of the business requires access to the right data, but governance must also be in place.

“[T]he technical aspects are becoming easier to solve and there are many more options for solving them, so the people and the process challenges that you’ll face obviously have to come along,” said Bill Franks, chief analytics officer at the International Institute for Analytics (IIA). “In a non-digital-native company, the people and process progress does not match the technology progress.”

Operationalizing analytics lacks buy in

Many analytics initiatives have struggled to get the executive and organizational support they need to be successful. Operationalizing analytics requires the same thing.

“When you operationalize analytics, you’re automating a lot of decisions, so the buy-in you require from all of the various stakeholders has to be high,” said IIA’s Franks. “If you’re a digital native, this is what you do for a living so people are used to it. When you’re a large, legacy company dipping your toe into this, the first couple of attempts will be painful.”

For example, if an organization is automating what used to be batch processes, there need to be more safety checks, data checks, and accuracy checks. Chances are high that everything won’t be done right the first time, so people have to get comfortable with the concept of iteration, which is just part of the learning process.

Analytical results are not transparent

If your company operates in a regulated environment, you need to be able to explain an analytical result. Even if you’re not in a regulated industry, business leaders, investors and potential M&A partners may ask for an explanation.

“We refer to it as ‘reasoning code’ or ‘the outcomes,’ but in AI it’s a form of explainable AI where you can explain to a business owner or a business user why the analytics came to the conclusion it came to,” said FICO’s Waid. “The second thing that you need to provide the business person with is some kind of dashboard for them to be able to change, adjust or accommodate different directions.”

© 2024 Lisa Morgan's Official Site

Theme by Anders Noren — Up ↑