Why is Artificial Intelligence still Science Fiction?

The first in a series of problem solving exercises aimed at determining what the next steps for AI might be.

Creating an Enterprise Data Strategy

An introduction to the process of developing comprehensive strategies for enterprise data manangement and exploitation.

Is Hype Killing IT Innovation?

Hype may not be as harmless as we think it is...

The Innovation Dilemma

What things actually promote or discourage innovation? We'll examine a few in this post...

Digitial Transformation, Defined

Digitial Transformation is a hot topic in IT and big money maker for consultants - but what does it really mean?.

Sunday, December 4, 2016

Technology & Election 2016 part 3 – The Failure of Data Science?

The first reaction on election night, November 8th, 2016 was – what, how did that happen? The entire country and in fact the whole world was more or less shocked at the unexpected outcome. But why was it so unexpected? The top level answer to that is simply that nearly every major poll or projection turned out to be wrong. What kind of numbers are we talking about? For example, Nate Silver’s FiveThirtyEight blog projected Clinton to win by about 5% in the popular vote (with 71% certainty). If we drill down to the state polls we see that the projections showed all 3 key turnover states, Wisconsin, Michigan and Pennsylvania going to Clinton by 3 to 5% in each contest. This is significant because many of these projections (and blogs like FiveThirtyEight) were using aggregates of dozens or hundreds of polls, not just one or a handful and the differences exceeded the margin of error.
Let’s step back for a moment and talk about the typical role that Data Science plays in the election process today. This role encompasses several well-known and some lessor known functions, including:
  • Predictive polling
  • Exit polling
  • Predictive modeling
  • Vote targeting (which facilitates a sort of CRM for campaign marketing as well as Get Out the Vote efforts)
There’s nothing new about polling, it’s been around for a long time. In fact, the last time there was a collective shock like this year’s outcome was in 1948, when the polls had predicted Dewey would win (by 50 to 45% - but Truman won by 50% to 45%). Polls have improved since then and of course, now we have the benefit of the latest data technology as well as 70 years of added experience, so how did nearly every major poll get it wrong this year? There are some theories; they include the following:
  1. A lot of people changed their minds at the last moment and weren’t particularly firm in their previous opinions.
  2. The Russians did it.
  3. Many polls were not properly targeting prospective voters for their models.
  4. Many people who had said they were voting for Clinton didn’t turn out to vote at all (e.g. the lack of enthusiasm)
  5. The poll numbers for the 3rd party candidates may have been inflated, and when it came to election day these candidates received far fewer votes that had been predicted (the implication being they went ahead and voted for one of the main candidates with Trump being the main beneficiary).
To be honest, we may never have a fully satisfactory answer for what happened in the 2016 election. It is likely that we’ve never had a race where both of the main candidates were universally unpopular and that kind of situation might never happen again (and there’s no telling how that may have impacted the polling results). How do we move forward then? Did technology, did Data Science fail us in 2016? Maybe, but probably not. What we witnessed however is that technology is only as good as our ability to apply it. If situations become more dynamic, complex within a relatively short window, do we stick with what we know or do we adjust our models or practices?
I’d like to step back for a moment here and ask a larger question. Do we really want ironclad predictions before elections in the first place? Before the big upset on election night, many pundits were talking about the lessons learned from the 1980 election where results from the East Coast encouraged West Coast voters to stay home thinking their votes didn’t really count. Because of that, the FEC passed a rule prohibiting networks from announcing winners before certain polls close. Don’t polls predicting a sure outcome before an election have a similar chilling effect? This year for example, how many voters may have stayed home because while they weren’t terribly enthused about Clinton, thought she would win? That’s a hard question to ask because people who don’t show up to vote can’t be interviewed in exit polls (or at least typically aren’t interviewed as part of the election post mortem).
What can we do in Future to avoid getting surprised?
I think it is important that in coming up with suggestions here, we need to weigh the relative value of using a particular solution with the potential impacts of using it. In other words, if we view predictive polling as a relativistic activity (e.g. a bit like Heisenberg’s principle in that taking a measure can influence the outcome), then we might conclude that the highest value of such predictive polls relative to possible harm might end several weeks before the election. How then could we be assured that elections are honest, that public sentiment is in fact aligned with election results? Well, that can still come through exit polling – polls taken of actual voters on election day and in addition, sentiment polls taken the day after the election of potential voters who didn’t vote (something that doesn’t typically happen now).  
Suggestion – Place a moratorium on predictive polls at least 2 weeks prior to election day and preferably 4 weeks prior. Why would this work? Here are few potential benefits of doing it:
  • This has the immediate effect of not making the election as much of a horse race and more of a contest of ideas.
  • It has at least the potential of driving up voter participation – a lot can happen in 4 weeks, people can vote based on their opinions and do a little less hedging in making their decisions.
  • It helps to combat Group-think (people swarming to the anticipated victor or becoming despondent about their own preferred candidates)
  • It certainly eliminates the main source of any potential surprise.
  • It may encourage candidates to take a more expansive view towards courting voters, recent trends have focused way too much attention on potential swing states and districts.
As you can tell, this and potentially other suggestions may have relatively little to do with Data Science itself, but have everything to do with how we apply it to given situations. I don’t believe the technology failed us here, I think we failed to recognize how much it already influences election outcomes. In my next post in this series, I’ll talk a bit more about Get Out the Vote (GOTV), politics as demographics and how technology has been and will be used to manage campaigns.

Copyright 2016, Stephen Lahanas 

Saturday, December 3, 2016

Technology & the 2016 Election Part 2: Voting Integrity

This is my second post on how the 2016 was defined not so much by flamboyant personalities but rather by the influences of technology in the election process. Today, I’m going to take a look at voting integrity, foreign influence and cyber threats.
At this very moment, election officials in Wisconsin are preparing to conduct an election recount requested by 3rd party candidate, Jill Stein. The request came after an blog post appeared written by a professor Alex Halderman who is on the board of advisors for the Verified Voting organization. In that post, Halderman stated that while there was no obvious evidence of voter fraud in the election, there were legitimate concerns regarding voting integrity based on a number of factors that have emerged from this year’s election.
So, why should we care about those concerns, isn’t just politics as usual? Good question. Here are some of the elements of this year’s election which seem a bit unusual:
  • Initial claims that all of the elections (and I refer to elections here because each State manages them somewhat differently, but we’ll come back that in a moment) were going to be rigged – this in itself was very strange and went so far as Trump stating he might not support the results if he lost. I can’t be sure of it, but I’m fairly certain that might be the first time in American history where a major candidate made such a statement.
  • Documented instances of election-related hacking by a foreign power and warnings of potential further acts from the US Intelligence Community. This accusation didn’t come from one party or another – it came from the FBI in October.
  • Potential interference by the same foreign power in other elections – There are indications that Russia may have also been involved in trying to influence the Brexit vote in the United Kingdom and this week, the head of German intelligence made an announcement that Russians may be prepared to launch attacks next year (with the aim of disrupting elections).
  • Discrepancies in exit polling and ballot results in some locations. Only one poll out of 100’s accurately predicted the national outcome and many state level polls turned out to be wrong as well. While this in itself may be explainable due to margin of error and last minute shifts in sentiment, when take into context with the rest of what’s been happening it seems at least a little bit fishy.
It is worth noting here that when we’re talking about potential election Fraud, there is a somewhat bizarre disconnect in the US as to what election fraud actually represents. Let’s take a look at that for a moment…
The Question of Election Fraud – The Big Picture
One thing that has puzzled me over the years, especially since the emergence of electronic voting, is why in the US, almost all discussion of voter fraud happens within the context of individual voter fraud. To me this is bit like saying that the only important crime that occurs is shoplifting at convenience stores while hacking into accounts and stealing billions of dollars isn’t troubling at all. Granted, petty theft is real and it is a problem, but one would have to ask does it represent a threat to our financial system? Obviously, it doesn’t. And perhaps this isn’t the best analogy ever (as there is far more petty theft than documented voter fraud on the individual level) but it helps to make the point in terms of scale and impact. We can’t see the forest for the trees here – we’ve simply been having the wrong conversation. The conversation should be about how intelligent enemies of Democracy, nation-states or terrorists could use their resources to influence or disrupt our political system by manipulating the American electoral process. And we need to keep in mind that this is not a partisan contention tied to any one election – it should be of equal importance to all sides within our political spectrum and applies to all current and future election cycles.
Another key point is the pragmatic nature of effective election manipulation. How would one of these rogue nations or organizations go about interfering in the American election process? Well, it would happen in one of the following ways (or potentially through a combination of them):
  • Through infection / control of individual voting machines with malware designed to manipulate votes. Moderately Effective
  • Through infection / control of election ‘back office’ systems, designated to aggregate vote totals. Now many people may not realize but these systems have been around for quite some time (since the 1960’s).  Most Effective
  • Through interception of vote totals between systems (data in transit). Moderately Effective
  • Through manipulation of Voter Registration information systems / data. Moderately Effective
We could also add to these the relatively intangible types of interference like those mentioned in part one of this article where I described briefly how fake news is being used to help shift voter sentiments. All of these things are hardly new (except that the technology has simply made it easier to manipulate large numbers of votes simultaneously) and have occurred in other countries before – some of them might be considered “Black Ops” and at one time were just standard features of Cold War proxy contests in the American & Soviet spheres of influence. The thing is though, back during the Cold War, these types of things happened elsewhere, not in the United States itself.
Bottom line - If someone really wanted to influence an election, they could do it in a number of meaningful ways and all of those ways involve technology at a system rather than an individual level.
This seems less about politics and more like Cyber Security, doesn’t it? In Cyber Security, the obvious question that one starts with regardless of the industry or context is this; if it is reasonable to assume that a vulnerability exists that someone, sometime, somewhere eventually could exploit – will it actually be exploited? The answer to this question is almost always yes and this has been borne out by events. While there is often a time delay between the appearance of a perceived vulnerability and the exploitation of that vulnerability, eventually the exploit does happen. Moreover, the incentive to perform an exploit is almost always directly proportional to the perceived value of the attack. What could be more valuable than changing the outcome of an American national election? Not much.
Elections and Foreign Interference
One of the most fascinating aspects of this election has been the unprecedented nature of foreign involvement or intervention in the process itself. And almost as surprising perhaps is the reaction or lack of reaction among the electorate in response. Until early this year, China had been the long-time headline grabber in relation to high profile hacks in the US. This year, Russia emerged as the most publicized perpetrator of cyber incursions, but the reality is that there are a number of other nations  and groups that are perhaps equally equipped to cause damage to either our political or economic systems. The identity of this moment’s headliner is much less important than the idea that we have weaknesses that any such attacker could choose to exploit. But that begs the question, what is it about our election process that makes it vulnerable. Here are a few points to consider:
  • We don’t have one election process, the reality is we have 50. Worse, we don’t have an ironclad underlying set of agreed upon technical standards which are consistent across all 50 states and there is no real authority to enforce it if we did anyway. There have been some standards developed by NIST and through a series of bills passed by Congress since 2000. But adherence to the standards (which are still somewhat inadequate) is contingent upon each State’s interpretation of them.
  • Even within a given state, from one State administration to the next, many if not most of the rules can change depending on the political agenda of whomever is in charge.
  • Conflicts of interest regarding who can own and sell electronic voting related equipment have never fully been resolved.
  • Then there is the Electoral College, which presents unique challenges to the country and one glaring vulnerability. The vulnerability boils down to this – given the way the college works – any attacker need only manipulate votes in a handful of states by a relatively small percentage to change the overall national result. Of course, this only applies to the national election but that is our most important one. For this to work, the polls in the targeted states would have to be within the statistical margin of error (1 to 3%), but that’s not uncommon at all. As we’ve seen time and again that margins of popular votes and electoral votes aren’t always aligned so an expected outcome can appear ‘normal.’ And this is why the current recount challenges have been targeted to Wisconsin, Michigan and Pennsylvania as all three came with 1% or less margins of victory. If we added the total of the 3 states’ electoral votes it comes to 46, which conceivably if all awarded to Clinton would flip the outcome of the election. I’m not saying that will happen, but it presents an example of the point here, that 3 states can easily tip a national election one way or the other.
So, what can we do if we do wish to recognize this threat and not dismiss it as politics as usual? Here are a couple of suggestions:
  • Provide tougher standards which ought to be applied uniformly across all 50 states for voting system security.
  • Ensure finally that every voting machine has a paper audit (e.g. prints a paper receipt ballot).
  • Regularly audit, monitor and test voting system security before during and after elections. Such audits must include random recounting of the paper trail (as opposed to an electronic recount) as well as examination of code both at the voting machine and back office level.
  • Give a more active role to America’s intelligence community to detect and counteract foreign threats or intrusions into our political processes. This is the real mission for which these agencies are chartered – it’s time we made election protection a national priority.  
  • Block spam / fake news sites from broadcasting into the US – specifically filtering out content originating from places where such campaigns a known to happen (Eastern Europe, etc.)
  • In situations where large scale breaches are detected, be prepared to redo elections as needed using paper ballots only (and provide funds to support these contingencies).
Ultimately, our form of government is only strong as the processes used to run it. Of those processes, elections are perhaps the most important single component. We can lose confidence in government but there’s always the hope of electing a better one, but if we lose confidence in voting, where does that leave us?

Copyright 2016, Stephen Lahanas

How Technology Defined the 2016 Election

There have been many fascinating stories, themes and memes that have emerged from the 2016 Election, but interestingly there is a common undercurrent running through them all that’s not being publicized so well. We’ve collectively focused almost entirely on the personalities behind the election with some minimal examination of processes, demographics, strategies and perhaps even less attention paid to policy & issues. But the real story may not be the “who” but the “how.” The actual game changer in 2016 wasn’t Donald Trump the candidate – it was technology as change facilitator.
Now, many of you might remember similar assertions being made after one or both of President Obama’s victories in 2008 or 2012. Social media and Big Data were credited with helping him to generate momentum and target voters. While it is true that those technologies made a difference for Obama in those elections, they weren’t exactly game changers in the overall scheme of things. Traditional media buys still happened, conventional wisdom on polling prevailed, Get out the Vote (GOTV) still functioned as usual and of course there weren’t any concerns that Russians were trying to pull a Watergate. This year though, all of that was turned on its head; the polls failed completely, Russians were launching cyber attacks, traditional media budgets and buys didn’t determine outcomes and GOTV was abandoned by one side entirely and yet they still won. Why did all of this happen?
Technology has finally caught up with the election process in almost all aspects of that process in 2016. One side understood the implications, the other didn’t and the rest is history.
At this point, I need to provide the obligatory disclaimer that like most of you, I wasn’t pleased with either choice presented in the November election. I’m not trying to compliment or side with one group or another here. But I do think it is important for both parties as well as the American Electorate to understand what mechanisms helped to shape outcomes in this year’s election. So, let’s take a look at each of these elements in turn. We’ll start in this post with Social Media.
Social Media – A lot was happening here and in truth it is still happening as Twitter has become the President Elect’s de facto Press Secretary. Some of us in the technology business had been positing for quite awhile that Social Media could and likely would begin to replace traditional media in terms of overall impact on the electorate so it wasn’t a complete surprise. However, when it finally happened this year it played out a little differently than we may have expected. For example, relatively few of us had taken into account the rapid and overwhelming influence of “Fake News,” on social media sites. However, to refer to all of this as fake news is perhaps a stretch, as there has been quite a lot of Advocacy journalism out there for awhile. But a certain percentage this year did fall into the totally fake news category and one might wonder if this in fact represents a form of Cyber attack or cyber warfare given the intent to change election outcomes. The reason I make the inference is due to the determination that much of this fake news was coming from Eastern Europe and may have been linked to the same Russian agenda that led to the confirmed hacks on Democratic Party servers, email accounts etc. But this is only part of the story behind the rise of Political Social Media.
Why is it that Social Media eclipsed traditional media in this year’s election? Here are some of the factors involved:
  • Social Media is now more used and trusted than it was even 4 years ago. Combined with smartphone mobile technology it is everywhere, all the time.
  • Traditional Media as we know it is beginning to submerge into a much larger and richer world of content which is being driven primarily by – you guessed it – technology. How soon will it be before Netflix or Amazon add news or information channels to the hundreds of cable & Roku channels already available. Once upon a time, we had 3 TV networks and perhaps 2 newspapers in each major city. Those days are gone forever.
  • Social Media is interactive and participatory – it isn’t just information consumption. While people can comment on traditional media sites, on social media ordinary people can actually drive the dialog. This is enormously powerful and it does in many ways represent a sort of technological populism. (and despite all of the recent news equating Populism with the Alt Right, Populism is not inherently Right Wing at all – in fact our entire form of government is predicated on what was in essence a Populist experiment).
  • People are tired of the art of negative advertising, at least in its 30 second format. Also, at a deeper level, when one advertises on traditional television or radio he or she is “interrupting” something (e.g. providing content we don’t want) whereas on Social Media the politics can be the entertainment in that people are directed to or served content based on their interests. The medium is the message here for real – in ways Marshall McLuhan never could have predicted.
  • Thus, people can tailor their media experience using Social Media based on their own comfort levels. This is perhaps the area that concerns most traditional journalists and social scientists who have complained ever since the first Mosaic browser came out that Internet content could not be trusted nor could the people viewing it. However, this does give people a feeling of empowerment they wouldn’t otherwise have.  
  • Social media is a much more cost effective proposition in terms of production of content and the ability to reach interested audience. A traditional political campaign may have spent ½ or more of their budget on ad production and ad buys in the past. This is no longer required and in fact may be the biggest single blow to the flood of dark money in politics that’s ever happened – we just haven’t recognized that yet given that the person who exploited this is a billionaire supported by a lot of dark money donors. Regardless though, the door has been opened.
  • And perhaps most importantly in this election, Social Media’s focus on advocacy and dialog allowed for a much more intense debate without some of the controls that would have been in place under traditional media. People enjoy spectacle, sensation and those who are willing to focus on that over ideas or facts tend to get a lot of free attention on Social Media. While there have always been Tabloid publications out there, they were never accepted into the mainstream the way Social Media has. We see all of these factors coming together to become game changing – fake news, comfort zone filtering of content, sensationalism and the ability to not only join, but to really influence the national dialog. While we might not like some of this, it may be problematic trying to impose limits on it, so for better or worse, we need to deal with it.
I think it may be a safe prediction at this point to say that any campaign for any office going forward that doesn’t view Social Media as its primary media channel, will likely risk losing their election and will certainly experience a diminishing return on investment for dollars spent on traditional media buys. This could, at least near-term, tend to even the playing field for many candidates.
In my next post in this series, I’ll discuss the Russians, Cyber threats and voting integrity.  

copyright 2016 - Stephen Lahanas

Wednesday, December 3, 2014

The Growing State of Cyber Insecurity

2014 will likely be marked as the year that the warnings from the past decade about Cyber threats were finally realized. Granted, not all of those warnings have come true, yet - but this year will go down as the worst yet for costly Cyber breaches. That begs an important question - why are we becoming less secure as time is passing - and why haven't the billions of dollars invested in Cyber Security worked?
This is a complex topic, so it will probably help to provide some high level context. We'll start with some definitions:
  • Security Architecture - the practice of actively designing security into complex systems or environments.
  • Intrusion Detection - the backbone for most perimeter-focused security solutions; focus is detection / prevention of breaches.
  • Threat / Vulnerability Management - the practice of tracking and adapting to specific threat vectors (attack signatures, exploits etc.)
  • Security Controls - usually standards-based system & process framework for assessing, securing and auditing security status.
  • Social Engineering - the practice of using non-technical persuasion or other techniques to gain information in order to access secure environments.
Now, let's ask the question again. Target, Chase, Sony Pictures - why is this year the year of massive security breaches? What went wrong?
There are 5 top reasons that this is happening; I'll introduce them together and then explore each one in detail later.
  1. It is easier to Cyber Attack than to Cyber Defend and likely always will be.
  2. Cyber Security is not viewed from a holistic perspective in most organizations today - this includes many military organizations.
  3. There is no one magic bullet technique or technology that can secure an organization - yet we spend a lot of our time looking for one or thinking we have one.
  4. Just as we secure one aspect of the enterprise, 3 new ones pop up that aren't secure - and in many cases each of these offer attack routes back through the areas we thought were secure.
  5. Cyber Security represents an intersection between (human) behavior and information patterns. We haven't yet resolved either of these issues separately yet and we definitely aren't close to dealing with how they intersect.
a representation of pattern identification in Cyber Attacks
So, who am I to discuss such matters? I'm not a recognized Cyber Security expert that's true. I'm just an IT Architect. But, I'm an Architect who has had the privilege of working on some fascinating Security related projects over the years; my first ones were in 1998 and 1999. In 1998, I worked on a research project for the AF to help develop a next generation Intrusion Detection system - we called it the Secure Adaptive Network Environment (SANE). As you can tell, it is was perimeter and data center focused. The second project was much more ambitious, I was brought in as a security architect (from the AF perspective) for the first iteration for GCSS-AF, which was and still is a large data center consolidation, application hosting initiative (now much of it is Cloud-based). Both of these projects helped (for me anyway) to illustrate a number of the key problems that would be associated with Cyber Security for the coming decades (although back then we didn't call it Cyber Security yet). Some of those observations included:
  • The notion that the landscape was going to get ever more complex
  • The need for unified access control (directory services as well as application logins etc)
  • The need for various levels network security (which was in fact already deployed in the DoD) as well as encryption across public networks
  • I saw how easy it was for dedicated enthusiasts to breach most systems they set their sights on (sat in on a few of the first 'hackathons')
  • I saw that static or reactive security was the standard operating approach behind most perimeter based security approaches and it was never going to work
  • I saw that we in the business we spending way too much time focusing on the products that were supposed to make us secure rather than understanding or controlling the holistic processes necessary for real security.
  • And then there is all that log data - which was only going to grow and grow until it would become unmanageable.
  • It was obvious that Cyber Space would become another 'field of battle' alongside air, ground, water and space. There would be both state-sponsored and free-enterprise focused organized cyber cadres. These groups have had nearly 20 years to mature in 2014 - the future of Cyber Security was not individual hacker like Neo (from the Matrix) but Cyber crime syndicates and armies.
Ten years after these initial security projects, things were developing pretty much the way I had anticipated. If anything, things may have developed slower than I had anticipated - in terms of the numbers or severity of the breaches happening in 2008 / 2009, but the trajectory was definitely on track. I thought the time was ripe for moving to the next stage of Cyber defense, but remarkably, I found quite a lot of resistance to the notion of taking a holistic view of Cyber Security, so I moved on to other my productive arenas.
Example of a Cyber (Defense) Collaboration approach across organizations
Holistic Cyber Security is of course where things have to go and the answer to what's missing. Let's look at each of the five issues I identified above in more depth:
  1. It's easier to attack: Why should this be the case? Well, the tools that Hackers, Crackers or rogue Cyber syndicates or armies use are less expensive and less complex to use than the tools we use to defend assets. A hacker can get started with almost no investment while each component of a let's say a perimeter defense architecture may cost millions and take months to implement. Worse than that though is that attackers work as a collaborative community - which means they can collectively share information on how to defeat that new defensive technology and eventually we end up playing a reactive role - fixing vulnerabilities only after they surface. This situation is unlikely to change under current defensive paradigms.
  2. Piecemeal Security: That's the opposite of holistic isn't it? Think about this. Every IT capability in a modern organization represents a potential threat to security. Whether we're talking about a Cloud, a mobile app, an edge device that needs to be secured, data in motion, applications (web based or otherwise), files and documents, email, portals etc.,etc.,etc. And usually all of these things are not managed by the same groups within an organization and often many of these things aren't considered as part of the security landscape at all. Most of the focus for Cyber Security in today's enterprise is still hovering around the perimeter and network. While this part of the picture is important - it is not the whole picture and never was; not in 1998, not in 2008 and certainly not now. On a recent 60 Minutes report, a famous security expert mentioned an even more telling aspect of this problem - even at the perimeter there is now so much information being generated there is no way to discern what are the real threats. We'll talk about that more in a minute.
  3. There is no magic bullet: This is a bad habit shared by other aspects of IT, but for Cyber Security this thinking is particularly problematic. In the late 90's and early 2000's the magic bullet was Intrusion Detection and Firewalls. Then there was PKI and host of other encryption protocols and products and of course anti-virus software has become more and more pervasive since the late 90's. Even the notion of security standards or controls has been viewed as a magic bullet, but the fact is whether it is processes, standards or products - all of these elements represent 'part' of a larger picture. That larger picture needs to begin with deliberate Security Architecture on an enterprise scale.
  4. Cyber Security is Dynamic: Yet most security organizations and products aren't. We understood that all the back in 1998, which is why we began building community contribution of exploits into Intrusion Detection products. Collaboration on the defensive side is there, but it still isn't as effective as the collaboration on the attacking side; mainly because the job of the defenders is many times more complex. Becoming dynamic is no small task - it requires a paradigm shift in thinking for most organizations and thusfar it is very rare to see it in practice.
  5. Cyber Security is Information & People: A proactive approach to security requires the defenders think like those who might attack them and predict or identify weakness. It requires the ability to discern or predict patterns in the ever growing sets of data (just as was highlighted on 60 minutes). This simply has not happened yet. Despite some progress with Security Controls and Vulnerability / Threat Management, we are still largely operating in a reactive mode. We don't have a good handle on stopping insider attacks or understanding threat behaviors.
In some ways, we've been lucky so far that the Cyber attacks have been primarily focused on stealing information or financial data, rather than attacks on systems dedicated to infrastructure. While many of those systems are somewhat more secure by design, they are not as secure as we might think (just as the breaches this year have called into question the efficacy of security associated with PCI standards and finance-related systems). We are becoming more Cyber Insecure because we are not as adaptive as our opponents and because we still refuse to recognize the full scope of the challenge. In many cases, we are spending perhaps exactly as much as we need to - but we're not spending it the right way or in the right context. We're paying for piecemeal security and unfortunately that's what exactly we're getting.

copyright 2014, Stephen Lahanas

Tuesday, September 23, 2014

The Art in Artificial Intelligence - Part 2

Part of the reason we decided to explore this topic on Technovation Talks was the claim made earlier this Summer that an AI had finally passed the Turing Test. So what's the Turing Test? It was a very basically described metric by which any sort of true machine intelligence might be assessed or otherwise verified. Here's the basic premise of the test - if an AI can engage in normal conversation with multiple human participants without the humans realizing that they were conversing with a machine (obviously it would be a remote conversation of some sort) - then the machine could be considered intelligent.

Alan Turing - BTW: a movie about him will be hitting theaters soon...
According to Turing's predictions in 1950, we should have already achieved this level of machine intelligence (by the end of the last century). Yet if you look at the story about this Summer's supposed triumph (which might be considered the first time it has in fact been achieved, there are nothing but problems and doubts):

  • First off, the answers are screwy and it's clear that much of what the computer heard it misinterpreted. 
  • Then they presented the AI as if it were an adolescent from war-torn Ukraine.
  • And they also used the lowest possible threshold to gauge success - this threshold which represented a part of Turing's paper on the subject - suggested that success be declared if on average at least 30% of humans judging the AI would be fooled into thinking it was a human. So, the AI named Eugene, scored a 33% - but that is only because judges lowered the bar thinking he was a semi-illiterate teen. 
More important than all of this of course is the central question as to whether or not the metric or test is actually an accurate way to assess machine intelligence anyway? In a way, every system that has ever tried to compete in one of these tests to date has been purpose-built to pass the test. But does that make it intelligent (if it were actually to pass it)? The technology necessary for a machine to "think" through a conversation the way a human does simply does not exist - nor are we even close to understanding what that model would even look like. The systems trying to pass the Turing Test are simply conversational "hacks," in other words they include built-in tricks like responding to a question with a question or trying to work off of keyword cues. What's missing of course is any continuity of thought - any consciousness - and even the most simplistic conversation requires that. None of these systems can think and none of them can really learn. 

Now it may be that conversation hacking may become sophisticated enough in coming years so that many of these systems may actually pass the Turing Test threshold of 30% on a regular basis. But that test as it is now defined will never provide us with an accurate assessment as to whether a machine has in fact achieved some innate level of intelligence. There is no way to determine through the conversation if the system has "added value" to the topic rather than simply replied phrase by phrase in rather one-sided dialectics. It will be difficult to assess or acknowledge any growth or change. There is no expectation in a simple conversation to determine if you are in fact conversing with a self-aware entity.

In the movie Her, this guy falls in love with his operating system (and it didn't come from the Apple store!)
The first thing we need to do before we tackle how we might achieve AI is to determine what the appropriate assessment or validation for human-like intelligence really needs to be. We are going to suggest one and explain the rationale for it...

The Technovation AI Test -
AI Test Prerequisites / Expectations
  • The Test is not meant to assess acquired knowledge per se, it is meant to assess cognitive ability. In other words, it is not about preparation or repetition of learned information, but is concerned with potential and / or application of any particular knowledge set.
  • The Test does not have to occur in one sitting, but take place over any duration (within reason).
  • The Test isn't merely concerned with correct answers or maturity in a point of time, but can also assess the ability to grow over time based upon responses to various aspects of the test (or other stimuli encountered within the time-frame of the test).
  • The Test is not merely a linguistic exercise - the machine must not merely demonstrate the ability to communicate like a human, it must also demonstrate it can learn. 
  • Foremost above all else though, the machine must demonstrate the one trait most closely associated human intelligence (as opposed to raw computing power) - it must demonstrate intuition. In this context, Intuition represents shorthand problem-solving (which we will discuss in much more depth in a future post). 
  • On last aspect of the test that must be included is a review of the code to ensure that "conversational snippets" are not allowed to be prep-programmed. This implies that the majority of dialog is generated 'real time' by the machine. Now, that would not prevent the machine from reviewing logs of previously generated dialog (in some database), but that review could not lead to verbatim quoting -  rather must paraphrase or other restate previous points. 
The AI Test 
In a series of panel interviews, the AI must convince the judges or reviewers that it should be hired to perform a complex human role. The type of job and foundational knowledge can cover any number of topics but must be sufficiently complex to avoid "lowering the bar." (so, any job that requires a degree). Also, the interview style must be open (similar to essay tests in written assessments) - the answers must not just be correct, they must demonstrate value added insight from the intelligence conveying them. And the answers may be entirely subjective... (even better as long as the machine can rationalize them)
This test necessarily implies a very high threshold - perhaps in excess of a 90% rating for a very complex set of conversations. Why raise the bar this high? Simple - this is the one way we can force the development of a system that can both learn and apply that knowledge to problem solving and do it on the fly. To have human like intelligence, machines must have the ability to understand nuances of human communication and psychology - thus it must not only be able to interact, it must be able to convince us as well.  
Now that we have a more concrete target to aim for - how do we get there. In our next post, we'll delve into Learning - what works and what doesn't and how human and machine intelligence differ today.

Copyright 2014, Stephen Lahanas