Capability-Driven Acquisition

The term ‘Acquisition’ is almost a bit misleading. The primary image that comes to mind when one first hears the term is that acquisition represents the purchase of some tangible set of goods or services. Once upon a time, that may have been true, but today things aren’t so clear cut. Acquisition goes well beyond procurement, and in the federal realm it has become an incredibly complex set of rules regulations and practices. Federal acquisition answers demands for specific oversight into the management and dissemination of tax dollars, but does it really work the way it was intended to? How can all of those rules, all of those folks watching still fail to obtain what they intended to receive on our behalf – it happens all of the time…

The federal acquisition community is well aware of these issues and has been struggling for the past two decades with new approaches to solve the problem. A lot of positive developments have come about; pieces to the puzzle, but yet the puzzle remains incomplete. The most significant advance in Federal Acquisition theory emerged about seven years ago; “Performance-Based Acquisition.” The best way to describe what that means is by reviewing its process elements or steps (taken from an executive summary here,

• Step 1 – Establish an integrated solutions team
• Step 2 - Describe the problem that needs solving
• Step 3 - Examine private-sector and public-sector solutions
• Step 4 - Develop a performance work statement (PWS) or statement of objectives (SOO)
• Step 5 – Decide how to measure and manage performance
• Step 6 - Select the right contractor
• Step 7 - Manage performance

So why did the federal government move in this direction and what were they doing before Performance-Based Contracting and what do the seven steps mean?

Background
Government acquisition didn’t begin in The United States. Many ancient civilizations had highly complex bureaucracies; Egypt, China, the Roman Empire. All of these societies had governments charged with managing the public wealth, and they exercised that power to build; roads, temples, ships and armies. And in every case, there was always the possibility for mismanagement and corruption. Some of this country’s most famous scandals involved misappropriation of public funds. Any time an event such as that has occurred it led to reform initiatives, that combined with the general evolution of procurement practices and technology gradually built up to the acquisition laws and workforce we have in place now. But something else has been happening recently that has fundamentally changed the nature of acquisition – the advent of information technology.

Dealing with procurement of relatively unsophisticated materials or services on an ‘enterprise scale’ can be challenging enough in its own right; however those both buying and selling shared a significant advantage in the great majority of instances that the nature of what was being transacted was thoroughly understood. A hammer, an engine a sack of potatoes all share a certain lack of ambiguity that is familiar and comforting. It is very easy to determine whether the hammer is faulty, whether an engine runs or whether the potato is fresh or rotten. We also have an easily identifiable cultural context for how much each of these should cost so that when someone discovers that the government paid $150 for hammer the discontinuity is obvious.

When the procurement world was largely concerned with this type of acquisition; the process of buying was directed mainly on materials management. The exception of course for the past 75 years or so has been research and development which has always been considered a high risk proposition. The focus on materials management translated into a contract-centric, bureaucratic approach to acquisition. There was an implied assumption that if all of the steps built into the process (designed to protect the government from fraud), were followed that all would be well – there was little need or incentive to actively oversee or otherwise examine procurements. This assumption began to break down during the cold war as the United States began to increase the number and complexity of its R& D projects and began deploying more complex technologies. The early manifestations of the problem mostly involved the failure, delay or cost overruns of a variety of complex weapons systems. At that point, the failures were still the exception, not the rule.

Then, beginning in the mid-1970’s, two things happened; first those complex weapons systems become dependent on software, and second the federal government became the largest systems development and integration entity in the world. In many ways, the commercial IT industry owes its very existence to the massive investments and fundamental breakthroughs provided by federal programs in the late sixties and throughout the 70’s. The federal acquisition community was being hurtled from a post civil war paradigm to the space age in little more than a decade. Materials management is an objective discipline; it depends upon clearly defined, deterministic principles. Software, systems, integration and their related services however define their own realities in entirely new contexts – assessment of their value is nearly always an exercise in subjectivity.

The larger the Acquisition, generally the more complex and subjective it becomes...

Performance-Based Contracting
By the 1990’s, it became apparent that the previous approaches to acquisition were no longer suited to the new environment. A series of reforms were undertaken by the Clinton administration to streamline processes, streamline the workforce and rethink the basic assumptions. Yet, problems remained. And then at the end of the decade a new philosophy was posited, one designed to deal with the subjectivity of modern day acquisition. When one examines the seven basic steps of Performance-Based Contracting a number of realizations become apparent, including:

• It is not materials focused
• It represents a lifecycle process; in a sense it is advocating that acquisition and development lifecycles are part of the same lifecycle.
• It forces the government to ask the question; what is that we’re really buying and why? This may sound like an obvious question but believe it or not, sometimes it is never answered properly or even asked.
• It attempts to superimpose some level of objective measurement atop areas that are notorious for their subjectivity. The Performance Work Statement is the most obvious manifestation of that.
• It acknowledges that from this point forward, the acquisition community must become active participants in the programs they fund – they are no longer limited to managing the dollars, or contracts or paperwork. Acquisition is now a vested partner in the process.

All of these developments associated with Performance-Based Contracting are positive and represent a move in the right direction. Unfortunately though, we’ve only come half-way to where we need to be.  For example, how do the other 6 steps ensure that we know we’ve selected the “right contractor?” Industry research may not be pertinent to the nature of the RFP you’re presenting. Worse still, most source selection processes have become so top heavy and burdensome that the only way an acquisition can deal with them is by arbitrarily limiting the number of participants; more or less ensuring that the same set of contractors compete for all of the business regardless of their previous performance (which is still not entirely clear due to problems with defining measures and political pressure).

Capability-Driven Acquisition
So what’s the answer? How do we get from here to where we need to be? Like most things, the answer begins at a philosophical level and all good philosophies begin with a question; ours is this - if we are measuring performance, what exactly is it performance of? Are we measuring the ability to solve a problem (as perhaps hinted by step 2) or is the problem resolution designed to provide something else? Put another way, do we want to judge a contractor by their effort, by their knowledge at a point in time or by what they produce for us? If they do produce something, will it be relevant or address what we had actually requested?

Quite a lot of questions; quite a bit of subjectivity if our primary focus is only performance, isn’t there. The truth is that a performance-only based paradigm will never work precisely because it stimulates an even greater level of uncertainty than currently exists if all things are measured only in this dimension. There needs to be a stable foundation to build upon before we can tackle and manage the uncertainty and complexity associated with modern acquisition. That foundation is an understanding of the expectations surrounding anticipated capability. By focusing on capability we completely redirect the attention and efforts of the acquisition community and we provide them hard targets for defining and managing performance. 

Capability implies an outcome. Capability can apply equally to hardware, software, integration or services; all of these elements imply use that will lead to practical application. This means that all performance must necessarily be directed to the delivery of capability. The capability becomes the primary criteria for acceptance, not knowledge, not effort, not pieces of work which cannot on their own accomplish a function, but only the resulting outcome-based test demonstrating a capability. So, for our previous materials management example with the hammer, that hammer would be stress tested with however many blows needed to show that it wouldn’t fall apart. For something more subjective, like software, the software must meet the exact capability and performance expectations specified in the requirements. If it is must manage 247 data sources, totaling as many as 400 million records with reports or queries that are returned in less than 2.5 seconds then the capability must be demonstrated and validated.

Now, this leads to some obvious questions. How does one know whether the program is working towards that effective capability or simply burning money? This is where performance measures are supposed to help, but again if those measures only track project progress in its own context (i.e. milestone’s A & B are complete), what does that really tell us about capability? Does it solve our problem – what is a milestone, is it a marker for how much resources were expended, how much code was developed or does it perhaps represent how much of the capability has been achieved? It has to be the latter to matter – and this is precisely the premise for most approaches to what is referred to as ‘agile’ development. For this to succeed, projects or programs cannot simply be divided across work breakdown units tracking progression across time and space. The project must determine up front how elements of capability can be subdivided and build the development structure around those so that each element can be released and demonstrated in turn. Progress must be tangible to ensure project success and reduce the risk typically associated with complex endeavors.

So how does one define a capability, let alone subdivided it; requirements, requirements, requirements? Requirements engineering is the most underestimated, undervalued activity in IT. Acquisition professionals have some idea of the importance of requirements engineering and management but still have not really grasped the most fundamental truth of acquisition – an acquisition is only as effective, will only ever be as effective as the requirements associated with it. And capability is expressed, chiefly, through requirements, both functional and technical requirements. This implies that the most important stage of any project or acquisition process is the beginning. The requirements provide the basis for all performance measurement, for capability demonstration and for the entire scope of acquisition.

If we were to develop seven steps for capability driven acquisition; it might look like this:

• Define a scenario in terms of the problem or challenge and the capability needed to resolve
• Validate the scenario
• Develop an acquisition strategy, determine capabilities
• Expand the scenario – build detailed requirements
• Build a scenario-based RFP, let the contractors describe how they will solve this problem; how they will meet the desired capability test.
• Build performance measures & project oversight based upon the detailed requirements, continue to validate requirements & performance throughout the lifecycle
• Continually demonstrate capability through the final iteration to achieve acceptance.

Capability-Driven Acquisition removes much of the ambiguity that is revolving around the current approaches. It provides a paradigm designed specifically to cut through the layer of subjective confusion that plagues us today. Either the acquisition passes its demonstrations or it doesn’t. A program that fails to pass its capability demonstrations is ripe for cancellation or restructuring – it should be obvious much earlier that it will fail, why wait until all of the money is spent? The application of Capability Driven Acquisition will likely involve new tools, techniques and methodologies – however if they logically flow from the core premise they should remain relatively straightforward. A first step might be the Capability Work Statement…

Copyright 2012, Semantech Inc. All rights Reserved 

Read More

Capability-Based Enterprise Architecture

Capability and performance; one logically precedes the other, if either is missing they both become meaningless. There has been a tremendous emphasis in recent years on the ability to measure and track performance – this has led to many positive improvements in the approaches to portfolio management, governance and accountability. However, when these efforts are discussed it feels as though we’re only getting it half right; if we artificially separate the two most important aspects of any IT project why should we be surprised if the outcomes still aren’t predictable ? IT projects as compared with other business endeavors tend to suffer from a much higher ratio of unpredictability and the risk of failure tends to increase in proportion to the relative complexity of the project. While there is no silver bullet to change this situation, improved outcomes are possible if we re-examine some of the core assumptions we tend to start with on any given IT project. There is also a set of actionable techniques to make this happen, something that could be referred to as Capability-Based Architecture. Why the emphasis on capability, because it is our logical starting point…

Enterprise Architecture (EA) is a very powerful tool, one that has yet to realize its full potential. Too often it has been applied to parts of the overall IT oversight process but seldom if ever to all of it. That is precisely what needs to happen; enterprise architecture can and should become the central organizing mechanism for management of IT capability, and not simply be employed as a mechanism to facilitate governance or design. What has been missing in this equation is some sort of methodological context that would allow for use of EA as a medium to both visualize all the associated concepts and track the pertinent data. The focus on capability management provides a suitable framework and blends very well into efforts already in place to manage performance. Let’s take a closer a look at several typical IT project assumptions and how employing a capability based architecture methodology could change our perspective.

A DODAF view of how Capabilities can be represented by EA...

 Assumption 1 – The RFP = Requirements

While many projects are different in that some are overwhelmingly dedicated to developing new (unfielded) capabilities and others are dedicated to the provision of services for existing capabilities, ultimately every IT project starts with a capability expectation. The problem with an RFP being requirements focused is that often as requirements change the core capabilities associated with them change or become distorted – this is the dreaded ‘expectation disconnect’ that tends to wipe out many large-scale projects. Each and every RFP that goes out should contain a capability expectation description and this should map directly into a capability contract. The main idea here is that specific requirements will roll up to the capabilities but it is understood from the beginning that under no circumstances will the capability expectation change, requirements can remain flexible as long as they do not exceed or underachieve the agreed upon expectations.

To combat this assumption, each project can model as the very first step in an expanded EA process the set of solution capabilities they are seeking to achieve and then drill down to “requirements” level objects. At the early stages these requirements objects would likely be equivalent to business or functional requirements. Any capability has a matching set of functions which can be described using requirements objects, this can be visualized as well as verbalized to help ensure understanding of project scope and expectations. Depending on the extent of integration an organization has achieved with its EA process, all capability sets can be derived from business goals & objectives and mapped to projects and project elements. The visual roadmap can scale up through the enterprise or down to the lowest level of development. 

Assumption 2 – It is difficult to determine performance before a solution has been fielded

If we follow the premise that all aspects of a project branch out from the initial capability expectations, then it stands to reason that capability expectations can drive definition of performance expectations before project launch. The key of course is to explicitly link them in the RFP or statement of work. The reason most people find metrics definition and performance work statements difficult to build is that they lack the solid foundation that a capability map and contract would provide them.

This assumption can be addressed through the capability based EA by creating Key Performance Indicators, Measure of Effectiveness or similar objects mapped to each capability. Each set of requirements under a particular capability object hierarchy would then have starting point for more specific measures. This can later be extended as the project progresses as long as it is clearly understood which measures are contractually binding are which aren’t (i.e. additional tracking of project or capability health may lead to metrics not originally considered but these would not be tied to service level penalties if not met). Of course, this highlights the importance of determining most of this conceptual framework up front, whatever is not defined and made legally binding becomes a risk.  

Assumption 3 – The Project and Capability are Separate

Many if not most IT decision makers tend to view the anticipated solution or delivery of capability as largely separate from the project which produced it. This seems a logical enough usually with capability only emerging towards the end of a project in milestones sometimes referred to as Initial Operating Capability (IOC) and Full Operational Capability (FOC). This leads to several problems starting with the inability to create a linkage from the larger set of enterprise goals and objectives to the project which is in reality solely defined by a subset of capabilities mapped to related enterprise goals. The other major problem that arises is that the project management itself loses its capability based context when schedules and work efforts are viewed out of the context that makes them relevant. The project is the facilitating medium for the evolution or emergence of specific enterprise capabilities, as such everything that occurs during the project itself necessarily impacts the outcome of those capabilities.

This basic premise has a number of common sense implications:

·       If the project isolates itself from the leadership who determined the initial linkage of goals to objectives to capabilities it is likely that there may be gaps in interpretation.

·       If the project isolates itself from the intended user community of the capabilities, they will have a difficult time mapping those capabilities to lower level design requirements.

·       The project thus represents both a lifecycle and communications continuum. The capability lifecycle begins at conceptual definition and continues through capability retirement – the development / deployment project becomes the primary means of linking all of the lifecycle elements and communities together.

·       All aspects of any IT project can be visualized or modeled using the same framework applied to the solution architecture (preferably within a larger enterprise context), and in fact this integration of project and solution is perhaps the most effective way to reduce uncertainty in complex IT projects. Existing technologies can provide the ability to merge data from various tools such as MS project or EA modeling tools allowing for comprehensive tracking and analysis of capability evolution.    

Assumption 4 – New Capability = New System

Too often in IT simple conceptual roadblocks tend to cause larger practical problems, some of which become so complex that it becomes difficult to untangle what went wrong. One of the conceptual roadblocks that is particularly dangerous is the notion that any new capability requires the development, procurement or major modification of a system. This problem translates even into several of the popular paradigms use for EA modeling. If decision makers followed a capability based approach this roadblock could be bypassed in many cases. The capability does not distinguish between the relative options which may allow for its actualization. In other words, new capabilities can be derived from existing technologies or systems which simply need to be re-purposed. The extent or scope of the work involved in repurposing can vary widely, however it is almost in every case smaller than the scope of work needed to generate a new capability from ground zero as a new system (be that as software or through integration of various system elements).

There are a number of examples of how this is emerging right now with web technologies:

• Podcasting leverages the RSS standard and available media formats to create a new audio delivery capability.
• Blogging is a variation of several web publishing technologies.
• Wikis & Commons are variations on previous collaboration and content management web technologies.

How does one build in the flexibility to determine whether a capability should manifest itself as a new system or whether it can leverage existing organizational assets & technology ? This can all be modeled as part of the capability based architecture approach. Each capability can display its own set of alternatives (objects) which in turn can display hierarchies of detail per the needs of the organization covering aspects such as risk, cost, complexity and assets available for reuse. In essence an Analysis of Alternatives (AoA) should occur for every capability-based process. While most view an AoA as an obscure engineering exercise it is in reality perhaps the single most important yet underutilized project management technique available to decision makers.

What has been described here doesn’t in most cases require significant new investments in technology or expertise. Capability based architecture and management is a conceptual framework for mitigating complexity – it accomplishes this by allowing any sized project to keep a clear focus on the core elements which remain constant throughout a lifecycle that extends beyond the project itself. Once this approach is adopted, a ‘capability roadmap’ can emerge, linking all enterprise projects and all stakeholders.     

Copyright 2012, Semantech Inc. All rights Reserved

Read More