Agile Data Architecture

Traditional data architectures are not by nature Agile. It may be time for us to reconsider some of our previous assumptions regarding data integration. For decades, we've presumed that the best method for managing data was through strict conformance and control – we viewed the enterprise as static, remaining stable once properly defined. What we've discovered is that quite the opposite is true. Today we are facing more complexity in data integration than ever before; more data sources, greater volumes of data, more solution paradigms to deal with and greater expectations for Cross Domain data exchange. Moreover, data integration has become the linchpin within holistic architectures based upon services and sophisticated business process orchestration.

Agile Data Architecture, defined

Agile Data represents the ability to quickly retrieve and dynamically manage data from any number of data sources - this also implies less development up front and more evolution as the solution matures. 

Agile Data Architecture represents the technologies and patterns used to facilitate that management; it builds upon the core premise and provides the combination of a dynamic set of inter-related best practices rather than a standardized or single architectural approach. Much of the strength behind this approach is based upon the ability of this philosophy to accommodate evolving technologies and architectural best practices. 

Agile Data Architecture is not a vendor-focused solution; it is a generally technology agnostic (although not necessarily standards agnostic) philosophy designed to address the new realities of enterprise integration. It adhere's to several core Agile principles including:

• The focus on rapid development and deployment of capability  

• User focused requirements

• The recognition of the evolutionary nature of most solutions 

An important concept within the larger Agile Data construct is the Dynamic Data Model. This is in fact an abstraction layer for integration that resides in Semantic formats – in other words OWL (Web Ontology Language) and RDF (Resource Description Framework). The key to making this concept work is not attempting to dynamically modify RDBMS technology utilizing previous modeling paradigms, but rather the ability to map Semantic data protocols to existing or new data structures, thus insulating them from system changes and allowing for automated updates.

This construct supports much of what we're talking about in other posts when we mention

• Semantic Integration - This provides a solution methodology that can be applied across a range of Agile Data solutions
• Intelligent Healthcare - This provides an industry specific application for Agile Data Architecture using Semantic Integration as a methodology.
• Semantic Common Operating Picture - an instantiation of Agile Data Architecture for Situation Awareness using Semantic Integration as a methodology.

While it is not specifically focused on Big Data technologies Agile Data Architecture certainly accommodates through its approach to management of heterogeneous data sources. We'll explore how Agile Data Architecture supports a wide variety of solutions in upcoming posts...

Copyright 2012  - Technovation Talks, Semantech Inc