IT Architecture for the Enterprise

Semantech: IT Architecture in the Enterprise from Stephen Lahanas

Read More

When to Create Conceptual Data Models

Why are Conceptual Data Models Important? Not everyone does them - especially in situations where many aspects of the data architecture has already been defined by default because commercial software is involved. Also, those folks who do use them don't always build the same way. Yet - perhaps there is more to the Conceptual Data Model (CDM) than many people realize. Moreover - how does the CDM fit into the larger realm of architecture or modeling activities that are likely to take place on typical IT project? First let's begin with an official - or at least semi-official definition...

The primary goal of the CDM is determine entities & the high level relationships

Conceptual Data Model

Describes the semantics of a domain (the scope of the model). For example, it may be a model of the interest area of an organization or of an industry. This consists of entity classes, representing kinds of things of significance in the domain, and relationships assertions about associations between pairs of entity classes. A conceptual schema specifies the kinds of facts or propositions that can be expressed using the model. In that sense, it defines the allowed expressions in an artificial "language" with a scope that is limited by the scope of the model. Simply described, a conceptual mode is the first step in organizing the data requirements. (Wikipedia)

Well, in this Wikipedia definition (there are several) they actually called it a 'schema' instead of a model and we don't think that's quite accurate - so we changed it here. The schema implies that the model is being built to a specific DB technology whereas a Conceptual Model is wholly agnostic. In fact, a CDM is a lot like a UML Domain Model and that's a good place to talk about the big picture nature of Conceptual Modeling:

• UML Domain Models represent the Conceptual Classes of an application-focused solution or those classes can be used to model database structures if the modeler doesn't wish to capture ERD type relationships (although many data modelers do).
• A CDM is an ERD construct - the highest level of relational database design. Although, it is possible to use both Domain Models and CDMs to capture Big Data structures as well (we'll talk about that in a future post).
• Entity Relationship Diagram (ERD) is a specialized graphic that illustrates the relationships between entities in a database. ER diagrams often use symbols to represent three different types of information. Boxes are commonly used to represent entities. Diamonds are normally used to represent relationships and ovals are used to represent attributes. 
• Conceptual Modeling in general is built into most architecture exercises and Data is always one of the dimensions captured (just not in the same way across all EA approaches).

UML Domain Models can be fairly detailed (and in fact may represent much of what will be become the Class diagrams)

There is no set method to going about this. One could use a UML Domain Model and then Class Diagrams to be their Conceptual and Logical Design before they move to the Database or one could also define a CDM and a Domain Model and Class Diagrams and then a more detailed Logical Data Model (then generate the Database schema from the Logical Model).

Another interesting consideration for the CDM is its relation to Master Data Management (MDM). Since the CDM is focused on exploring what the key data entities in the solution are up front it becomes the first and perhaps the most important reference for determining which of those entities ought to be managed as Master Data.  

Now, with this background, this is our recommendation of when to use Conceptual Models:

1. Use a CDM when you need a better understanding of the data behind Data Flow diagrams or other business process models or analysis.
2. Use a CDM if you are sure you will be employing any sort of MDM-related capability.
3. Use a CDM if you intend to use database modeling tools to generate the DDL for your DB schemas from a Logical Model. 
4. Use a CDM for sure if you're not using UML or another EA framework to capture data definition /models. 
5. Use a CDM if you want to understand early in the process where integration may need to occur and whether you will need to establish data interfaces (and if you need to examine data in advance that you may not own).

Our next post will explore how to build CDMs.

Copyright 2012  - Technovation Talks, Semantech Inc

Read More

Understanding Master Data Management

I was speaking with recently on the topic of MDM - it occurred to me not too long after we began the conversation that we more than likely had differing perspectives as to what Master Data Management meant. That's inspired me to write this post to talk a little bit about how better to understand MDM.

MDM, The Core Concept:
Let's start with what Master Data is not, Master data is not:

1. Meta-data, which is a description  of data  (or data about data as it's commonly referred to as).
2. Ontology,  Taxonomy or Vocabulary - Master data can be derived from these but is not in itself a formal semantic construct.
3. Software Tool - ultimately, Master Data is technology-agnostic; it is a logical construct which can be defined through various modeling tools and realized through a variety of data management software solutions.  At the point where Master Data becomes tightly coupled with any one software tool or any one modeling technique it will likely loose a great deal of its potential value to the enterprise.

So, then what is it? How would we characterize what can become Master Data or not ?

1. It may be considered "data of record" or an authoritative data source, but it might not be also. Data of record implies that there is a system of record with sanctioned data elements that are not meant to be repeated throughout the enterprise across other systems. Or this might refer to data entities which are determined to be unique and authoritative across the enterprise regardless of their current use (in a system).
2. Master data is reference data, sort of. If we consider that reference data is a definitive set of element definitions or entities associated with any particular business domain, sub-domain or problem space. In this capacity, Master Data may serve multiple roles, including: discovery, registry or repository access, data dictionary foundation.
3. Benchmark - this is a critical consideration; any data entities defined as Master Data elements within an enterprise are unlikely to remain unchanged or unmodified. Eventually there will be variations of Master Data sets, these variations must be tracked back to their source and there must also be a mechanism whereby others in the enterprise can understand where, why and how those modifications occurred 'atop' the core sets of Master Data. Thus the Master Data is a baseline or benchmark wherein the data chain of custody can be managed or tracked. 
4. It can be a canonical data model or data exchange model - this is important in cases where the core data architecture has not yet been designed or deployed, or in cases where it is anticipated that there will be a major or radical transformation of the existing architecture to a new one. The model can contain Master Data elements or sets within it.

MDM in Today's Implementations
Much of what we refer to now as MDM solutions have been borne out of previous product solutions that were describe as meta-data management solutions. For many of the MDM solutions on the market, the "repository or registry" architectural construct / pattern is how this capability is harnessed.  Another architectural approach related to MDM might be referred to as the middleware design - this extends MDM into data transport and is focused on supporting accurate message translation. And of course there are solutions that combine both aspects.

One of the most important aspects of MDM is identifying what constitutes Master and Reference Data

Perhaps we can consider that there are at least two philosophical approaches to MDM:

• Passive MDM - This is most closely aligned to the original Meta-data management solutions with a central repository to support discovery and high level data reconciliation.
• Active MDM - This is most closely aligned with solutions stemming from EAI, Middleware, ETL based solutions where data reconciliation rules are being applied at multiple levels and in more detail.
• Hybrid MDM - Both solutions are relatively weak in dealing with bi-directional reconciliation focused heavily on transactional systems (it is much easier to reconcile historical data from multiple sources than real-time data from multiple sources).  Hybrid MDM applies both previous techniques and new ones to tackle the most problematic use cases.

It is clear to anyone who has worked with database development and data-system integration that having the ability to reconcile data sources adds tremendous value to the enterprise helping to improve performance, integrity and overall efficiency. Being able to do some of this automation using COTS tools is even more appealing, however there is still a set of processes which ultimately takes precedence here if one is to deploy a successful MDM solution. The enterprise data governance approach must be defined first, the data and business environment must be modeled and if ownership of data sets is to be handed off to user groups (either fully or partially) the impact to both governance and model maintenance must be considered and mitigated in advance.

As we've discovered with nearly every IT technology and product over the past 40 years - implementation without process or architectural considerations leads to many issues, often more issues than existed before the technology was introduced.  This is no exception with MDM - the most important thing to consider here is that deployment of MDM software can significantly impact or influence both solution performance and integrity but proceeding without working through the implicit architectural / enterprise issues is risky.

Copyright 2012  - Technovation Talks, Semantech Inc.

Read More