The Data Architect's Manifesto

2011-06-18T15:01:00.000-07:00

Its easy to find pithy and authoratitive exhortations to a data-driven design approach.

Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be selfevident. Data structures, not algorithms, are central to programming

Pike R (1989), Notes on Progamming in C

Representation Is the Essence of Programming

Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowcharts; they'll be obvious

Brooks F P Jr (1975), The Mythical Man Month, Addison Wesley

Smart data structures and dumb code works a lot better than the other way around.

Eric S. Raymond (1999). The Cathedral & the Bazaar. O'Reilly

There's a lot to gain by progressing beyond qualities such as "right", "organized", or "obvious", "smart" or "dumb". Dave calls the formalisation of these qualities the Data Architect's Manifesto. In my mind, the manifesto enumerates the mental alarm bells I aim to prime before reviewing or constructing a data model.

The success of an information system is dependent on the following qualities of its input, output, intra-component and even internal data structures; their relationships; and the defining domain data:

1 Completeness
2 Correctness
3 Clear Shared Meaning
4 Conciseness

5 Malleability

One of the key practical lessons Dave taught me was that you can organize the presentation of a data model in such a way that if you know how to read it these qualities are pretty much revealed to you.

In the next post, I'll begin exploring how this is done.

The Matisse of Data Modelling

2011-06-13T09:12:00.000-07:00

[ These ] miracles of pure line ... beguile us and take our breath away at Matisse’s sheer virtuosity in making a simple line evoke the complexities of space and form.

Christian Zervos (1936), Henri Matiise: Drawings 1936, Thames & Hudson

I am fortunate to have worked alongside some very good software architects. The best show refined instincts for designing solutions to complex and seemingly intractable problems with apparent ease. These solutions seem natural, with well defined components inter-operating predictably. They are readily adapted to a changing environment. The same practitioners also show skill in immediately locating exactly how and where other solutions fail.

Between 2005 - 2008 I led a database team during a period of frenetic change. One of the first recruits was an experienced contractor called Dave. Dave had learned data-centric development while working under Richard Barker (author of the Oracle Case Method) at Oracle. He effortlessly and repeatedly absorbed the awkward and incomplete problem specifications, producing blueprints that both described and addressed the problem more accurately, more completely, and infinitely more plainly than its author or sponsor had begun to articulate it. And with infinitely more clarity than any accompanying documentation.

His designs were complete, correct and concise. They were rarely substantially re-drafted, largely proofed against challenge, and almost magically adaptable to "new" requirements. The resulting software was delivered on time, and worked. The designs and resulting software seemed to define the essence of the subject quite completely. To my eye they were done with more than a degree of artistry.

I was forced to accept that Dave had perfected a practical art-form in our professional domain, an art-form in which I was no more than a neophyte, and in which it was clear most of our peers lacked both skill or even interest.

I was compelled to learn how this was done. I don't believe there is a process to follow. Rather, as fortune favours the prepared mind, there are concepts and techniques which prepare a data architect's mind to understand, design and communicate information problems and solutions.

What follows, will I hope, be a formalisation of what I learned: from Dave's zen-mastery of this art, and from the other expert practioners I have been privileged to work with.

Simon Grabinar

The Data Architect's Manifesto

The Matisse of Data Modelling