The Data Architect's Manifesto

Its easy to find pithy and authoratitive exhortations to a data-driven design approach.

Data dominates.  If you've chosen the right data structures and organized things well, the algorithms will almost always be self­evident.  Data structures, not algorithms, are central to programming

Pike R (1989), Notes on Progamming in C

Representation Is the Essence of Programming

Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowcharts; they'll be obvious

Brooks F P Jr (1975),  The Mythical Man Month, Addison Wesley

Smart data structures and dumb code works a lot better than the other way around.

 Eric S. Raymond (1999). The Cathedral & the Bazaar. O'Reilly

But we need to get past the subjective measures ("right", "organized", or "obvious", "smart" or "dumb") to get past the endless circular debates about data, and data quality.  Dave suggested some less subjective measures,  which he expressed in his Data Architect's Manifesto:


The success of an information system is dependent on the following qualities of its data:

1 Completeness
2 Correctness
3 Clear Shared Meaning
4 Conciseness

5 Adaptability

One of the key practical lessons Dave taught me was that you can organize the presentation of a data model in such a way that if you know how to read it these qualities are pretty much revealed both to you and, with a little effort, your business.

In the next post, I'll begin exploring how this is done.

The Matisse of Data Modelling

[ These ] miracles of pure line ... beguile us and take our breath away at Matisse’s sheer virtuosity in making a simple line evoke the complexities of space and form.

Christian Zervos (1936), Henri Matiise: Drawings 1936, Thames & Hudson 

I am  fortunate to have worked alongside some very good software architects. The best show refined instincts for designing solutions to complex and seemingly intractable problems with apparent ease. These solutions seem natural, with well defined components inter-operating predictably. They are readily adapted to a changing environment. The same practitioners also show skill in immediately locating exactly how and where other solutions fail.

Between 2005 - 2008 I led a database team during a period of frenetic change. One of the first recruits was an experienced contractor called Dave. Dave had learned data-centric development while working under Richard Barker (author of the Oracle Case Method) at Oracle. He effortlessly and repeatedly absorbed the awkward and incomplete problem specifications, producing blueprints that both described and addressed the problem more accurately, more completely, and infinitely more plainly than its author or sponsor had begun to articulate it. And with infinitely more clarity than any accompanying documentation.

His designs were complete, correct and concise. They were rarely substantially re-drafted, largely proofed against challenge, and almost magically adaptable to "new" requirements. The resulting software was delivered on time, and worked. The designs and resulting software seemed to define the essence of the subject quite completely. To my eye they were done with more than a degree of artistry.

I was forced to accept that Dave had perfected a practical art-form in our professional domain, an art-form in which I was no more than a neophyte, and in which it was clear most of our peers lacked both skill or even interest.

I was compelled to learn how this was done. I don't believe there is a process to follow. Rather, as fortune favours the prepared mind, there are concepts and techniques which prepare a data architect's mind to understand, design and communicate information problems and solutions.

What follows, will I hope, be a formalisation of what I learned: from Dave's zen-mastery of this art, and from the other expert practioners I have been privileged to work with.