Thoughts on CouchConf: Data Modeling for Flow

Last Friday, I had the pleasure of attending CouchConf in San Francisco to learn about the new and exciting features of Couchbase 2.0. With the new release, Couchbase becomes a full document oriented database . Meanwhile, Couchbase keeps its focus on performance, reporting some very impressive performance statistics comparing Couchbase to other NoSQL databases. Of all the exciting presentations at the conference, I most enjoyed the session on data modeling by Chris Anderson, Chief Architect for Mobile at Couchbase.

Document-oriented databases such as Couchbase are schemaless, making data modeling more flexible. While in the relational world there is generally one right answer for how to divide data across tables to achieve fifth normal form, in the document-oriented world there are fewer clear cut rules. For example, whether blog comments get stored with the post or in separate documents depends more on expected usage than any inherent constraints of the system or rigid design rules.

Without the schema constraints of the relational model, developers become free to think more about data flow. Chris Anderson uses phrases such as “emergent schema” and “runtime-driven schema” to describe this new emphasis. He concludes “thinking about throughput, latency, update and read patterns is the new data modeling.”

In my experience, emphasizing data flow, or data in motion as opposed to data in rest, makes developers better at understanding user needs. In my early career, when playing the role of business analyst, I went too far in allowing the relational model to shape my thinking about business requirements. With a relational schema always in the back of my mind, I focused my questions on whether data relations were one-to-one, one-to-many, or many-to-many. While I did write use cases, normalization rules shaped my thoughts.

Then several years ago, I worked with a colleague who focused more on process. He’d start conversations with “…now walk me through your process.” And that led us to a more fruitful discovery. At some point, I’d ask questions to better understand the data model, but we were in a better place when I started to ask those questions.

Of course, developers have always thought about use cases and update and read patterns. But if Chris Anderson is right and document databases make it easier to focus on data in motion rather than data at rest, then I think we’ll do a better job of thinking about user needs.