Architecting for Focus, Flow, and Joy: beyond the Unicorn Project



Kim: I’ve had the privilege of studying high performing technology organizations since 1999. That was a journey that started for me back when I was a CTO and founder of a company called Tripwire in the information security and compliance space. Our goal was to study this amazing organization that had the best project duty and performance, the best operational stability and reliability, as well as the best posture of security and compliance. As you can imagine, in a 21 year journey, there were many surprises. By far, the biggest surprise was how it took me into the middle of the DevOps movement, which I think is urgent and important. In that journey, a particularly huge surprise, was my learning about the importance of architecture. That architecture was one of the top predictors of performance, even larger of a predictor than even continuous delivery.

I think what is so amazing is that, it is architecture that really dictates to what extent teams can work independently. To what extent can they develop, test, and deploy value to their customers without being coupled to potentially scores of other teams? I know that architecture is a particularly favorite topic here within the QCon community. When Randy Shoup asked me if I’d be willing to chair a panel on the subject, I jumped at the chance, because I wanted to explore these topics more deeply with two people whose achievements I deeply admire, Carin Meier and Mike Nygard.


Could each of you briefly introduce yourselves? Tell us what you’re working on these days?

Meier: I am a data engineer right now at Reify Health. I’m also the author of, “Living Clojure.” I’m very interested lately in the Clojure data science, machine learning areas.

Kim: It’s now leveraging the Python amazing ecosystem of libraries. Mike, how about you?

Nygard: I am currently SVP of platform and architecture at Sabre, where I’m helping this company through a digital transformation and turnaround. The subject of architecture and coupling is very much on my mind every single day, at Sabre. I’ve also been a developer for a long time. In fact, Carin and I were colleagues at Cognitect, where we were both working in Clojure. We got to partner on a pretty amazing project, building a desktop IDE for building games using Clojure, which is not exactly considered the sweet spot for Clojure. It worked exceedingly well in that situation.

Kim: I saw that video. The name of that editor is called Defold.

When Coding Was Most, and Least Fun

We’re going to talk about what productivity, focus flow, and joy look like in the large, but maybe we could talk about first in the small, at the individual level. Could you both talk about when coding was most fun for you? Maybe even let’s explore the opposite, when was it also the least fun for you? Maybe talk about the factors of what made it so fun and what specific factors made it so not fun?

Meier: I’ve been doing programming for quite a while. It became really fun once I started coding in Clojure. I think that is a lot due to the nature of the language, and how fast you can get feedback, and how close you feel to the code. You can really interact with it and almost sculpt this, and the data. That really makes it a pleasure to work with during the day. It’s very productive too, it gets rid of all the boilerplate and you can just focus on the real problems. When I’m working with Clojure, is one factor. I also like to explore and do a little bit of research. That really attracts me to the data science, machine learning realm because there’s always something new to explore, and novel applications that can help businesses and just people.

Kim: In fact, one of the things I really admire about your work is it’s so clear that you’re really on the frontier of some of these problem domains in terms of solutions that are being developed across a very wide surface area of technologies. Am I overstating the case?

Meier: I do like to read the latest papers that come out, and old papers too. It’s always nice to see, what problem could that solve? It’s a new way of thinking about it and this new tool could open up possibilities that we haven’t thought about before. That’s really exciting to me.

Kim: I just want to concretize some of the things I heard. Some of the things I heard was fast feedback in your work. You get really fast feedback in terms of whether something is working or not. I know in one of our conversations, you said just how miserable you are when it takes a long time to get feedback. By long feedback cycles, you were talking about minutes, not hours, not days, the fact that it even took minutes. Am I recalling that correctly, Carin?

Meier: Yes. I get a little down when I have to do my Docker builds and deploys, that takes a little long. It’s a far greater cause building out your pipeline, but I much prefer to meet my REPL.

Kim: Creating immutable build artifacts. That’s important. It does amaze me that when you have 4, 5, 6, 7 minute build times that actually does suck the joy out of our work.

Mike, does that resonate with you? Tell us about specific times when you were having just a lot of fun, and versus not fun?

Nygard: First of all, waiting on a Docker build or anything else that’s over that one minute threshold, like it’s time enough to flip away and go look at social media and get angry and stressed out and depressed. It forces you off task. The times I’ve enjoyed most have been the ones like the project I worked on with Carin, aside from the fact she’s just a joy to work with. It was a project that we were working on all the time. We were focused on it, really focused. The previous projects I’ve enjoyed most, there was a project for Photo Studios, where we were in a full XP lab situation. We controlled our equipment. We had our OS builds, they were ours. We controlled our build infrastructure. It’s not that we were control freaks so much, it’s just that it allowed us to cut off all those external dependencies, and really focus on the work itself. There were times when we would joke about whether this was a one can of Pringles problem or a two can of Pringles problem. What it really meant was, is this something we’re going to get done in a morning, in the short block? Is it an afternoon, or is it an afternoon and an evening? Sometimes you’d get into that state of flow, and suddenly realize it was a two can of Pringles problem, because it was 9:00, and that was dinner. Not great on the health front, but it was a really enjoyable experience because we were able to work together, work closely, and create everything we needed to create. I think that’s one of the ideals you wrote about around locality. We really had that. Those are the times that I’ve enjoyed the most.

Kim: Locality was really meant to suggest, to what extent can an individual or team do what they need to do by themselves, without having to open up tickets with potentially 15, 20 different teams? Mike, you had shared a story about your least fun moment. Can you describe what that looked like, and what are the factors that made that so frustrating and miserable?

Nygard: I don’t even remember which example I used before.

Kim: Let me remind you. It was a floor with 300 people on it, tables.

Nygard: I figured it was either that or one other. No, it was a vast project. It was late and getting later, and so their solution to solving the late project was to add another 100 people. They had emptied out a floor of a bank and had rows of folding tables with all of us in there. The team was strictly organized by layers of the application, so we had a persistence team that had no idea what the domain needed. We had a domain team that had no idea what the users needed. We had a presentation layer team that also didn’t know what the users needed. We were all trying to model the world while also managing these vertical dependencies that were just changing all the time. It was a tough slog.

Kim: Just to, again, concretize that. The fact that you had these teams organized, whether it was technology, it meant that anything that you need to implement from the user’s perspective, from the customers’ perspective was now smeared across those teams, and that involved communication, coordination, prioritization, sequencing. These are the things that suck.

Nygard: Exactly. Big documents, lots of meetings.

The Intolerable 4-Minute Build Times

Kim: I’ll be the first…


Read More:Architecting for Focus, Flow, and Joy: beyond the Unicorn Project