← All talks
Build Turing Fest 2021

Conor Winders - Scaling Product Engineering Teams for Success

Conor Winders

Talk transcription

Conor Winders:

Good morning, and thank you. So this talk is going to be about my experience in building a software company, and scaling multiple product engineering teams over the years. I hope that some of the takeaways you'll get from this will be principles that can be very effective in how to focus your teams on building great products, how to design good team structures, how to make sure that you're constantly leveling up your team, and then some simple processes to empower that team.

Ultimately, I've learned all of this from working with incredible teams, working with incredible leaders. And I'm very lucky to say that I've worked on some of the biggest apps in the world, that literally have millions of users. Some of the biggest platforms in the world, that literally track millions of things.

But before we go much further, I should issue a little health warning here, and a little caveat. For somebody who's up here going to talk about how to scale teams and how to scale things, this was my car before I had kids, and it was the greatest car I ever had and I recommended it to everybody, and then it became the least practical car ever. So I like to think I know what I'm talking about when it comes to software, but if I give you advice on cars, I'm probably not the right person to talk to.

So who am I? My name is Connor. I am based Dublin in Ireland, and I'm the executive vice president of engineering at administrate, who are headquartered here in Edinburgh.

You should know about me that I love technology, I love software, I love tiny cars and gadgets, and I love building things. I also love whiskey, with an E, correct, which is the way we make it in Ireland. And when I'm not working, I'm coding. So yeah, I have an app called the Irish Whiskey App that I spend my spare time writing code and working on that. So as well as leading teams, I like to steal code as well.

I first learned to code on a BBC microcomputer when I was a teenager. My dad arrived home one day with one of these, and I thought it was really cool, and I really wanted to learn how to use it. Now, this thing had no hard drive, had no disc drive, had no nothing, except it came with a book of like 1000 pages of different code. So I used to spend hours every night just sitting down, transcribing instructions it into the computer, in the hope that it would work. It rarely did, but when it did, I fell in love with that process of, "I've just built something, and now I can see this animation, or I can play this game."

So that led me into software engineering in college, and from there, my first job was in IBM. Obviously IBM was a huge company, and it wasn't necessarily for me. So from that point I moved into much smaller companies. I worked in web consultancies, I worked in the telco industry for a number of years building backend systems. And then in 2008, when Apple launched the iPhone, that was a bit of a watershed moment for me.

So I founded a company called Redwind Software. Kind of accidentally founded it. I just wanted to build an app. So myself and my best buddy and I decided we'd make a game and see what we could produce with the iPhone SDK. That was pretty early in the App Store days. So back in 2008. In November, we released our game and it took off and did pretty well, and very quickly that became a business and that became our full-time jobs. So over the years doing that, we scaled that business from the two of us who founded it, building a single game, to working with some of the biggest brands in the world and having over 20 people as part of our company. I learned a huge amount in doing that, because I really hadn't got a clue what I was doing when I started.

We got some really interesting experience. I'm a big music fan. My playing T-shirt earlier was a Bruce Springsteen T-shirt, but I'm also a huge Elvis fan. And I got the opportunity to work with Elvis Presley Enterprises in building the official Elvis Presley game. And we had multiple hits on the app store. So in 2012, we had a game that we worked on called Numerosity, which Apple called out was the best game in 2012. And we had permanent features for other apps like this one, Life in the Womb.

When I decided to move on from that business, I joined a startup in Dublin called Mobile Travel Technologies as a director of engineering. And in Mobile Travel Technologies, I was responsible for a team of about 30 engineers. And through my time there, I scaled that up to over 100. We had people all over the world, but predominantly we were based in Dublin, and we built software for the travel industry. So we built apps like the Easy Jet app, Singapore, [inaudible 00:04:37], some of the biggest, most popular airline apps in the world, where literally millions of people are using our software every week to check in, to book flights, to use boarding passes, et cetera.

That company was then acquired by a huge company called Travelport, and I decided to move on, and I joined another startup in Dublin called Fleetmatics. Fleetmatics is all about vehicle tracking and vehicle telematics. So what we would do is pull information from a vehicle every 30 seconds, send it to our platform and process it, and then provide insights and provide reporting, et cetera, and intelligence to our customers about their fleets. So our customers were mainly commercial fleets, trucks, vans, that sort of thing. We were tracking millions of vehicles every day, and taking all of their data every 30 seconds. We also got to really cool things like launch a dash cam product with chat intelligent video processing as part of it as well.

So in Fleetmatics, we were then acquired by Verizon, the company became Verizon Connect, and I grew that team from about 50 when I started to well over 300 by the time that I finished there. And my teams were largely based in Dublin, and about half of the team was also based in New Zealand. And that's a very, very interesting time zone challenge to work around. But it was something that the tricks on how to structure teams really, really helped with those time zone challenges and those location challenges.

So there's a bit of a pattern here in the sort of companies I end up in. Today I'm at Administrate, and we have a booth over there in the expo hall if you want to learn more about us. But at Administrate we are building a platform, we're building an infrastructure to allow the world's enterprises to level up through training. Administrate is still very much in a startup phase. The company has been around for a little while, but we're still grasping to prove that repeatable model. And we are going through scaling challenges as well. We are hiring, as the slide says. So if the sort of things I talk about resonate, or the mission of Administrate resonates, please do get in touch.

So there's some key principles that I hold dear when it comes to building teams and building software. I think it's worth covering those quickly, because you'll see how they weave through everything I do in terms of structuring teams and the processes that I put in place.

Product principles first. Speed. And I don't mean speed is in rushing towards a deadline or cutting corners to get things out the door. I really mean speed as in, how quickly can we get something valuable to a customer so that we can learn from it? And that leads into the second point there. Data driven decision making. I have a lot of opinions. I have a lot of very strong opinions, but I found in my career that the best decisions I make are based off data. So getting value to customers, learning from it, and making decisions based on that is critical.

And then cost efficiency. Having run my own business, having been involved in smaller companies, I've learned probably the hard way a few times. Cash is a finite resource, and we need to make sure we spend our investment dollars very, very carefully and very wisely, using the data that we have learned. And I like my teams to feel like they are all part of that. I like people in my teams to feel like they are the GMs or the CEOs of their business.

So that leads into engineering principles. And now you start to see how the product principles filter into this. Visibility. When we build something we should immediately be thinking, "How are we going to measure this, and how are we going to learn from it?" And that's visibility at all levels of the organization. From the code we write, to how the team performs, to the processes we use.

Decomposition. So dependencies are one of the great evils in software development. That's dependencies at a code level, like we can't ship this library without shipping that library, or at a team level. I can't do my job unless somebody from a different team helps me out as well. I like my teams to figure out the ways to eliminate those dependencies at a code level or at a team level.

UX might seem like a slightly unusual thing to have in an engineering principle slide, but I truly believe that the best teams, the best product teams, are customer centric and user centric. And we need to understand the role that we play in our users and our customers' lives, and how we are making them successful. Our software is typically just one small piece of what they do every day, and we should understand how we fit in and how we can make their lives better.

Ownership, comes back to the idea of cost efficiency and being GMs of the business. Everybody should feel like they own the code that they're writing, the code that they push out, and that they're going to support that code, improve that code over time.

And then automation. We should spend some of our investment dollars on automating the things that add less value to by humans being involved. So things like testing the products, building the product, shipping the product. We should invest in processes like continuous integration, continuous delivery, so that we can add value in other ways.

And then agile principles. It won't be a shock that I'm an agile advocate. And in certain teams over the years, I've run really successful scrum teams. In particular at Travelport and MTT, we had a phenomenal scrum set up there. But over the years, I've come to lo to love Kanban more and more, and really some of that is just the simplicity of it. Because software development is hard enough and complex enough without some of the abstractions that we add, and ceremonies that we add in processes like Scrum, or in methodologies like Scrum. I love the idea of what is the most important thing that I could be doing right now, and I'm going to do that. I love the idea of, what is the thing that is nearly finished? I'm going to finish that before I start something else. I love the idea of as a team having that mentality.

So engineer X has a ticket in progress, or a feature in progress. There is literally less value to engineer Y starting something else. It is much more valuable, by the definition of priority, for the team to swarm and get something done together. And I love the idea of managing work in progress, or WIP.

Work in progress is probably the single biggest productivity killer and software development. The more things you try to do at a point in time, the slower you will become. So there are the principles that I like to use to inform how I build teams. The sort of structures then that we put in place, I've learned in my career, is one of the most critical things to get right. And it all starts early on, when you have a small team where everybody does everything, everybody understands the product, roles are quite fungible, and frankly, everybody knows where the bodies are buried, because everybody's been in there from the start, together.

But as your product is successful and your company's successful, you're going to scale, you're going to hire new people, and your team is going to evolve. It's inevitable that some specialization starts to occur, because you're now hiring people with domain expertise, or you're hiring people with technical expertise who don't know where the bodies are buried. And that's perfectly fine, that's perfectly normal. And that generally leads to a structure that is very, very functional, and the idea of departments starts to appear. So it's very common. A lot of companies I've worked and talk to you have things like QA department, and engineering department. Some of them are very extreme with the engineering department where you might have a mobile department or an iOS department, an Android department, an API department, and so on.

And again, that's very common for that to happen. And there's nothing necessarily wrong with that, as long as you figure out the right way for those teams to work together. Because if you don't, you find yourself back in this dependency hell. Where as an engineer, I cannot ship my code to production unless I get time from the QA department. And they need to get time from the DevOps depart, or the deployment department. And as a UXR, I cannot impact how this product works unless I get time from the engineering department. And so it's very common to start to see these departments end up with different priorities, and very disparate. You often end up trying to optimize for utilization. So how do we keep everybody busy, as opposed to for the flow of value through your system.

And that's why I love the idea of cross-functional teams, which is nothing new, it's been around for quite a while, but it is truly, in my experience, the best way to build software. When you sit the software engineers beside the UX designers, beside the DB engineer, beside the QA automation engineer, whatever roles you need in order to build your software and ship it should be a single team. They can belong to different departments, but they should be a team. They should have common metrics, common KPIs, common goals, common priorities.

The best teams also get to own a piece of the product, and they start to identify with the product above their function. So if you talk to somebody in my last company and ask them what they did, you would likely hear an answer along the lines of, "Oh, I work on the math team," versus, "Oh, I'm a QA engineer." The best teams will start to feel that ownership over their part of the product. Cross-functional teams is a really simple, effective way to start to eliminate dependencies, and to build that ownership and autonomy.

And so this is kind of a blueprint for how you can then scale cross-functional teams outwards as the product becomes bigger and more complicated. This nomenclature of tribes, clans, squads, pods, details, et cetera, I quite like. Because it forces you immediately to rethink what you mean by team. And it also starts to foster a sense of identity and a sense of ownership, because we're now something different. This layout here is basically what a fictional team, when my whiskey app is massively successful, would look like. The squad level here is what your typical cross-functional team would look like. And that squad level owns components like microservices, microsites, and so on. They sit inside what we call a clan.

And this is where the art comes in. You have to look at your product and figure out, how can I break this down into logical domains that people can own, that people can identify with? So on a whiskey app, that might be something like distillery functionality. Might be searching, and scanning, and that sort of thing. And then that all comes together in this notion of a tribe. And the tribe should have leadership, which is your architecture, your product manager, and so on.

So when we apply this at Administrate when I started a few months ago, we immediately found we had a scalable structure to build this team out as we're successful, and as we had different needs in the product. So Administrate is a training infrastructure and training platform. But one of the things we do is help customers get on board with implementations, and integrations. And so that became a tribe. We also have learning and subscriptions, became a tribe, training and schedule became a tribe as well. But it's a very obvious, even in just the names, what these people do. So straight away, when you talk to somebody you know what they do, they know what they do, they know what they own. But even when you have something like learning and subscriptions together, you immediately see how this can scale in the future. We can have a learning tribe, we can have a subscription tribe, and this can grow independently. We also have the idea of an integrations and scale tribe, which is a little broken off from the box at the moment, because we've identified now that we need that, and we can plan towards building that, but we don't have it today.

We use this diagram, which I pretty much live by this. I'm not sure how well it comes across on a screen, but each one of the big vertical boxes is going to show you all of the tribes inside of Administrate, and it's shows you very clearly their identity, what they own, their mission, who the leaders of that tribe are, within that tribe who the clans are, and down to the component level, what everybody owns.

One of the big concerns that people have when you move to a very product focused engineering structure like this is, "Well, what about that functional expertise?" How do we make sure that our test cases are written in consistent fashion, our user stories are written in consistent fashion, how do I make sure we're using the same technologies from team to team where everybody's independent and autonomous? And that's where the idea of chapters and guilds comes in, and is very, very effective at doing that.

So a chapter would generally be a local group. So for Administrate, for example, in Edinburgh, we might have a local QA automation chapter, where the QA engineers will get together. They will decide themselves on what the standards should be, they will decide on the best practices, and then they go back to their teams, to their squads, and they're able to implement them. These are self-organizing groups, they're support groups, and then they also scale. So if we have a global company like Administrate, we have people in the US, we have people in Lebanon, local chapters come together and form guilds. And it's a very, very effective way of making sure you have standardization and best practices across your product-focused team.

So now you have your structure in place, you want to make sure that you're constantly level up your team. The first thing that every product engineering group should have is a skills matrix. It should be very clear what your levels even are. So again, this is just a really, really high level snapshot of our skills matrix at Administrate, but you can see across the top, junior engineer, intermediate, senior, all the way up to principal engineer. And then we have different aspects of the jobs that those people do. And in the matrix, we fill in what we expect of the different levels. This is incredibly effective then. And letting you understand where your seniors are, where your juniors are, constructing the right teams with the right balance, but also being able to have conversations with your team. "You need to level up in this area. You need to scale up in this area. You're already doing a great job in this area." And you have it documented, and it's a very transparent conversation. You can then pair that with things like a 9-box so that you can have calibration across your whole group and a true meritocracy.

There's lots of different ways to calibrate a team. I personally, a bit of a theme here, love the simplicity of a 9-box, which is the horizontal axis, we're going to talk about performance. So how you perform in the role that you're currently in. And the vertical axis what your potential is to move to the next role. So the way we use a tool like this is we'll have a single 9-box for every level of engineering. So let's say all of the seniors, their names will appear in a 9-box, and every quarter all of the managers will get together and justify why every single person is in the box that they're in.

That way we make sure we get rounded feedback that can go back to the engineers, they know the areas they need to level up in, they know the areas they need to improve, and they know how they're viewed across the organization. We can use that to inform performance conversations, pay conversations, promotion conversations. In Administrate we calibrated the entire organization in about a week using this tool.

And then you're obviously going to hire people in as well. You're going to hire people in, and you're going to know what you expect based on your skills matrix, but you should also be very, very clear on what you expect from every stage of your hiring process. This hiring process is pretty standard. At least the first three, you're going to have CVs getting reviewed. There you're really looking for the relevant experience and relevant locations. There's no point in hiring somebody in a location that's in an unfriendly time zone, and they're never going to get to work with a team.

You're likely going to do a code test. And don't get me wrong, I find code test to be a pain in the ass, but they're actually a great way of somebody demonstrating that they're motivated to come to your company. And so I always believe in putting code tests in place. Technical interviews should be an opportunity to meet the team and the people that you're going to work with, or the people that are likely to join your team.

At every step along this the people involved in the process should have the opportunity to say no. And it should be saying no based on, "We want to get better, not bigger." A really common mistake that people make in hiring is, "Well, we have 10 open roles. Let's just hire 10 people as quick as we can." That will make you worse. If you hire one person who makes you better, that will make you better.

And then one of the things that I really like about Administrate, and I've seen it in a couple of places, but it's not necessarily widespread. You need to understand the values that people hold dear. And so at Administrate, we have our final stage interview centered entirely around questions that drive at what a person values, and does that align to the values of our company? Our values are on the screen there. So things like transparency and truthful, built on team, and we really drill into those topics to make sure that when somebody joins, not only can they technically and functionally do the job, but they're going to be successful because their values align with ours.

So we've talked about structures, we've talked about leveling up the team. The last thing I want to go through quickly is the ceremonies that I like to put in place and that I've seen be effective. So one of the things that I like about Kanban over Scrum is I think it takes out some of the ceremonies that don't necessarily add as much value. But there are things that are still worth coming together and talking about. It's really important, again, going all the way back to the principles at the start, that we have visibility of the work at all different levels of the team. So squads, clans, cross-functional teams, you still should be having daily standups. The team I worked with in the past, they told me they didn't need standups because they worked so well together. And then the next day discovered that one of them was leaving the following week, and nobody knew. Standups are a great way of just bringing people together and making sure you're discussing what's going on, what your plans are.

The next level up, be it clan or a tribe, I like to do ceremonies in the scrum world, like scrum of scrums, where you're bringing a wider group together so people are aware of what everybody's working on. Or in a Kanban world, a ceremony called walk the wall, which is literally, let's just walk all the Kanban walls and talk about the work that's in progress, and talk about how we can get it closed out. Demos are incredibly important as well.

And then at a product level, you should have a very visible, and a very changeable roadmap. Priorities are going to change. Priorities of the business, the team, it's going to change all the time. But you need to make sure that's kept up to date and that's visible. I've found a good cadence for roadmap reviews to be every quarter. And you should always have confidence in what you're going to do in the next three months, less confidence the following six months, and so on. But it should happen, it should be reviewed, and it should be visible.

Operational reviews I find really effective as a replacement for the likes of a retro in a Scrum world, because operational reviews are based on facts and based on data. So the team will look at things like, "How long did it take us on average to ship a ticket to a customer?" So from the point we created a ticket, how long did it wait in certain states until the customer actually got value out of it? And when we understand that, then we can say, "Okay, well what are our outliers? What tickets took significantly longer to close, and what can we do about that, and we can create actions and we can improve going forward?" It takes some of the, "I feel," out of the process and bases it in facts. You also look at things like production issues, customer defects, and even team health and dependencies. So where are we slowing down based on dependencies, and what's our plan to get away from those dependencies?

Priority reviews are something that can be done either monthly as part of a town hall, or even just a quick 20 minute get everybody together. One of the ways you drive people having ownership and autonomy is they have to understand the value of their work, and how their work relates to the objectives of the company. Really simply, this can be done by just talking about the priorities. So every month my team and I will get together, and I will talk about these are the three most important things for the company. And these are the three most important things for engineering, and you can see how they relate to each other. And then each tribe lead and each product manager will talk about the three most important things for their team. And in 20 minutes, everybody understands what's important. Everybody understands exactly the projects that they're working on that are driving the company forward.

Insights and actions is another way to get teams really bought in and owning their areas of the product. So I find it important to make sure the teams have bandwidth and capacity so that they can make the product better, outside of needing to do business cases, outside of needing to do very heavy processes like roadmaps and things like that. Set aside some capacity in every team so that they can do things like look at the analytics, look at customer feedback, read NPS verbatims. What is a customer trying to do, and how can we make that better? And then just run a experiments. And get together every month and look at the experiments we're running, and are they making a difference or not?

And then the last piece is making sure that a team has KPIs, a team has scorecards, and that those things align back to the company objectives. So whether you're using OKRs, or whether you're using something like a Hoshin canary, the most important thing is that the team should understand where the company wants to be. I like to call it what the company wants to be when it grows up. So three years from now, what do we want? Do we want to have a kick ass NPS? Is it our ENPS we're going after? But making sure that's well understood, and then at an engineering and a product engineering level, leadership should look and go, "Okay, well if the company wants to be that in three years, what can we contribute this year?" So if we want to have an NPS, a world class NPS of 60 in three years time, and our NPS is currently negative 20, maybe we'll say, "You know what? We think with what we plan next year, we can get us to an NPS of 10." And that should be the leadership's end of involvement in how we set OKRs and KPIs for the team.

From that point, you have to empower the team themselves to then go and look, "Okay, as an engineering department, we want to get to a positive NPS of 10 this year. What are we going to do about that?" And let the team set metrics like, "Okay, we're going to improve our automation coverage, because that'll make our quality better, and we believe we can get two NPS points from that," et cetera. And then again, make sure you actively manage this, come together every month and review it as a team.

And that is ... I'm amazed I got through it all on time, but that is the end of my presentation. So I want to thank everybody for being so attentive. I really appreciate it. And if you have any questions, please reach out to me.