On becoming a senior technical leader

Jesse: Hey! So we’re just going to have a casual conversation for an hour about how you got to where you are today. And I’d love to just start with — what did you do before Coinbase?

Michael: Immediately prior to Coinbase, I was running the technical side of a startup; a consumer to business feedback startup. Similar to UserVoice. That helped teach me how to self-manage technical work.

Jesse: How big was the team?

Michael: Two people. I think once the technical challenges faded away I became less interested in the startup. So I was looking for opportunities. Before that I was working mostly in government work — my main project was an earth science visualization app doing OpenGL and Java; mostly desktop programming. We got into a little bit of WebGL at the end but mostly building a Desktop app on Eclipse RCP and OpenGL for visualization. So that was interesting. And then before that I worked for a little C# web consulting firm.

Jesse: Any big takeaways from doing the startup?

Michael: It’s hard. It’s especially difficult to work really hard at something and not get paid.

Jesse: You didn’t get paid?

Michael: Yeah we didn’t get paid for two years. I was really motivated by the technical challenges, and I was also living in a beautiful city. So that was nice. But at one stage I thought, I can’t do it anymore.

Jesse: And tell me more about the technical challenges? Like what were the technical challenges?

Michael: I had to figure out everything: how we deploy on AWS, how to build backend, how to interact with the data store. We used Ember on the frontend and I was pretty new to frontend applications. So learning how to build an application from the infra to the frontend, end to end. And then the other challenges were how do we get out there and how do we pitch this app. That was the work that was far harder for me. I mainly leaned on my business partner for that.

Jesse: What was the hardest moment?

Michael: I think it was hard to let it go. It was something that we’d worked hard on for two years and really tried to sell it. We got a tiny bit of traction but it wasn’t worth keeping on pushing. I thought that the platform was technically pretty cool, so letting go of that was hard.

Michael: Also, it was really perfect timing to be available — I had to fly over to SF to interview with Coinbase and work for a week. If I had a normal job I probably wouldn’t have considered it. So, I was really set up for that.

Jesse: And what were you looking for when you started looking for a job?

Michael: I grew up in Australia and had always wanted to work in the U.S. The work culture here is different. At university I’d met some people who had worked in Silicon Valley and I was impressed. To work in another country had always been a dream. And then towards the end of the startup journey, I got really interested in cryptocurrency, and Coinbase just happened to be looking for people on Reddit. I had no idea about Coinbase at the time, I wasn’t interested in the company in particular, I just really liked cryptocurrency.

Jesse: So you didn’t know about Coinbase until you saw a Reddit thread looking for engineers?

Michael: Yep. It was Brian that posted. Even when I came over, I thought: “they don’t seem that legit.” Then once I started the work trial I was super hyped. That initial casualness possibly helped me there, because I was a less nervous.

Jesse: So tell me about the work trial? What did you do?

Michael: I worked with New Relic. Going through our top transactions and making them more performant and finding scaling bottlenecks.

Jesse: So you were working in our monolithic Rails app? Had you ever worked on Ruby or Rails?

Michael: Nope. Never seen Ruby before and never used New Relic. Never used Redis or Mongo or anything like that before, it was all new.

Jesse: How did you make progress on that?

Michael: I think I just I wanted to do a good job, so I worked long hours and made sure I had something to show for it. I had a goal of having four transactions improved. It was a very methodical process. You could go into New Relic and have a look at the different traces and see where things were.

Jesse: What was the end result?

Michael: Different endpoints had different problems. We found one that had 100 Redis calls when it should have only been one; it was an n + 1 query. That was an interesting one that you could put in the presentation at the end and show the drop-off in the graph. I found that really fun. I was talking to the hiring manager before the work trial and he said “what are you interested in?” and I mentioned “I really like optimizing code” but I’d never done it in a web context before. So I got thrown into that and I think I just really liked it and kept doing it when I started.

Jesse: Then you started — what level did you come in as? What were your expectations coming in?

Michael: Yeah, I started out as an IC4. A mid-level engineer. I’d never worked in the US before, so I didn’t know what to expect. I expected to be really challenged, having no experience in any of these technologies. I mentioned the culture is different. And so I really didn’t feel like I belonged in the first month or so. Heavy imposter syndrome.

Jesse: What did that feel like?

Michael: I said to my wife: “I’m going to get fired any day.” It was always so new. But also really exciting. We were a pretty small team.

Jesse: What was the team?

Michael: We were four people and we were responsible for Coinbase.com, the web side of things; the API, backend and the front end.

Jesse: What was your role on the team?

Michael: I got hired as a full stack engineer, to work on the web app. I did a little bit of frontend stuff, worked on the charts a little bit, but took a liking to the backend side of things more. Because I was looking at performance on a weekly basis, certain things started to really stick out and I became more and more involved lower-level platform work. One of my early tasks was the portfolio API and to do that I had to to run a migration on our historical transaction data from the beginning of time to change their IDs. It took around three months to actually get this done. That was a hard one and we ended up compromising. We did the 20 percent work to get 80 percent of results. There was a bug at the start which meant I had to redo the whole aggregation, but after that it worked really well.

Jesse: Pretty amazing. Talk me through your first 6 months. Things got crazy — what was that like?

Michael: I started February 1st. In May, Ethereum started blowing up. So I had 4 months to learn Ruby and our monolithic Rails app and figure out how things work and then Ethereum went crazy and we had scaling challenges. That was our main focus. We had Redis challenges and Mongo challenges and once that happened, all my product work was basically out the door. I was still doing product work, I was still assigned that stuff, but it ended up slipping most of the the time. That was a hard part — it was kind of like my role changed without it being explicit.

Jesse: Talk to me about that.

Michael: Our team was built around sprints doing product work and there was scaling work that was really hard to prioritize because it didn’t really contribute to any of our product goals. So I needed to just tell the team “this is what I’m working on this week” — and at times I felt like I was letting people down because I’m not working on the product pieces that were pushing the company forward. And I remember saying that explicitly to people. So I think that was tricky. Not having explicit goals on fixing this stuff.

Jesse: So why did you do that? Why did you do the scaling work rather than the product work?

Michael: I was scared. I knew that we had problems and it seemed like the highest leverage thing.

Jesse: So it sounds like you stepped into a — to quote Brian, our CEO — “war mode” like crisis. Stepping out of being a strong contributor on the day to day and stepping into some massive challenge that was unknown that you could help solve.

Michael: Yeah. And that group of people that got together, we were super tight knit and were very much focused on this singular goal of being able to serve more customers. And because we we’re always in this war mode, that really brought us together as a group of people. It was super fun.

Jesse: In the shift to this tour of duty, what was the hardest thing and what was the funnest thing?

Michael: You could say that the hours were hard, because we were doing long hours fighting these scaling challenges. But they were also just so much fun. I think nobody would have given that up for the world. We were seeing things that we would probably never see again in our lifetime, scaling challenges of that magnitude, everybody wanted to be there, no matter how hard it was.

Jesse: And what was your role in that group of people? How did it change from May to January, after most of the scaling challenges ended?

Michael: I don’t know. I don’t know if I see myself as any different from other people on that team. Maybe there was a part of it where I would push a riskier, or a bit more high leverage things, but I’m not sure.

Jesse: Ha. I was there and I feel like you were the leader — at least from a technical perspective. There were a bunch of important people, but when shit went really wrong, everyone was like: “where’s Michael?”

Michael: Yeah, I think I remember some of those times. I think being able to think methodically and practically during fires is super important. Clearing your head and not panicking but being able to think clearly so you can find the root causes of the problem or come up with really innovative solutions.

Jesse: And then the craziness kind of ended right in January, and what happened next?

Michael: After January, we put this performance and reliability team together — we finally took that implicit team and made it explicit. And goaled people on it. I think that was really helpful because we finally had something that we could measure ourselves on. And we had a mission statement for our team.

Jesse: How do you think did? How do you think that team did?

Michael: We had these metrics that we were trying to fix and because they were easily measurable, we could dive deep on certain solutions to move those metrics a lot. Some of the most exciting times are when you make a change and you push it to production and you see those graphs dive. So we were always chasing the thrill of that feeling. Every time we did something like that which was successful, we made Coinbase a healthier place. And prior to that team, most of the focus on platform was a little bit ad hoc. And now that we had someone that was focused on it full time — we could think about not just performance, but how do we build tooling for people to be safer? How do we make our monolithic Rails app a safer place? I see those things as successful and I think that without that team we wouldn’t have been able to push that because we would have mostly worked on product.

Jesse: And what was your role in that team?

Michael: I was tech leading that team — and was supported by the two other team members. Charting the direction of that team from a technical standpoint. Figuring out what was most important to work on. This was the first time when I really started understanding how to solve problems at a level above code. I was learning how to think at a higher level, and then how to communicate the importance of the problems and work we were seeing. And communicate to the company — hey, this is really important, here’s why it’s so important, here’s why not investing in these areas would be a mistake. Working and communicating at this level was a new challenge, but I feel like we made a lot of really good progress. In the meantime, our SRE team had formed on the infrastructure side of things and there was a lot of overlap there with the performance work. So we decided to merge the performance & reliability team with the SRE team. But I stuck around in Consumer in a generalist, cross-team role, reporting directly to you.

Jesse: You had really proven that you could think at a high level about what problems were most important to solve. I wanted your help in solving some of the bigger problems we had at an organizational level. I was looking for a partner who could take on doing technical leadership inside of Consumer. What was that change like?

Michael: Yeah. I didn’t know what direction that was gonna go in. I didn’t know what I was going to be working on. So, I was really nervous about that. It was unclear, but I was also excited about having my time freed up to think about cross team initiatives and to find the most problematic areas, or where the most pain is, and then focus on those areas.

Jesse: You pulled yourself into a role where you were not attached to a team, it was pretty uncertain, and you felt scared. In the face of that fear, what did you do?

Michael: Similar to what I was doing before. You just try and find the where the most pain is. Then figure out ways to fix it. One of the first things I focused on was the Consumer Engineering Review process. That was something that was cross team and had big impact on quality of engineering in Consumer. Beforehand, there wasn’t very much thought or discussion around architectural decisions for new or existing projects. And so adding a forcing function to have engineers describe those decisions in a technical document, then review them with a group, has been really helpful.

Jesse: That sounds like a cultural problem that you solved. We’ve talked a lot about technical problems — when did you start thinking about solving cultural or organizational problems?

Michael: I just got really frustrated with things and then wanted to take initiative or ownership of them, like the interview stuff.

Jesse: What was the interview stuff?

Michael: We knew that the tech screen had gotten leaked and so we were just interviewing these candidates with this tech screen that half of the candidates clearly knew. We found candidates that had the answer in another tab. And nobody was really owning that or taking responsibility for that. I saw the pain there and wanted to help out. So I took on building a new tech screen and training the organization to use it.

Jesse: So it sounds like you were looking at which organization or technical problems were most painful and you weren’t particular about which ones you solved?

Michael: Yeah, I think so. Taking initiative on things, and not waiting to be asked or waiting for things to get so bad that somebody else solves them, is a really important part of that.

Jesse: Yeah, well said. So, In this new role where you were floating and not touching any team, what was the hardest part and what was the funnest part?

Michael: It’s hard not having a team. I was used to having people around me to bounce ideas off of and work with. People who I could share load with if I felt overwhelmed, and do the same for them. And then just figuring out how to manage my time has been probably the hardest thing. There are so many things that we can do and figuring out how to prioritize is really hard. The funnest part? Being able to have a bigger say in the future shape of Coinbase. I find that really rewarding and enjoyable.

Jesse: What is a piece of advice you’d give to someone who is currently a senior engineer on a team and who wants to have more of an impact at the organizational level?

Michael: Like I said before, being aware of cross team pain or organizational pain and then taking initiative to fix it, is just something that everybody appreciates. We can’t get enough of it. Like the interview process, like building tooling in Monorail to make spec faster or less flaky, or anything that improves your teammates’ lives or improves the health of organization. It’s really pushing on that and taking initiative that I think is the best advice.

Jesse: I love that. Is there anything you would have done differently in your path to getting here?

Michael: Yeah. I continually need to get better at communication, especially as I’ve started working in a cross-team role. Communication breaks down the bigger the team or the wider your reach gets. And that’s something that I still really need to work on. Making sure the right stakeholders in the room, getting people’s buy-in, communicating things broadly and clearly.

Jesse: What’s something you didn’t realize about operating at your level before you started doing it that you realize now?

Michael: It’s the communication thing I think. It’s much harder to push things forward when there’s more people that have a stake in the decisions. Even when you think that something is obvious to you, other people have very good opinions on that and why you’re wrong. Figuring out how to navigate that is a constant lesson.

Jesse: What are you working on right now?

Jesse: So I’m working on two things. The first is the API paved road: creating a consistent and supported way to build APIs at Coinbase. Figuring out how we do that internally, how we do that externally. Different teams have different ways of coding APIs currently and there’s real advantages to having a consistent approach. The second thing is building typing in our monolithic Rails app, in Ruby. As we start to servicify our monolithic Rails app and split things out, there’s a need for increasing safety. Having types allows you to reason about your code more.

Jesse: What’s your day-to-day schedule like?

Michael: I’m probably in 40% meetings? I try and block out Wednesdays and then Monday and Thursday mornings. I find myself helping out in Slack a lot and have a lot of people pinging me with questions about our monolithic Rails app, so I find that I get most of my heads-down technical work done outside of work hours. That’s another thing that’s been really hard actually. The more you know about systems, the more questions you get asked. And it’s hard to really focus in on a problem when you keep on getting asked questions. But it’s also very high leverage to unblock people — I see it as a good thing, just something that I need to continue learning how to manage. Also, it’s kind of like a pulse. If a bunch of people say “I’ve seen this problem recently,” then clearly something that’s going on that needs to be fixed.

Jesse: Yeah, interesting. What’s something that you’re really proud of?

Michael: I’m really proud of a couple things. Getting Sorbet into our monolithic Rails app is really exciting — this is a new technology and a new product that not much of the world has access to and people are really excited by it. I’m interested languages so having static typing is really cool. Then, the process of scaling MongoDB and then giving presentations on the work. I’m proud of that as well.

Jesse: Amazing, thank you for your time. Feel grateful for the last two years we’ve spent together.

Michael: Me too!

Add a comment