Staff Engineer

Staff Engineer by Will Larson is a great guide to building your career toward a Staff+ engineering role and succeeding within the role. This note consists of quote-worthy excerpts from the book and aims to serve as a verbose version of the book's table of contents.


Overview

This book standardizes on the most common sequence of titles: going from Senior to Staff, followed by Principal, and then Distinguished. It uses the term Staff-plus as an overarching label for Staff, Principal, and Distinguished titles. Many companies only have a subset of these titles, slowly adding more as their team grows, but companies that only have one technical leadership title almost always use Staff. A few companies use an alternative sequence, but they're in the minority.

Staff engineer archetypes

The four common archetypes of Staff-plus roles:

  • The Tech Lead
  • The Architect
  • The Solver
  • The Right Hand

This taxonomy is more focused on being useful than complete.

What do Staff engineers actually do?

They still do what made them successful as Senior engineers: building relationships, writing software, coordinating projects. However, previously these were the core of their work, now they're auxiliary tasks.

A shared foundation across all archetypes:

  • Setting and editing technical direction
  • Providing sponsorship and mentorship
  • Injecting engineering context into organizational decisions
  • Exploration
  • Being glue

Setting technical direction

Staff engineers speak for their companies' technology. It's far more about understanding and solving the real needs of the organization around you and far less about prioritizing technology and approaches that you personally are excited to learn about.

Mentorship and sponsorship

You're far more likely to change your company's long-term trajectory by growing the engineers around you than through personal heroics.

The most effective Staff engineer pair a moderate amount of mentorship with considerably more sponsorship (see the distinction).

Providing engineering perspective

Effective organizations streamline routine decision making. But even companies that are great at making routine decisions often struggle when an unexpected decision shows up -- the sort which is both time-sensitive and important, and it's challenging to even pull the right folks together before the decision needs to get made. Examples are organizational restructure or interview loops for infrequent roles (e.g. those hiring only one person each year)

Staff-plus engineers often get unexpectedly pulled into the room where this sort of decision is happening, to inject the engineering context and perspective into a decision while it's still possible to change the outcome. These brief moments of inputs on critical decisions are unduly impactful.

Exploration

Hill-climbing can't solve every problem. In the long-term, companies either learn to explore, or they fade away. Assigning a team that's mastered hill-climbing to do exploratory work is far from a sure thing. They instead find a couple of trusted individuals with broad skills, allocate some resources, and check back in a few months.

Examples include reducing infrastructure costs by an order of magnitude, identifying a multi-region strategy that takes 6 months instead of 3 years, addressing the sudden realization that your primary database only has 3 months of remaining disk space and you can't upgrade to a larger size.

Being Glue

Doing the needed, but often invisible, tasks to keep the team moving forward and shipping its work. See the wonderful post Being Glue by Tanya Reilly.

Slow but rewarding

Timeframes of Staff-plus work are longer. They can feel surprisingly demoralizing.

Does the title even matter?

Three (or four) consistent advantages that generally come with a Staff-plus title:

  • Bypass informal gauges of seniority: Many companies describe themselves as pursuing meritocracy. Given there isn't any widely accepted measure of individual merit, such companies come to rely on "informal gauges of seniority". The sheer informality becomes a broad vector of bias and often conflates confidence with competence. A Staff-plus title allows you to reinvest the energy you've previously spent on proving yourself into the core work you're evaluated on.
  • Facilitating access to "the room"
  • Increase in current and career compensation
  • More agency to select projects, but some find it counteracted by a commensurate increase in accountability to the business

Final remarks

Staff-plus titles are a very different job rather than a better job.

Focusing on developing your approach and skills will be far more important than the title (with exceptions such as for underrepresented groups).


Operating at Staff

There is a significant learning curve in Staff-plus roles. This chapter is about overcoming that curve.

Work on what matters

Pacing yourself becomes the central challenge of a sustained, successful career: increasingly senior roles require that you accomplish more and more and do it in less and less time.

First, a discussion on a few common ways to get tripped up.

Avoid snacking

When an organization runs out of things that are both high-impact and easy, this leaves you with things that are hard and high-impact and things that are easy and low-impact. Doing the latter is "snacking".

These snacks might give a sense of accomplishment, but you're unlikely to learn much from doing them, others are likely equally capable of completing them (and for some it might be a good development opportunity), and there's a tremendous opportunity cost versus doing something higher impact.

Stop preening

A particularly seductive subset of snacking: preening is doing low-impact, high-visibility work.

Many companies conflate high-visibility with high-impact. So it's also an important factor to consider when choosing a company. For long-term career growth, it's far more important to strike a balance between valued work and self-growth.

Stop chasing ghosts

It's common for a new senior leader to join a company and immediately drive a strategy shift that fundamentally misunderstands the challenges at hand. The ghosts of their previous situation hold such a firm grasp on their understanding of the new company that they misjudge the familiar as the essential.

Take the time to understand the status quo before shifting it.

Now, what should you work on?

Existential issues

If your company is experiencing an existential risk, then always focus there.

Work where there's room and attention

  • Priorities that will become critical in the future, where you can do great work ahead of time.
  • Areas that are doing ok but could be doing great with your support.

Sometimes there's work that's worthy of attention but which the organization is incapable of paying attention to, usually because its leadership doesn't value that work. If it's very important, you're going to want to advocate for it.

Foster growth

Grow the team around you. Hiring is usually optimized, but onboarding, mentoring, and coaching are wholly neglected at many companies despite being at least as impactful as hiring to your company's engineering velocity.

Edit

A surprising number of projects are one small change away from succeeding, one quick modification away from unlocking a new opportunity, or one conversation away from consensus. I think of making those small changes, quick modifications, and short conversations as editing your team's approach.

With your organizational privilege, relationships built across the company, and ability to see around corners derived from experience, you can often shift a project's outcome by investing the smallest ounce of effort.

From Linghao: I wrote a short post on this.

Finish things

One special sort of editing: helping finish a project that just can't quite close itself out. E.g. a talented engineer earlier in their career who can't quite create buy-in or rescope the project into finishable work.

What only you can do

An intersection of what you're exceptionally good at and what you genuinely care aout.

Writing engineering strategy

A good engineering strategy is boring. It's easier to write an effective strategy than a bad one.

Write five design documents, and pull the similarities out. That's your engineering strategy.

Write five engineering strategies, and forecast their implications two years into future. That's your engineering vision.

Resist the urge to include your most brilliant ideas in the process.

Why

Strategies allow everyone to make quick, confident decisions that might have otherwise cost them a week of discussion. Strategies are also the bricks that narrow your many possible futures down enough that it's possible to write a realistic vision.

When

  • If you realize that you've rehashed the same discussion three or four times, it's time to write a strategy.
  • When the future's too hazy to identify investments worth making, it's time to write another vision.
  • Otherwise, move on to other work for now and return later.

Write five design docs

As a rule of thumb, you should write design documents for projects:

  • whose capabilities will be used by numerous future projects
  • that meaningfully impact your users
  • taking more than a month of engineering time

Design documents have what bad strategies lack: detailed specifics grounded in reality. It's easy for two well-meaning engineers on the same team to interpret an abstract strategy in different ways, but it's much harder to stay misaligned when you're implementing a specific solution.

Recommendations:

  • Start from the problem: the clearer the problem statement, the more obvious the solutions.
  • Keep the template simple.
  • Gather and review together, write alone: most folks are better writers than they are editors so it's usually harder to edit a group document into clear writing.
  • Prefer good over perfect.

Synthesize those five design docs into a strategy

Sit down and review the five design documents together. Look for controversial decisions that came up in multiple designs, particularly those that were hard to agree on.

Good strategy guides tradeoffs and explains the rationale behind that.

Recommendations:

  • Start where you are: do not wait for missing information because every missing document is missing for a good reason. Just dive in and start writing.
  • Write the specifics.
  • Be opinionated.
  • Show your work: make it possible for others to modify and extend your work as the underlying context shifts.

Some of the best strategies you write may at the time feel too obvious to bother writing. E.g. "when should we write design docs". But remember that writing strategy is not a show of brilliance. We can write them more casually. We can always deprecate them if they end up not being used.

Extrapolate five strategies into a vision

As you edit through the contradictions between strategies, you've written an engineering vision.

Recommendations:

  • Write two to three years out
  • Ground in your business and your users
  • Be optimistic rather than audacious
  • Stay concrete and specific
  • Keep it one to two pages long

A great vision is usually so obvious that it bores more than it excites, so expect a muted initial response and don't get discouraged.

Managing technical quality

Low technical quality isn't a crisis. It's the expected, normal state. Closing the gap between your current and target technical quality is a routine, essential part of effective engineering leadership.

The toolkit of managing technical quality (think of ascending a staircase):

  • Fix the hot spots that are causing immediate problems
  • Adopt best practices that are known to improve quality
  • Prioritize leverage points that preserve quality as your software changes
  • Align technical vectors in how your organization changes software
  • Measure technical quality to guide deeper investment
  • Spin up a technical quality team to create systems and tools for quality
  • Run a quality program to measure, track and create accountability

Do the quick stuff first and fail fast. Premature processes add more friction than value.

Hot spots

When confronted by a quality problem, the first instinct is often to identify a process failure that necessarily requires a process solution. If a deployment causes an outage, it's because the author didn't correctly follow the code test process, so now we're going to require tests with every commit – that'll teach those lazy developers!

However, process rollout requires humans to change how they work, which you shouldn't take lightly. Instead, understand the problem at hand and try to fix it directly.

In the above example, you might benefit from giving the author feedback about changing their testing habits. Alternatively, maybe you're better served by acknowledging that your software design is error-prone and it's better to "define errors out of existence".

If you find that your organization is creating quality problems faster than you're able to fix hot spots, it's time to move on to adopting best practices.

Best practices

A good process is evolved rather than mandated. Study how other companies adopt similar practices, document your intended approach, experiment with the practice with a few engaged teams, sand down the rough edges, improve the documentation based on the challenges, and only then roll it out further.

Also, limit concurrent process rollouts. Adopting multiple new practices simultaneously creates contention of folks' attention, makes it hard to attribute impact later if you're considering reverting or modifying one of the new practices.

When you find yourself wanting to adopt a new best practice before your in-progress best practice is working, rather than increasing your best practice adoption-in-progress limit, move on to the next tool.

Leverage points

There are a small handful of places where extra investment preserves quality over time. I call those quality leverage points, and the three most impactful points are interfaces, stateful systems, and data models.

As you identify these leverage points in your work, take the extra time to approach them deliberately. If it's an interface, integrate half a dozen clients against the mocked implementation. If it's a data model, represent half a dozen real scenarios. If it's stateful, exercise the failure modes, check the consistency behaviors, and establish performance benchmarks resembling your production scenario.

Take everything you've learned, and pull it into a technical specification document that you socialize across your team. Gather industry feedback from peers. Even after you begin implementation, listen to reality's voice and remain open to changes.

Technical vectors

If you plot every technical decision as a vector on a grid, the more those vectors point in the same direction, the more you'll accomplish over time.

Fundamental tools for aligning technical vectors:

  • Give direct feedback.
  • Refine your engineering strategy from tech spec, to strategy, to vision.
  • Encapsulate your approach in your workflows and tooling. Documentation of a clear vision is helpful, but some folks simply won't study your document. Deliberate tools create workflows that nurture habits far better than training and documentation. E.g. provisioning a new service might require going to a website that requires you to add a link to a technical spec for that service.
  • Train new team members during their onboarding.
  • Use Conway's Law. Conway's Law argues that organizations build software that reflects their structure. If your organization is poorly structured, this will lead to tightly coupled or tangled software.
  • Curate technology change using architecture reviews, investment strategies, and a structured process for adopting new tools. Most misalignment comes from missing context, and these are the organizational leverage points to inject context into decision-making.

If the above is not enough, the first step towards heavier approaches is, as always, measurement.

Measure technical quality

It is possible to usefully measure code quality, and it comes down to developing an extremely precise definition of quality.

Some representative components to consider including in your quality definition:

  • What percentage of the code is statically typed?
  • How many files have associated tests?
  • What is test coverage within your codebase?
  • How narrow are the public interfaces across modules?
  • What percentage of files use the preferred HTTP library?
  • Do endpoints respond to requests within 500ms after a cold start?
  • How many functions have dangerous read-after-write behavior? Or perform unnecessary reads against the primary database instance?
  • How many endpoints perform all state mutation within a single transaction?
  • How many functions acquire low-granularity locks?
  • How many hot files exist which are changed in more than half of pull requests?

You're welcome to disagree that some of these properties ought to exist in your codebase's definition of quality: your definition should be specific to your codebase and your needs. The important thing is developing a precise, measurable definition. There will be disagreement in the development of that definition, and you will necessarily change the definition over time.

Instrumentation complexity is the biggest friction point for adopting these techniques in practice, but if you can push through, you unlock something pretty phenomenal: a real, dynamic quality score that you can track over time and use to create a clarity of alignment in your approach that conceptual alignment cannot.

Technical quality team

A technical quality team is a software engineering team dedicated to creating quality in your codebase. Start with a fixed team size of three to six folks. Having a small team forces you to relentlessly prioritize their roadmap on impact and ensures you'll maintain focus on the achievable.

Some fundamentals of success are:

  • Trust metrics over intuition.
  • Keep your intuition fresh.
  • Listen to and learn from your users.
  • Do fewer things, but do them better.
  • Don't hoard impact.

Quality program

  • Identify a program sponsor.
  • Generate sustainable, reproducible metrics.
  • Identify program goals for every impacted team and a clear path for them to accomplish those goals.
  • Build the tools and documentation to support teams towards their goals.
  • Create a goal dashboard and share it widely.
  • Send programmatic nudges for folks behind on their goals.
  • Periodically review program status with your sponsor.

Stay aligned with authority

Titles come with the sort of power called organizational authority, and that variety of authority is loaned to you by a greater organizational authority. What's bestowed can also be retracted, and retaining organizational authority depends on remaining deeply aligned with the bestowing sponsor, generally your direct manager. To remain effective within a Staff-plus role, you have to learn the art of staying aligned with organizational authority.

Staff-plus roles are leadership roles, and in leadership roles, the support system that got you here will fade away. Often abruptly, you're now expected to align the pieces around you for your own success.

To align with your manager:

  • Never surprise your manager -- it's the fastest way to destroy trust
  • Don't let your sponsor surprise you. Be proactive to facilitate information flow.
  • Feed your manager's context. Help your manager not get surprised by the wider organization.

Part of growing as a leader is developing your own perspective on how the world should work, and you can't reach the Staff-plus level without that perspective. Having a clear sense of how things ought to work sharpens your judgment and enables you to act proactively. As you reach this next step of leadership, you increasingly have to merge your vision with those held by more senior organizational leaders.

I recommend sharpening your awareness of the distinctions between the values that you hold and those that the organization operates under and find a way to advocate for them without getting kicked out of the room.

To lead, you have to follow

Management is a specific profession, and leadership is an approach one can demonstrate within any profession.

First, leaders have a sufficiently refined view of how things ought to work such that they can rely on their distinction between how things are and how they ought to be to identify proactive, congruent actions to narrow that gap. Second, they care enough about the gap to actually attempt those narrowing actions.

The most effective leaders spend more time following than they do leading. They move in and out of leadership and follower roles with the folks around them.

  • Be clear with yourself what your true priorities are, and don't dilute yourself across everything that comes up. If there's something you disagree with but only in a minor way, let others take the lead figuring it out.
  • Give your support quickly to other leaders who are working to make improvements. Even if you disagree with their initial approach, someone trustworthy leading a project will almost always get to a good outcome. If someone trustworthy is leading a project, and you're still uncomfortable letting them move forward, consider why you lack confidence in your ability to influence them and if you're bad at giving feedback.
  • Make your feedback explicitly non-blocking. This can be classifying a code review comment as an "optional nit", but it can also be writing up detailed feedback but delivering it to someone mentioning that you wanted to share your perspective rather than necessarily change their approach.

Continued growth requires learning to incorporate your worldview into the worldviews of those around you, accelerating overall progress around you even if it means tolerating a detour from your vision.

What you can accomplish alone is far from what you can accomplish by creating leaders. To be a great leader, take your time learning to follow.

Learn to never be wrong

More complex projects get derailed by personal conflict than by technical complexity.

Find a way to never be wrong without dominating the room. To be right while creating space for others.

Approaches to effective meetings

  • Listen through questions: ask good questions to open up conversations and create space and safety for others
  • Define the purpose if there isn't a clear one
  • Know how to read the room: if the folks in the room are too far apart, identify a subgroup who are able to spend more time digging into it together or identify an appropriate party to escalate to outside of the room

Dealing with jerks

  • Including someone they can't be a jerk to in the meeting (like their manager or the CTO)
  • Investing heavily into aligning with them before the meeting, so they feel heard and are less likely to derail the discussion

This works best especially if it's a jerk you interact with infrequently. If it's someone you're responsible for then you have a different obligation.

It's also useful to recognize that the authority created by your title shelters you from many of these folks, so whoever you're experiencing is being less of a jerk to you than they are to others. If the behavior feels borderline for you, it's potentially more egregious for others.

Create space for others

One of the best measures of your long-term success as a Staff-plus engineer is that the organization around you increasingly benefits from, but doesn't rely upon, your contributions.

Discussions

When you start thinking about creating space, a good discussion is one that it turns out you didn't need to attend. When you make a key contribution, feel good about it, and then think about what needs to happen for someone else to make that contribution next time.

  • Shift your contribution towards asking questions.
  • If you see someone in the meeting who isn't participating, pull them into the discussion.
  • Be the one to take notes.
  • If you realize someone's missing from the discussion who should be there, be the person to pull them into the next occurrence of the meeting.

Decisions

It's possible to take an incremental approach to shift increasingly complex and important decisions to your wider team.

  • Write it down.
  • Circulate early, and do it before you've crystallized on a decision.
  • Separate style from substance, and stop giving style feedback on other folks' decisions.
  • Don't try to show value. Some senior folks feel like they need to weigh in on everything to justify their seniority. Others require each decision to exactly mirror a similar decision they once made. Both of these center insecurity over impact and prevent others from growing as leaders.
  • Change your mind. One of the biggest signs of respect for your coworkers is listening to them and then changing your mind afterward. If senior leaders don't change their mind, then soon everyone will correlate bluster with success.

Sponsorship

It's great to involve folks in your work. But at some point, you have to take the next step -- make the work theirs.

When critical work comes to you, your first question should become, "Who could be both successful with and grown by this work?"

Ultimately sponsorship includes letting them take an approach that you wouldn't.

Build a network of peers

https://lethain.com/meeting-people/

  • Be visible: build an external network via writing, speaking at conferences, twitting, discussing on Slacks, etc.
  • Internal networks, too
  • Ambient networks: keeping current with industry books and following industry leaders on social networks.

Value quality over quantity.

Present to executives

Why this is hard

As you get further into your career, increasingly your impact will be constrained by your ability to influence executives effectively.

Almost all executives are outstanding at something; it's just that often that something isn't the topic you're communicating about with them. When you combine that lack of familiarity with your domain with limited time for the topic at hand, communication is a challenge.

In addition, communicating with executives can be unexpectedly difficult for a less apparent reason: the executive has become accustomed to consuming reality preprocessed in a particular way.

How to communicate effectively

When you're communicating with an executive, it's almost always one of three things: planning, reporting on status, or resolving misalignment.

Although these are distinct activities, your goal is always to extract as much perspective from the executive as possible.

The best way to extract their perspective is by writing a structured document. While many structures can work, I recommend every document's opening paragraph follow the SCQA format:

  • Situation
  • Complication
  • Question
  • Answer

After you've written your structured document, gather feedback on it from your peers and stakeholders. Aligning with stakeholders before your presentation, sometimes called nemawashi, is extremely effective at reducing surprises. Some of your peers should have experience presenting to the executives and will have useful feedback on improvements.

For the presentation itself, set a clear agenda, but don't focus on rote conformance. A great meeting with executive leadership is defined by engaged discussion, not addressing every topic on the agenda.

Mistakes to avoid

  • Never fight feedback.
  • Don't evade responsibility or problems.
  • Don't present a question without an answer.
  • Avoid academic-style presentations.
  • Don't fixate on your preferred outcome.

Getting the title where you are & Deciding to switch companies

As you can infer from the titles, I find these two chapters more situational and not that worthwhile the time (maybe a quick skim). Therefore, I'm not including notes for them.


Stories

Much of the wisdom from these interviews with staff+ engineers is already put in the first half of the book. Give them a read if you enjoy interview transcripts, though.

Subscribe via RSS

CC BY-NC-SA 4.0 © 2016 - 2024 ❤️ Linghao Zhang