Starting from "time is our most limited resource", The Effective Engineer by Edmond Lau first establishes the methodology of using "leverage" to guide our actions. The book then, from multiple angles, discusses how to become a more effective engineer by focusing on high-leverage activities that produce a disproportionately high impact for a relatively small time investment. Ranging from adopting mindsets, actual execution, to building long-term values, topics involved are complemented with ample examples within the industry. Much content are easily generalizable to areas beyond software engineering as well. A must-read for software engineers.
This post is a refined version of the notes I took while reading this book.
Leverage = Impact Produced / Time Invested
Leverage is critical because time is your most limited resource.
Pareto principle: 80% impact from 20% work -- high-leverage activities that produce a disproportionately high impact for a relatively small time investment.
Three ways to increase your leverage:
Developing a customer-facing feature
Fixing bottlenecks in a web application
Don't confuse high-leverage activities with easy wins. Many high-leverage activities require consistent applications of effort over long time periods to achieve high impact.
Find leverage points, establish high-leverage habits.
Own your story. Instead of apologizing for where your resume doesn't line up, take control of the parts that are within your sphere of influence.
Lessons from compound interest:
When companies pay you for cushy and unchallenging 9-to-5 jobs, what they are actually doing is paying you to accept a much lower intellectual growth rate.
Treat yourself like a startup. Startups initially prioritize learning over profitability to increase their chances of success.
You would rather invest your financial assets in accounts that pay high interest rates, not low ones. Why would you treat your time -- your most limited asset -- any differently?
One of the most powerful leverage points for increasing our learning rate is our choice of work environment -- because we spend so much time at work.
Factors to consider:
Borrow the idea of 20% time from Google, but take it in one- or two-hour chunks each day, because you can then make a daily habit out of improving your skills.
Gain experience in adjacent disciplines:
Some skills we learn could be cross-functional and help our engineering work. For example, increasing your comfort level in conversing with strangers can help with meeting and interviewing. Other skills might not translate directly into engineering benefits, but the practice of adopting a growth mindset toward them makes us better learners and more willing to stretch beyond our comfort zone.
Continual learning is inextricably linked with increased happiness.
The human brain is optimized for processing and not for storage. The average brain can actively hold on 7 +/- 2 items. Expending effort on remembering things reduces our attention, impairs our decision-making abilities, and even hurts our physical performance.
To-do lists should be 1) a canonical representation of our work and 2) easily accessible.
Instead of accurately computing the leverage of each task (which is incredibly difficult), compile a small number of goals to complete. Pick initial tasks towards these goals, and then make a pairwise comparison between what you're currently doing and what else is on your to-do list. Continuously shift your top priorities towards the ones with the highest leverage.
Activity is not necessarily production. Activities like writing status reports, organizing things, creating organizational systems, recording things multiple times, going to meetings, replying to low-priority communications only have a weak and indirect connection to creating value.
Once you're producing results, few people will complain about declined meetings, slow email response times, or even non-urgent bugs not being fixed.
Defer and ignore tasks that don't directly produce value.
Urgency should not be confused with importance. Put first things first.
Label to-dos from 1 to 4 based on which quadrant the activity fall under.
Oftentimes, the root cause of a Quadrant 1 problem is an underinvestment in a Quadrant 2 activity.
The act of prioritization is itself a Quadrant 2 activity, whose important often gets overlooked because it's rarely urgent. Prioritize the act of prioritization.
Engineers need longer and more contiguous blocks of time to be productive than many other professionals.
Managers traditionally organize their time into one-hour blocks. Makers generally prefer to use time in units of half a day at least.
Increasing work linearly increases the likelihood of failure exponentially.
Constant context switching hinders deep engagement in any one activity and reduces our overall chance of success.
Many people do not have sufficient motivation to summon the activation energy required to start a difficult task.
Planning creates a link between the situation or cue and the behavior that you should follow, which follows automatically without any conscious intent when the cue triggers.
Subconscious followup is important because procrastination primarily stems from a reluctance to expand the initial activation energy on a task. This reluctance leads us to rationalize why it might be better to do something easier or more enjoyable, even if it has lower leverage. When we're in the moment, the short-term value that we get from procrastinating can often dominate our decision-making process. But when we make if-then plans and decide what to do ahead of time, we're more likely to consider the long-term benefits associated with a task.
If-then planning also can help fill the small gaps in our schedule.
Take general principles and iteratively adapt your own prioritization system.
The actual mechanics of how your review your priorities matter less than adopting the habit of doing it.
Investing in iteration speed is a high-leverage decision. The faster you can iterate, the more you can learn about what works and what doesn't work.
You can build more things and try out more ideas. Not every change will produce positive value and growth. But with each iteration, you get a better sense of which changes will point you in the right direction, making your future efforts much more effective.
Almost all successful people write a lot of tools.
Sometimes, the time-saving tool that you built might be objectively superior to the existing one, but the switching costs discourage other engineers from actually changing their workflow and learning your tools. It's worth investing the additional effort to lower the switching cost and to find a smoother way to integrate the tool into existing workflows.
One side benefit of proving to people that your tool saves times is that it also earns you leeway with your manager and your peers to explore more ideas in the future.
As engineers, we can shortcut around normal system behaviors and user interactions when we're testing your products, extending the concept of a minimal reproducible test case.
When you're fully engaged with a bug you're testing or a new feature you're building, the last thing you want to do is to add more work. When you're already using a workflow that works, albeit with a few extra steps, it's easy to get complacent and not expend the mental cycles on devising a shorter one. Don't fall into this trap!
Effective engineers have an obsessive ability to create tight feedback loops for what they're testing.
Given how much time we spend in our programming environments, the more efficient we can become, the more effective we will be as engineers.
Mastery is a process, not a event. As you get more comfortable, the time savings will start to build.
One common type of bottleneck is dependency on other people. Oftentimes the cause is misalignment of priorities rather than negative intentions. The sooner you acknowledge that you need to personally address this bottleneck, the more likely you'll be able to either adapt your goals or establish consensus on the functionality's priority.
Project fail from under-communicating, not over-communicating.
Even if resource constraints preclude the dependency that you want from being delivered any sooner, clarifying priorities and expectations enables you to plan ahead and work through alternatives.
Another common type of bottleneck is obtaining approval from a key decision maker. This kind of bottlenecks generally fall outside of an engineer's control. Prioritize building prototypes, collecting early data, conducting user studies and so on to get preliminary project approval. Don't defer approvals until the end.
A third type of bottleneck is the review processes that accompany any project launch. Expend slightly more effort in coordination and communication.
Premature optimization is the root of all evil. Find out the biggest bottlenecks and optimize them.
If you can't measure it, you can't improve it.
Good metrics accomplish a number of goals:
What you don't measure is important as well.
Choose metrics that 1) maximize impact, 2) are actionable, and 3) are responsive yet robust.
When it comes to diagnosing problems, instrumentation is critical.
Adopting a mindset of instrumentation means ensuring we have a set of dashboards that surface key health metrics and that enable us to drill down to the relevant data. However, many of the questions we want to answer tend to be exploratory, since we often don't know everything that we want to measure ahead of time. Therefore, we need to build flexible tools and abstractions that make it easy to track additional metrics.
The knowledge of useful numbers provides a valuable shortcut for knowing where to invest effort to maximize gains.
Internalizing useful numbers can also help you spot anomalies in data measurements.
Knowledge of useful numbers can clarify both the areas and scope for improvement.
To obtain performance-related numbers, you can:
The right metric can slice through office politics, philosophical biases, and product arguments, quickly resolving discussions. Unfortunately, the wrong metric can do the same thing -- with disastrous results.
All data can be abused. People interpret data the way they want to interpret it.
Untrustworthy data that gets incorporated into decision-making processes provides negative leverage. It may lead teams to make the wrong decision or waste cognitive cycles second-guessing themselves.
Our best defense against data abuse is skepticism.
Metrics-related code tends to be less robust. Errors can get introduced anywhere in the data collection or processing pipeline:
Invest a small amount of work to gather data to validate your project assumptions and goals.
One way to validate your idea would be to spend 10% of your effort building a small, informative prototype.
Even if you were absolutely convinced that a certain change would improve metrics, an A/B test tells you how much better that variation actually is.
A/B tests also encourage an iterative approach to product development.
Additional risks introduced by working on a one-person project:
Tips for setting up feedback channels to increase chances of success:
Creating a feedback loop is necessary for all aspects of a job. Many of our work decisions are testable hypotheses. You may not be able to test an idea as rigorously as you could with an A/B test and ample amounts of traffic, but you can still transform what otherwise would be guesswork into informed decision-making.
Managers and business leaders specify targets. Engineers create estimates. A good estimate does not merely reflect our best guess about how long or how much work a project will take. Instead, it's an estimate that provides a clear enough view of the project reality to allow the project leadership to make good decisions about how to control the project to hit its targets.
Project schedules often slip because we allow the target to alter the estimate. A more productive approach is to use the estimates to inform project planning. If it's not possible to deliver all features by the target date, we could hold the date constant and deliver what is possible, or hold the feature set constant and push back the date.
Tips for producing accurate estimates:
Use multiple approaches to estimate the same task.
Acknowledge that the longer a project is, the more likely that an unexpected problem will arise.
Separate estimated work time from calendar time.
What frequently causes a project to slip is a fuzzy understanding of what constitutes success.
Setting a project goal produces two concrete benefits:
Building alignment also helps team members be more accountable for local tradeoffs that might hurt global goals.
Define specific goals to reduce risk and efficiently allocate time, and outline milestones to track progress.
As engineers, we like to build things. This tendency can bias us toward making visible progress on the easier parts of a project that we understand well. We then convince ourselves that we're right on track, because the cost of riskier areas hasn't yet materialized.
Effectively executing on a project means minimizing the risk that a deadline might slip and surfacing unexpected issues as early as possible.
Tackling the riskiest areas first helps us identify any estimation errors associated with them. The goal from the beginning should be to maximize learning and minimize risk, so that we can adjust our project plan if necessary.
One effective strategy to reduce integration risk is to build end-to-end scaffolding and do system testing earlier. Front-loading the integration work provides a number of benefits:
Our initial project estimates will exhibit high variance because we're operating under uncertainty and imperfect information. As we gain more information and revise our estimates, the variance narrows. By shifting the work that can take highly variable amounts of time to earlier in the process, we reduce risk and give ourselves more time and information to make effective project plans.
Trying to rewrite stuff from scratch -- that's the cardinal sin.
Rewrite projects are particularly troublesome because:
The second system is the most dangerous system a man ever designs.
Engineers should use a series of incremental, behavior-preserving transformations to refactor code. Rewriting a system incrementally is a high-leverage activity. It provides additional flexibility at each step to shift to other work that might be higher-leverage.
Sometimes, doing an incremental rewrite might not be possible. The next best approach is to break the rewrite down into separate, targeted phases.
Don't sprint in the middle of a marathon.
Reasons why working more hours doesn't necessarily mean hitting the launch date:
Tips for increasing the probability that overtime will actually accomplish your goals:
The benefits of code reviews:
Fundamentally, there's a tradeoff between the additional quality that code reviews can provide and the short-term productivity win from spending that time to add value in other ways.
Code reviews can be structured in different ways to reduce their overhead while still maintaining their benefits. Experiment to find the right balance of code reviews that work for you and your team.
How the right abstraction increases engineering productivity:
When we're looking for the right tool for the job and we find it easier to build something from scratch rather than incorporate an existing abstraction intended for our use case, that's a signal that the abstraction might be ill-designed.
Bad abstractions aren't just wasted effort; they're also liabilities that slow down future development.
Good abstractions should be:
Tests allow engineers to make changes, especially large refactorings ,with significantly higher confidence. When code does break, automated tests help to efficiently identify who's accountable.
Tests offer executable documentation of what cases the original author considered and how to invoke the code.
The extent to which you should automate testing again boils down to a matter of tradeoffs. The inflection point came when a simple unit test visibly started to save time.
Since our initial understanding of problems always will be incomplete, incurring a little debt is unavoidable. The key to being a more effective engineer is to incur technical debt when it's necessary to get things done for a deadline, but to pay off that debt periodically.
Simple solutions impose a lower operational burden because they're easier to understand, maintain and modify.
Having too complex of an architecture imposes a maintenance cost in a few ways:
People often say, "Use the right tool for the job" -- but that can also increase the number of moving parts. Does the complexity of having more parts outweigh the benefits of simplicity through standardization?
By failing fast, we can more quickly and effectively surface and address issues.
You can take a hybrid approach: use fail-fast techniques to surface issues immediately and as close to the actual source of error as possible; and complement them with a global exception handler that reports the error to engineers while failing gracefully to the end user.
Engineers automate less frequently than they should, for a few reasons:
Automation can produce diminishing returns as you move from automating mechanics to automating decision-making.
Idempotence offers another benefit that many effective engineers take advantage of: the ability to run infrequent processes at a more frequent rate than strictly necessary, to expose problems sooner.
Running batch processes more frequently also allows you to handle assorted glitches transparently. A system check that runs every 5 to 10 minutes might raise spurious alarms because a temporary network glitch causes it to fail, but running the check every 60 seconds and only raising an alarm on consecutive failures dramatically decreases the chances of false positives. Many temporary failures might resolve themselves with in a minute, reducing the need for manual intervention.
The best defense against major unexpected failures is to fail often.
It's important to focus on uptime and quality, but as we go down the list of probable failure modes or known bugs, we will find that our time investments produce diminishing returns. No matter how careful we are, unexpected failures will always occur. At some point, it becomes higher leverage to focus our time and energy on our ability to recover quickly than on preventing failures in the first place.
We can script for success and shift our decision-making away from high-stakes and high-pressure situations and into more controlled environments.
The higher you climb up the engineering ladder, the more your effectiveness will be measured not by your individual contributions but by your impact on the people around you. Thinking early in your career about how to help your coworkers succeed instills the right habits that in turn will lead to your own success.
Your career success depends largely on your company and team's success. You get more credit than you deserve for being part of a successful company, and less credit than you deserve for being part of an unsuccessful company.
A good interview process achieves two goals:
Tips for improving your interview process:
A good initial experience influences an engineer's perception of the engineering culture, shapes her ability to deliver future impact, and directs her learning and activities according to team priorities.
Quora's onboarding program:
There's a common misconception that being the sole engineer responsible for a project increases your value. When you're the bottleneck for a project, you lose your flexibility to work on other things.
Tips for increase shared ownership:
Meet and conduct a detailed post-mortem after a site outage, a high-priority bug, or some other infrastructure issue. Try doing the same healthy retrospection to projects and launches.
Ultimately, compiling team lessons is predicated upon honest conversation -- and holding an honest conversation about a project can be uncomfortable. It requires aligning behind a common goal of improving the product or team, and not focusing on where to assign blame. It requires being open and receptive to feedback, with the goal of building collective wisdom around what went wrong and what could've been done better.
Great engineering cultures: