From Priority Queues to Priority Sprays

How data structures explain engineering leadership

I moved into a director role this year, and my VP, Mark, has been gradually adding more responsibilities to my plate as I ramp up. Recently, I had a conversation with him about stress, and we began discussing how we operate. I was talking about how I feel like I operate on a priority queue, where I'm constantly popping and pushing items on and off the queue, and he said, "I work on a priority spray."

This resonated immediately. Over the past six months, I've transitioned from pulling one item off the queue to prioritizing multiple high-priority tasks and working on them concurrently. Enter: the Spray k-Queue.

I started thinking of this as a Spray k-Queue – a concept inspired by probabilistic data structures where, at an arity of k=1, it operates as an actual priority queue. At higher arities, the queue becomes probabilistic by randomly selecting items from multiple priority queues. At k=1, you are guaranteed to work on the highest priority item, and at k > 1, you are guaranteed to work on some of the highest priority items. At this point, you're optimizing for throughput and not perfect ordering. This arity progression maps remarkably well to engineering career levels.

Lower on the engineering ladder, you typically work at k=1. You pull the first item from the backlog and work on it. Lather, rinse, repeat. This is what's expected, and it facilitates deep flow state. Your queue is ordered in accordance with business priorities, and that order is maintained by someone else or potentially by a team of people, including you.

As you move up the ladder, your k-value increases. At the senior IC level, you're likely to operate at k=2-3. You are doing your own work, and you're contributing to others' – likely through mentoring. Possibly, you're leading a small effort with one or two other engineers. This low level of context switching should be generally manageable.

When you move into engineering management, your k-value likely increases to 3-5. At this point, you're managing multiple team priorities. You have several engineers counting on you to help them maintain flow. You must ensure that the work continues to come in and receive priorities. 1:1s start to take up some of your time. And, you're still doing some technical work, either coding or leading at an architectural level.

At the director level, you're operating at k=5-8. This wider range reflects the variability in director scope across companies. A higher-order spray enables you to manage the priorities of multiple teams. You are likely leading strategic initiatives requiring significant coordination with timing sensitivity. Cross-functional projects require coordinating one or more of your teams with other organizations. Not only are you balancing your teams' priorities, but you're also balancing them against multiple competing business objectives.

At the VP level, you're working at k=10+. Concerns are now organization-wide. You're working directly with executives to lead strategic initiatives across the entire company. You require maximum throughput, and the ordering of your work matters less and less.

The level of context-switching as you progress upward requires artful execution – and this is where things can go wrong. When transitioning from being an IC to a manager, it is all too familiar to downshift to k=1 and stay there. While this may be acceptable during a ramp-up period or when tasks require deep focus, after a few months, you will become a significant roadblock for your team. You'll need to shift out of the strict priority queue mindset to lead effectively.

Similarly, some engineering managers can't slow down when deep work is required. Sometimes, you have to focus on the highest priority and nothing else. There may be a major impediment to progress for your team. It's your responsibility to remove that roadblock, and that may require focusing only on that for some time. If you're constantly allowing yourself to be interrupted, you become the bottleneck.

It's critical that, as either a technical or organizational leader, you can scale up and down in arity depending on context. This requires constant situational awareness and reassessment. Maintain the ability to accept input to the queue while shifting between multitasking and single-threaded executions.

You can see this at play everywhere. I'm in the middle of hiring entry-level to staff+ engineers. I've been looking for candidates with the appropriate level of multitasking capability. It's fine if entry-level candidates indicate that they prioritize a queue. It's a red flag if someone at the staff level asks if the team "suffers" from shifting priorities.

When mentoring or developing engineers, I must consider whether they're ready for the spray or not. Additionally, it's essential to work with engineers as they progress in their careers to help them understand that their ability to operate at k=1 consistently will diminish as they advance. You may be in the midst of executing a project, but you're likely planning the next one at the same time.

Almost every day, I have to reassess the arity at which I need to operate. I have a new hire right now, and I need to monitor her ability to get things done closely. My team manages a diverse range of services across multiple technologies. I need to make time to conduct in-depth reviews of each one so that I can provide practical technical guidance. That requires shifting to k=1 sometimes – I block off time as necessary.

While Mark and I may have stumbled across a great analogy, it's rooted in a critical problem in high-throughput systems. The SprayList, MultiQueue, and a few other implementations of this exist. They're lock-free data structures optimized for high throughput instead of strict ordering. You accept that you're not always working on the absolute highest priority, but there is fairness in these kinds of probabilistic systems.

At higher arity, you're ensuring that you're working on at least some of the highest-priority items. It may not be perfect, but it's not intended to be. We're optimizing for reducing distractions and increasing throughput for ourselves and our teams instead of constantly worrying about whether or not we're "really" doing the most important thing. You don't need to worry about resource starvation. You'll get to everything... eventually.