#151 - 14 Jan 2023

Leaders choose their own problems; No is positive; Managers matter; Effective training and effective communication; Your first months as a director; Fail slow hardware

Happy 2023, everyone! I hope you enjoyed the holidays and that the new year has been good to you so far.

This will be a short newsletter while we all get back into the swing of things.

First, I want to share a success story from a reader who wrote in to share with the community:

[…] I used the feedback discussion from December to write a short but detailed email to each team member at the end of December. I told them what impressed me most and related it to how they helped the team overall. The emails were VERY well received and told me how they were feeling, so the feedback to them also was feedback to me. […] Thanks for the idea

That’s great! I’ve done this for a team as a whole, but never to individuals. People really want to hear about the impact they’re having, and the end of a year (or project or quarter or..) is a fantastic time to share that with them.

Secondly - I’ve gotten a lot of other questions and discussions about feedback recently. Feedback is something we all know that we should be doing, but it can be hard to get started with. On top of the list of resources I share here regularly, I’d like to put something together something on getting started with giving feedback specifically for our community. This would be along the lines of the getting started with one-on-ones resource I shared at the start of the pandemic.

Is that something you’d be willing to help review (and provide your unvarnished feedback on)? If so, in exchange I’d be happy to set up a series of calls with you to help overcome that potential barrier, and start giving more frequent feedback to your team members. Let me know if that’s something that would be of interest - as always, just hit reply or email me at jonathan@researchcomputingteams.org.

Now, on to the first roundup of the year!

Managing Teams

Leaders make their own problems - Jade Rubick

I talk a lot about prioritizing and choosing strategies as managers of teams. The reason for that is because it’s often up to us to set these directions, even though no one actually explicitly tells us so.

Rubick describes the realization he came to as a director, that it was up to him to set direction for his teams. We’re often hired as experts, and expected to handle more ambiguity than would usually be the case for someone of our seniority. That means even as first-level managers for our teams (especially research computing and data teams), we have the opportunity - really, the responsibility - to set priorities and goals for our piece of the organization.

A lot of managers and leads I work with hadn’t come to that realization before. They think they have to do everything, and so burn themselves sand their teams out; or they think they just have to execute on exactly the list of tasks the team was doing when they arrived. But our job is one notch harder and more ambiguous than that. It’s generally up to us to figure out what we should be doing as well as how to do it well.

Rubick passes on the advice his peers and manager gave him in this situation - to look at the organization’s situation from several points of view; to seek out and review big picture objectives; to review past progress against those objectives; and to start setting directions of your own, informed and given at least tacit approval by stakeholders.

No is Positive - Avishai Ish-Shalom


Every “yes” is a mindless, implicit “no” to many other more valuable things; Use “yes” sparingly.

Longer-time readers will remember the “stop doing things challenge” from a couple of years ago (#56, #57, #58).

Research isn’t a zero-sum game in general; one of the few zero-sum aspects of it is where we spend our time. Our and our teams time is very much finite. Every task we take on corresponds to at least one task we are implicitly rejecting because we will no longer have the resources to do it.

You can say yes to anything; you just can’t say yes to everything. Choose wisely.

To Retain Your Best Employees, Invest in Your Best Managers - Erica Keswin, HBR

Just a reminder that our work is important, and our work together in this newsletter, sharing ideas with each other, is an important way to help each other out.

Keswin’s article points out how much influence managers have on people’s experience of their jobs, and so how willing they are to stay. The article describes a number of initiatives companies are taking to support and train their managers; we… probably don’t get that kind of support. But we can share ideas together, at conferences and online.

Do you have any ideas about how we can use this newsletter community to do a better job of sharing ideas from the readership? I’d love to hear them - let me know at jonathan@researchcomputingteams.org.

Managing Your Own Career

Managing Your Career Without a Manager - Saswati Saha Mitra
Effective Communication requires more than just Good Communication - Matt Schellhas

In RCD, many of us don’t get a lot of input from our managers. Even if we do, our own career management is ultimately our responsibility, and it’s worth managing actively.

Mitra describes her own experience, focusing on her craft (the work itself), connections within and across the organization, better understanding that organization, and looking for stretch opportunities.

Coming from the world of research, most of us are pretty good at making sure that our craft - our skills - are maintained and grow. Maybe that’s IC hands-on work, or maybe you (say) subscribe to a newsletter on management! And we have a pretty good idea of what projects or efforts are high-visibility and could grow us and our careers in ways we would like.

But it’s easy for us to leave out the people-systems side of things. The connections across an institution (or across institutions), and really understanding how our organization (or our funding agencies, or other stakeholders) make decisions that affect our work. This side of things is really important, and rewards careful attention and investment in time. That’s especially true if you want to enact change (by for instance following advice from Hack Your Bureaucracy, #144).

Shellhas’s article talks about what effective communication looks like when you’re first talking with other teams, and then attempting to collaborate with them. Effective collaboration is more than just knowledge transfer; building collaborations means you’re trying to affect behaviour. That means how well you word something doesn’t really matter in and of itself; what matters is what happens at the recipient’s end of your communications.

Effective communication means understanding the others’ point of view; communicating in such a way to be valuable, relevant and actionable to them; and comunication in a timely way (before key decisions are made).

The Seven Dispositions of Task Management - Michael Lopp

Our tasks as managers and leads have a much richer lifecycle than just “not started”/”in progress”/”done”. “Not started” and “in progress” can cover a lot of ground, and being honest with ourselves about why our task is not started yet or is started but taking so long can uncover a lot. This is a super short piece discusses that more eloquently than I can.

How To Spend Your First 30 Days In a New Senior-Level Role - Lara Hogan
An in-depth guide to everything you should do in your first three months as a first-time manager of managers - Lena Reinhard
Becoming a Manager of Managers - Brandford Fults

Whether you’re thinking about becoming a manager-of-managers (the usual name, if not always title, for this job, is “Director”, but Director can mean a lot of things in our line of work) or just want to understand the constraints your manager is operating under, learning what the job entails is useful.

Hogan and Reinhard have very specific advice for the first weeks in this role (which echo my suggestions for taking on any new responsibility as in #139). Hogan calls it “Sponge mode” for the first 30 days; Reinhard talks about gathering information. The first task is to talk to seek out information, talking to everyone, and only once you’ve been listening a while look to synthesize. Reinhard particularly calls out your own boss as part of that, to understand their goals for your organization. She also spells out the next two months - finding support, understanding what you need to do vs your teams need to do, developing communications “infrastructure”, and start planning for next steps. Both articles are very good and well worth reading.

Fults takls about the job once you’re settled in. First The delineation between your work and your direct report’s work can seem blurrier as a manager of managers. As a manager of individual contributors, there was a clear delineation for at least some tasks between IC and manager work. As a director, your direct reports are also managers, so overstepping lines can be worryingly easy.

Also, just the scope of work is much larger. You’ll be increasingly expected not just to collaborate and coordinate with your teams, but across larger swaths of the organization. The time horizon is much larger, too.

Finally, by the time problems reach you, they’re already Big Problems; the ICs and the managers have already tried to fix them.

Product Management and Working with Research Communities

Ten simple rules for implementing open and reproducible research practices after attending a training course, Heise et al., PLOS Comp Bio

Heise et al’s paper describes some clear next steps for research groups to tackle to start working on open and reproducible science practices in their own group. They’re good set of steps for structuring a plan of change in practices of a team generally:

  • Shortlist possible practices to implement in your current project
  • Join a community to access expertise and support
  • Talk to your research team about what to implement
  • Prepare for resistance and address it constructively
  • Decide what to implement and create an implementation plan
  • Compromise and be patient
  • Reassess and adapt your implementation plan when circumstances change. Look after yourself.
  • Share best practices and lessons learned
  • Get credit and make your contributions visible
  • Seek out supportive future employers who value your skills and interests

But I think their framing of “after attending a training course” is particularly valuable for us to think about.

We learned how to train and educate people, by and large, in an academic context. Success was teaching people a foundational body of knowledge, part of the canon of our field, so they could build on that later. It was all about information transfer and checking for understanding.

But when we’re teaching in an RCD context, we want to do more than that - we want to change behaviour. We don’t just want the attendees to know about scripting/R/the scheduler/SQL/open science/whatever - we want the attendees to be able to go back to their labs and apply what they’ve learned right away. If it doesn’t get applied right away, the chances of it ever being applied drops precipitously.

So including a plan of action as sketched out in Heise et al’s paper as part of the open science course itself seems really useful.

What has your team done to make sure that training courses are as immediately act-on-able as possible? I’d love to hear ideas or suggestions - email me at jonathan@researchcomputingteams.org.

Research Data Management and Analysis

It’s not news, especially for people it directly effects, but NIH’s Data Management and Sharing Policy comes into effect later this month. All, not just the largest, new grants and contracts, will have to have data management plans and improved sharing processes for “any data needed to validate and replicate research findings, regardless of whether it is used to support scholarly publications.”

NIH is a leader here and we’ll see other countries and funding agencies policies coming into force in this and coming years. This is really exciting! And the policies “turning on” fairly quickly are going to mean a lot of work for local research data management teams.

Does your team have a strategy for handling an influx of incoming requests, and onboarding a number of researchers and research projects at once? I’d love to hear about it - hit reply or email jonathan@researchcomputingteams.org.

Research Computing Systems

Fail-Slow at Scale: Evidence of Hardware Performance Faults in Large Production Systems - Gunawi et al, ACM Trans. Storage

The more catastrophic the failure, the easier it is to find and reason about, generally speaking. When parts of systems “fail” by completing their tasks but slowly, that can be really tough to track down.

Gunawi et al examine 114 reports of fail-slow hardware incidents from 14 institutions (7 companies, 4 universities, 3 national labs), looking at disk, SSD, CPU, memory, and network. It’s a compendium of failure modes (and how different failure modes present - or worse, cascade), so it’s hard to summarize, but it’s worth a read if you’re responsible for keeping a system fully performant.

Illustration of fail-slow modes from Gunawi et al - permanent, transient, and partial slowdown, and transient stop


A thoughtful reflection on going from a low-process culture to a high-process culture, and what process is and isn’t good at.

How many computers are on your computer?

Signed distance functions, for 3D ASCII renderings, in 46 lines of Python.

Or if you just want to draw graphical pixelated lines, draw ugly lines fast.

Fx, an interactive terminal based JSON viewer.

Learn how CDNs work by writing one from scratch.

SQLite’s automatically generated indices; and writing an SQL engine from scratch.

Oh sure, you could make sure you never access an array out of bounds by constantly checking each access, like a chump. Or you could use GPT3 to extrapolate arrays beyond their endpoints. What could go wrong?

Lock-free, wait-free, and what does and doesn’t mean.

What Ada Lovelace’s computer program actually did (calculated Bernoulli numbers, and without any bugs!)

This text presents a method for embedding a programmable, general-purpose, digital computer into Tetris. It describes the capabilities and performance of an implementation that runs Tetris on Tetris.

That’s it…

And that’s it for another week. Let me know what you thought, or if you have anything you’d like to share about the newsletter or management. Just email me or reply to this newsletter if you get it in your inbox.

Have a great weekend, and good luck in the coming week with your research computing team,


About This Newsletter

Research computing - the intertwined streams of software development, systems, data management and analysis - is much more than technology. It’s teams, it’s communities, it’s product management - it’s people. It’s also one of the most important ways we can be supporting science, scholarship, and R&D today.

So research computing teams are too important to research to be managed poorly. But no one teaches us how to be effective managers and leaders in academia. We have an advantage, though - working in research collaborations have taught us the advanced management skills, but not the basics.

This newsletter focusses on providing new and experienced research computing and data managers the tools they need to be good managers without the stress, and to help their teams achieve great results and grow their careers.

Jobs Leading Research Computing Teams

This week’s new-listing highlights are below in the email edition; the full listing of 134 jobs is, as ever, available on the job board.