#147 - 26 Nov 2022

What should we teach new RCD managers? Retrospectives are a powerful tool; HN response to ‘Consider Working on Genomics’; Gratitude works; Scientific website redesign bootcamp; University finances; AI4Science

I spent much of this week helping teach a small part of EMBL-EBI’s excellent Managing a Bioinformatics Core Facility course (Here’s the 2021 public materials). It was my first time participating, and I was really impressed. There were a large number of new, motivated, engaged core facility leads; the workshop was very interactive, with multiple exercises every day, lots of group discussion and sharing of experience, and a lot of material covered.

The audience being leaders of core facilities, much more time was spent on cost recovery models than would be common in research computing and data teams (there was even one brief section which touched on hourly vs fixed-price billing, which was fantastic to see - readers will know where I stand on this, e.g. #140). There was also a terrific series of exercises on service design, which also isn’t common to discuss in our circles but really should be.

But it was fascinating to see how similar the fundamental issues were. Many of the questions which came up would be very familiar to you, reader — hiring and retention, motivating staff, working with recalcitrant researchers, dealing with benign neglect from leadership, demonstrating impact, improving execution, prioritizing efforts, juggling the needs of a diverse and growing research community.

I think it’s pretty clear that in the RCD community we’re currently about five years behind our core facility colleagues on issues of building a community of practice and professional development, certainly for managers and leaders. Honestly, this is great news! We were much further behind a decade ago. The tireless and thankless work of groups like CaRCC and CASC have been pushing us ahead in terms of building a community of practice around research computing systems in particular, and in building an IC career ladder framework; and of course the RSE movement has been great for building similar things at least at the IC level for software.

I think we need to do more, in particular across RCD silos, but the fact that there’s been as much progress as there has been is remarkable and a testament to a lot of unsung work by a large number of people.

Still, it’s interesting to think about what we should be teaching new RCD managers. I have my usual 10-minute “Help, I’m a manager!” talks, but what would we cover in a half-week course?

My ideas: People, Projects, Products, Funding, and Strategy.  What do you think should go here?

My own ideas, pictured above, wouldn’t be surprising - it’s the topics covered here, plus funding (which is a bit too jurisdiction-specific to cover in this newsletter). People, projects, products, funding, strategy. Maybe also on communications/marketing/positioning.

What do you think? What would have helped you when you started, or what do you see newer colleagues struggling with? Are there other topics that would have to be covered? Things that could be cut that are covered well elsewhere? Let me know - hit reply or email jonathan@researchcomputingteams.org.

One last note on the EMBL-EBI course - the short section I covered was a crash course into the 20% of project management that gets you 80% of the benefit. I had what I figure are nuggets of hard-won wisdom gold in there, but what really captured the attention of attendees was routinely having retrospectives after projects, and continually improving. (Core facilities execute on a lot of projects, with each service provided being a project).

It’s pretty easy to underestimate the power of continual incremental improvement. These new leaders saw what I think more experienced and jaded leaders can sometimes be more hard-pressed to see: things can always fairly easily get a little bit better, and sustaining those small improvements over time compounds into something remarkable.

The lovely thing about retrospectives is that once you start taking action on suggestions (whether from the team or the researcher client) there’s a virtuous cycle. People are more willing to make suggestions when they see that suggestions actually matter; it builds trust and confidence, in addition to the improvements it brings. It boosts engagement because people feel more ownership over something they have influence over. People feel respected and more effective.

And internal retrospectives help the team decide collectively how they want to work. Issues with processes or conventions inside the team will naturally come up, and a good facilitator can help the team trial new processes.

Retrospectives are an under appreciated tool, and they don’t have to be a project management thing; they’re something that can happen regularly in any team.

Finally, just a small housekeeping item: we’ll have newsletter issues Dec 3, Dec 10, and Dec 17 as usual, and then RCT will settle in for a long winter’s nap. We’ll start up again Jan 14.

And with that, on to the roundup!

Managing Teams

Comments: Consider working on genomics - Hacker News
Consider Working On Genomics - Clay McLeod

I’ve been writing a bit on recruiting and hiring lately, and I’ve mentioned before that there’s a bit of a sense of helplessness in some RCD team leaders; they can’t compete with salary, so why even try attracting staff from outside of academia?

I’m not sure if this post is going to make people feel more or less helpless, but it will at least point out that salary is not the only or even main thing that matters to most of our candidates.

I don’t generally link to Hacker News - frankly, the comment section can be pretty toxic. If you aren’t familiar, the contributors here are by and large tech workers (software, systems, data scientists…), largely but certainly not exclusively from industry. “The Orange Website”s failure mode is that it can be very tech-bro-y.

But the discussion here is engaged and on-topic and fascinating. It’s in response to McLeod’s article, urging software developers to consider spending some time working in companies and research institutes working on genomics. McLeod points out, correctly, that there’s lots of great work to be done and not enough people doing it.

The hacker news thread is likely eye-opening if you haven’t heard these discussions from this audience before. It’s great to get a chance to “listen in” to a community having this conversation.

The comments from people who have tried working in research before moving to industry are scathing. Most poignantly, there’s a “more-in-sorrow-than-anger” feel to much of the discussion, coming from people who would have very much been willing to take up a career (or at least part of one) supporting research.

Salary certainly comes up, but it’s usually mentioned as an aggravating factor, something along the lines of “there’s no way I’m going to put up with X when they’re paying me so little”.

Coming up much more often is:

  • Lack of respect for staff from researchers
  • Lack of autonomy
  • Poor tooling
  • Job security
  • Poor state of code/infrastructure, with little appetite to improve things
  • Isolation from the research work: The largest centres silo production development from research in a way that likely makes things more efficient but makes the job much less interesting

We can flip these around and cultivate working environments that are attractive:

  • Close interactions with the real research work
  • Autonomy within their domain
  • Providing decent tooling
  • Respecting staff and their work output
  • Respecting their opinions when they say something needs to be improved

How to… make people happy - Ethan Mollick
Notes of Appreciation Can Boost Individual and Team Morale - Whitney Johnson and Amy Humble, HBR

I’ve mentioned research before (#64, #112) that there’s basically no plausible amount of gratitude we can express, to team members or broader community members, which is too much. Saying thank you in person, or in a short written note, takes approximately zero effort and yet is extremely impactful.

In celebration of US Thanksgiving, Mollick (a prof at Wharton) summarizes a paper on written expressions of gratitude:

[T]his paper shows we undervalue showing gratitude. We think it will be awkward, we think people know we are grateful, we think it won’t matter much. All of that is wrong. People who were asked to write letters of gratitude to other people overestimated the awkwardness of the experience, and underestimated the impact on the recipient’s mood and happiness.

There are other papers summarized, as well:

  • One points out that we also think complimenting people will be unappreciated and awkward, and it isn’t.
  • Maybe particularly appropriately for our line of work, another paper demonstrates that even if we can’t completely solve someone’s problem, people appreciate partial help almost as much as full help.
  • And a final paper covers spontaneously contacting someone just to say “hi” and catching up. Again, we overestimate how awkward and unappreciated it would be, where in fact it is generally appreciated, whether it’s someone we know well (strong tie) or less (weak tie).

Similarly, Johnson and Humble describe work they do at their leadership retreats, with everyone charged with writing notes of appreciation. They describe the results:

  • They help the recipients see their strengths
  • They focus the sender and the receiver’s attention on what’s working
  • They signal to people th atthey matter

Product Management and Working with Research Communities

How to redesign a scientific website in three simple steps with limited budget and time - Nikiforos Karamanis

One of the sessions in the core facility manager training was a very effective set of exercises on service design. The exercises themselves were enough to serve as gentle nudges to better thinking of interactions with the service from the users point of view.

The instructor shared a blog post he wrote using a stripped-down method of the same approach to resigning a scientific website, with (basically) no budget and a very modest investment in time:

  • Write a content model for the site
    • Identify audiences, audience goals, and our goals
    • Identify where people would be coming from to visit the site, what they’d want, and what they’d do afterwards
  • Observe or interview a small number of target users, develop a persona
  • Explore alternative designs using pen and paper based on the content model
  • Develop “higher fidelity” sketches of a subset of designs
  • Get feedback on the designs and iterate

The post is worth reading. It’s so easy to get caught up in how things are and our own internal goals. Going through this process confronts us with what the site visitor/user’s goals, and how things could be.

University Finances 2021-22 - Alex Usher, Higher Education Strategy Associates

It’s always worth paying attention to trends in our institutional finances. While our kinds of organizations typically aren’t buffeted as wildly by economic forces as others are, they aren’t impervious to them, either.

This article focusses on Canadian universities, but the pattern (if not the exact timing) is playing out elsewhere and in other kinds of R&D environments, too:

Income from government fell by 9% in real terms in fiscal 2022 – partly the result of the withdrawal of emergency COVID funding and partly the result of inflation. […] In any event: universities didn’t do too badly as a group in 2021-22 but there are good reasons to think that 2022-23 will be substantially worse as income sources lag inflation.

The broad strokes were foreseeable (and foreseen) some time ago. Certainly, experienced nonprofit leaders saw this coming. When sharp downturns happen, emergency funding (from government and/or donors) is followed by retrenchment, and that retrenchment basically always happens well before the downturn is completely over. Today’s inflation and geopolitical uncertainty amplify the impact of this back-swing of the pendulum.

As institutional budgets tighten, teams that can not clearly communicate their value to core institutional missions in ways that decision makers care about are going to have a bad time. Large regional centres are probably going to be ok, and few-person teams are probably going to be too small to attract much attention. Medium-sized generalist teams in public institutions are going to have large enough budget lines to catch the eye, without necessarily having a ready-made compelling narrative supporting their work.

If you’re in that situation, you probably have multiple decision makers that affect your budget. It isn’t too late to start talking with them, to make sure they can be armed with justifications they individually find compelling for your team’s budget and impact, should tighter budgets come. For some, that compelling justification will be quantitative, like grant numbers. For others it will be qualitative evidence of you enabling research projects they personally prioritized, maybe with a few images and pull quotes from the researchers involved. Others may value something else entirely (Contribution to the education mission? Workforce development? Community outreach?) It’s only through conversation that you’ll know what matters to them.

The goal is to have one or more decision makers actively championing your work, and for the rest to be at least lukewarmly supportive enough not to advocate cutting your budget.

Does that make sense? Do your decision makers have other things they care about? Are there areas you’re concerned about? Hit reply or email me at jonathan@researchcomputingteams.org to share your thoughts or to ask me questions.

Emerging Technologies and Practices

AI4Science report - CSIRO

Australia’s CSIRO has a nice report covering the increasing role AI methods broadly are playing in the sciences. There are some nice plots by field, and some (Australian) highlights.

Section 6 covers what will have to be done to continue to support this, with implications for research organizations: increasing software, hardware, and open access resources; better data; education & training; better workforce diversity; and likely upcoming ethics and regulatory issues.

This report could be a useful starting point for advocacy with your own decision makers, as our teams start wrestling with the increasing AI needs and workloads in our research communities.

Table 2 of CSIRO’s “AI for Science” report, showing a heat map of publishing in fields from 26 coarse-grained fields, from 1970-2022, and the fraction of AI-enabled papers.  Other than obvious fields like CS and EE, 7.2% of Physics & Astronomy papers, or 7% of Health professions papers, or 8.3% of neuroscience or 5.4% of chemical engineering papers in 2022 (through to Sept) appear to be AI-related in some way.


Oh My GitHub - work with GitHub (including creating PRs) from within Emacs, if you’re into that. I won’t judge. (I lied, I’m 100% judging).

You like having nice clean git history? How about having your commit hashes listed as “00000000”, “0000001”, “0000002”? (Ah, I miss SVN). Now you can, with extremely linear git history.

Meta is open-sourcing their new git-compatible source control client, sapling. Interesting to see what can be done without sacrificing existing git tooling.

RSS meets Usenet: rssnix, collecting RSS/Atom/JSON feeds to the local filesystem, with a browser.

Enjoyed geoguessr? Codeguessr shows you code snippets from one of the top 200 most starred GitHub repo and asks you to guess the project.

Another datapoint that “modern”, “innovative” workspaces (open concept, beanbags, etc) don’t promote creativity or collaboration.

Pretty good argument about why FOSS so often has bad UI/UX - the very distributed decision making that comes with successful FOSS projects are at odds with the holistic, coherent design process that UI/UX benefits from.

Nice to see projects like Flux that try to extend Rust’s weirdly narrow definition of code safety, to include focus on correctness with Ada-like pre-post conditions and refinement types.

With SSDs and much higher-speed interconnects, old assumptions like “I/O is slow” isn’t true for local disk streaming access and hasn’t been for years. (This IMHO is the biggest difference between HPC with parallel POSIX filesystems and workstation computations - not MPI, not scaling concerns).

Good quick introduction of of Kubernetes, RDMA and OpenStack using ROCE and SR-IOV.

I’m unreasonably excited about advent of code starting soon.

That’s it…

And that’s it for another week. Let me know what you thought, or if you have anything you’d like to share about the newsletter or management. Just email me or reply to this newsletter if you get it in your inbox.

Have a great weekend, and good luck in the coming week with your research computing team,


About This Newsletter

Research computing - the intertwined streams of software development, systems, data management and analysis - is much more than technology. It’s teams, it’s communities, it’s product management - it’s people. It’s also one of the most important ways we can be supporting science, scholarship, and R&D today.

So research computing teams are too important to research to be managed poorly. But no one teaches us how to be effective managers and leaders in academia. We have an advantage, though - working in research collaborations have taught us the advanced management skills, but not the basics.

This newsletter focusses on providing new and experienced research computing and data managers the tools they need to be good managers without the stress, and to help their teams achieve great results and grow their careers.

Jobs Leading Research Computing Teams

This week’s new-listing highlights are below in the email edition; the full listing of 192 jobs is, as ever, available on the job board.