One final challenge of research computing is the breadth of what we’re expected to know and be able to respond to.
“Computing for questions that haven’t been answered before” is a pretty broad remit. Even in pretty tightly-scoped efforts like the one I’m working on in genomics, I’m routinely expected to have opinions on and possible approaches for any random thing a researcher could possibly want to do with health genomics data.
This is a problem that comes with a field’s success; when I began in research computing in the mid-90s, the problems that were amenable to the computing of the day were a fewer and less heterogenous. You could call a person a “research computing expert” and it would have a plausible and well-understood meaning - it probably meant familiarity in a Unix environment, with numerical methods knowledge, programming skills in both C and FORTRAN, and probably some domain-specific expertise in partial differential equations for some discipline.
We’re still expected to be research computing experts today, but even within our own bailiwicks that’s a pretty high expectation. There is a rapidly growing range of technologies: when I started, there was only one kind of database, now there’s easily a half-dozen entire rich categories; cloud computing (on- or off-prem) have made possible many quite complex architectures; containers, new programming languages, and emerging networking options offer new approaches to thinking about our solutions.
At the same time, research in our fields are keeping apace and wanting to ask new kinds of questions on new kinds of problems or data. Data volumes and diversity have been increasing so quickly for so long as to be a cliché, as have the power of simulations, but the complexity of the questions being posed are increasing rapidly too.
As managers of teams we don’t have to, and can’t, maintain a working knowledge of all of the research questions being asked and all the tools being used in our fields, and the methods connecting the two. But we do need to stay on top of the fields, detect patterns, and see openings; we need to be able to ask sensible questions when our team members or users propose something new.
Conferences are good for this, although conference talks tend to focus on what worked rather than the false starts that didn’t. Social media and blogs can help a lot for more informal communication, and many of us have found good and relevant online versions of our research and technical communities. But a lot of exciting ideas come from taking approaches from other fields and applying them to our own, and it’s harder to find places where we can hear what’s going on in research computing more broadly. Here I want to start discussions across disciplines about what people are doing, what’s working, what hasn’t, and why.
Thanks for following along this week’s emails. Tomorrow will be the usual link roundup, and next week I’ll talk about some technology trends in research computing that I’m following closely.
Best,
Jonathan
