At some point in the past few years, all of us were sold on the idea that boxing people into roles was not an ideal situation. It was not so much that we wanted every one to be a polyglot but that we wanted people to care about more than just what they had worked on. For example a developer should care about:

1. Page performance, error reporting etc when developing relevant parts of the application.
2. How the application she/he developed is performing in production.
3. How the infrastructure is created and maintained that serves as production.

so on and so forth.

What I have noticed recently (possibly could have existed long since then) is that while the developers are leaning towards understanding the state of the world as it exists post deployment, we are perhaps not allowing sys admins an opportunity to contribute pre deployment. The extent of their involvement as it exists is during the inception phase, designing the infrastructure (Thank you AWS) and possibly informing the development team of setting up endpoints for monitoring. At no point are we allowing them the opportunity to write code.

The unidirectional flow emerges from a reasonable assumption. Knowledge of how an application is going to be deployed and used is very beneficial for structuring the application itself at times. And a developer has a better chance of quickly pinpointing source of any and all errors that emerge upon usage. All these perceptions, and more have correctly started the thought process that a developer _should_ care and know more about things like deployment, monitoring etc. But its a two way street.

One of the other aspects of the DevOPS movement has been the refusal to silo people. And while the above reasons point out why silo’ing developers is bad, siloing ops is still happening to some extent in my opinion. I look around and I can count on one hand the number of sys admins who are writing code that does not have anything to do with an AWS stack, or a monitoring plugin. We are somehow still preventing operations from pairing on straight and simple business logic development. This is a case for many companies. A simple answer to this is usually an explanation of the workload that requires pure operational expertise. That usually happens when:

1. Sys admins are still treated as a separate pool of knowledge which feature teams regularly pull from in order to get work done. They are not embedded into feature teams as a resource for purely that team.

2. A business is doing so much work that it is impossible for a sys admin to be focussed on expanding her/his development skills. So they must concentrate on operational exercises and actually require help from developers to do the small hanging fruit while they do the more important stuff.

There are other reasons but the above are the main ones that I keep hearing. Both of these are not reasons but excuses that can be worked at in a similar manner to how we convinced developers that caring about prod is good. But the lack of that thought process is whats troubling. We are not even willing to think about our day to day work in a different manner which would in the long term allow sys admins to contribute more towards development.

A question was asked recently when I was discussing this issue with some people as to the benefit of a sys admin pairing on development that is not ops focussed. I see many:

1. Continuously working on different areas of the application would give a sys admin an in-depth idea of what was important and what was not. This can feed into monitoring prioritisation.

2. Applications can be structured better to deliver things like faster page times, better DB structures and more efficient use of external dependencies based on the experience and knowledge of the sys admin.

And many more.

There is also the case that we are neglecting the people who might want to learn more about software development. Not all, but there are definitely those who do want to. And if the reason why they cannot or don’t want to participate is that their workload does not allow them to, then that is an issue much bigger than propagation of dev ops culture. Hiring good sys admins is not hard. Hiring good engineers is hard. Hiring good engineers is hard and both sets of people, developers and system administrators fall into this category, IMHO, of engineers. If you simply wanted to hire anyone, thats not hard. Hiring someone good, well thats a different thing altogether.

I understand that there are obstacles to achieving the goal of helping sys admins contribute to writing non operational code but we should at least acknowledge that there is a gap and do something about it. Treat them as a any development resource just as we are starting to think of developers as possible operational resources.

Its time we made this stream bidirectional.

The following two tabs change content below.
Software Developer at Ardent rock climber, long distance runner and cricketer. Passionate about geometry, algorithms, persian poetry and test cricket. Loves to climb at the Sundeck Valley in Grampians, Australia.

Latest posts by Mujtaba Hussain (see all)

  • Valerie

    I was with you until you said hiring good sysadmins is not hard. It’s extremely hard. Like developers, you can easily and quickly get lots of subpar ones. I’m curious what you think makes a good sysadmin. I would include development skills among the necessary skills of a good sysadmin.

    • epiineg1

      *sigh* I had meant to change that line about a week ago when the exact sentiment was raised on a twitter discussion. Alas! I only now have gotten to it. Thanks for pointing it out again. I have now, I think, expressed my thoughts better wrt that sentiment. I have also left the previous expression there as a reminder to myself to word things better 🙂

      Pairing on application level code is the start of these discussions IMHO. Its where we do the thing that we end up maintaining. Does it have to be a case where everyone always pairs? Probably not! But I would stress that it start that way and then be discarded if proof of value is absent rather than not do it because it seems that it might not add value.

      • Valerie

        The edit is more in line with your sentiment in the rest of the article. Thank you for the clarification. And, yes, collaboration is a big start in fixing the divide.

  • Greg Collins

    Great article Mujtaba, we here at carsales are working in the same cultural direction

    • epiineg1

      Awesome. How are you approaching it? Also, have you guys considered engineers/developers on day pager?

  • Jeff Smith

    Great post! I think it’s a difficult role to fill and sometimes an even harder role to cultivate. Lots of systems admins I know got into it specifically because they hated coding. I’m not sure how much longer admins will be able to afford to have that sentiment. Mature OPS teams are a must for high performing organizations and DevOps is not just a great philosophy, but a great tool for creating a new wave of System Operators. Soon the only difference in roles will be domain expertise.