Jon Scheele (00:00)
DevOps is considered a core capability in many technology organizations It had its origins in the concern that was expressed about having developers who create software, completely different career path and often completely different groups within the organization from the people who actually run it. And it aims to promote collaboration between those. But it's not always so simple.
To discuss some of these complexities and how to actually achieve the aims of DevOps.
I'm very glad to welcome Nilesh Gule to discuss. He's an enterprise architect, a Microsoft Most Valued Professional and a Docker captain. So welcome Nilesh.
Nilesh GULE (00:51)
Hi Jon, it's a pleasure speaking with you.
Jon Scheele (00:53)
Right, so the aim of DevOps is to increase collaboration between developers and operations. And this is pretty complex. The idea has been around for at least 20 years. I wouldn't say that everyone's achieved the optimum level of DevOps. We hear terms like "You build it, you run it", which some organizations aspire to. Can you sort of paint us a picture a little bit about where you see DevOps in the organizations that you've worked with and what they try to achieve with DevOps?
Nilesh GULE (01:46)
Yeah, as you rightly said, Jon DevOps is a set of principles, right? And if we look at some of the core tenets of DevOps practices, it's about collaboration, improving the culture of automation. And I remember when I started my career, we used to have this distinct set of personas or teams and I started as a developer.
But my role was more to just write code and then we would just give it to a release management team who would have their own set of tools and processes. We never really worried about how the software was packaged. This team would then take it. They would use some scripts. They would deploy it. And then when issues came, it became like a big investigation. Like where was the failure coming from or what was the root cause?
And as you start moving into, let's say, late 2010 or early 22nd decade of this century, we saw a lot of automation happening in terms of not just the software development, but also on the operations side. And that's where DevOps practices, they really picked up. Now, if I see I've been in the industry for more than 21 years, the things which we started back in 2005, 2006. They are very much part of the tooling now. But that time, if you could just build your software continuously, continuous integration in 2007, 2008 was a big topic.
Nowadays, if you see it just become part and parcel of the tooling itself, people can just click a button and say, this is my repository, build it and deploy it to my cloud provider or my target environment. So I think over a period of time, the practices and the tools have matured to a great extent. But as you rightly pointed out, the complexities of software development and the organizations itself, they pose a bit of challenge in terms of implementing some of the best practices. In my experience, I've worked with organizations which are, let's say medium size and even very large, where there are more than 10,000 developers as well in Teams. So I think the implementation part of DevOps, it varies largely based on how the organization is structured.
Sometimes you have teams which are co-located. So it's very easy for them to work out the collaboration part of DevOps. But as you move to distributed teams, you need to have those certain processes and some of these things which are part of DevOps, they have to be formulated more into like process guidelines and things like that. And that's where the differences come in terms of implementation is what has been my experience so far.
Jon Scheele (04:36)
So to me, the reason for having DevOps is to help with organizations become more agile. So agile is a concept that can be applied in any industry, in any job role or function, but in technology, it seems to me like DevOps is an enabler of agility.
But if we look at say the Agile manifesto, there are some things that seem a little counterintuitive because for example, in the Agile manifesto, it favors individuals and interactions over processes and tools. And you just mentioned some processes and tools. But another aim is responding to change over following a plan. And the aim of to achieve that in an agile organization, you need to go through very rapid iterations and cycles. So this is sort of the reason for having these processes and tools like continuous integration, continuous deployment to enable you to go through those cycles faster.
So how do we balance the needs of the automation, which defines and you could say it constrains you a little bit to follow a process with the aim of rapidly iterating.
Nilesh GULE (06:20)
Yeah, I think Agile is more about doing iterative development. it's things like shift left, for example. If I look at when I started my career, testing used to happen quite late in the development cycle. And now if you look at most of the matured organizations, they integrate continuous testing as part of their development life cycle.
So that's an area that's a kind of thing which I feel has helped as part of agile practices to make teams make those minor changes in their software development lifecycle, but still achieve the principles of agile software development. The other example I would like to cite is that of doing like the vulnerability scanning or code scanning as part of your continuous integration builds.
It's not necessary that you do it with every check-in, but you could do it periodically. So instead of, let's say, doing your quality checks and saying you need to achieve a minimum code coverage, you wait all the way till it goes to UAT, and then you start thinking about making
the unit test or writing unit test to achieve your thresholds for code coverage. If you have it as part of your nightly build, say for example, it could still help the team members to achieve that objective of writing quality code. Same goes with the open source vulnerabilities. Instead of waiting for the security team to do the scans just before the release goes out to production, you can integrate these kinds of things quite early in your development cycle.
And I think this is where the integration of processes and tools comes into the picture and they go hand in hand. We are not deviating too much from the standard patterns and recommended practices. It's just that as part of agile development, we are shifting them earlier in the development lifecycle, which helps everyone in the process.
Let's say you find out that you have a big vulnerability to address, you could take adequate actions quite early in development cycle instead of waiting just a week or just few days before the release. So I think that's in my personal experience, a good thing which has happened over the last few years that things like DevSecOps integration in the development lifecycle has helped to shift many of the things early in the development cycle.
Jon Scheele (08:50)
So that goes hand in hand with how you organize teams. So there have been, in order to achieve shift left, it means that you have to have some knowledge of the testing and security vulnerability scanning within the team that actually creates things. whereas previously you'd have a very large development team handing over to a very large testing team and being checked by a very large security team. Now organizations are trying to make all those functions embedded within a single team in order to achieve that. And we hear terms like the two pizza team. But when you do that and you devolve large organizations into these semi-autonomous, smaller, multidisciplinary teams, how do you make sure you're still covering things and that you're providing the tooling so that they don't have to invent everything?
Nilesh GULE (10:10)
Yeah, this goes back to my earlier thing which I was trying to highlight that it depends on the size of the organizations and also the maturity in terms of their processes. I think if someone is starting quite new, it's like a startup. For them, it's quite easy to adopt all the latest and greatest tools and technologies. But in a big enterprise, you might already have systems, processes, which are already in place. And change management plays a key role in this. If you have to move from, let's say, one source code management system to another, it could take you years to do that.
if it's a very large organization. So again, in my experience, I've seen different types of models being used when we talk about DevOps and SRE kind of teams. In a medium-sized enterprise where I was working for a big insurance company in Singapore, we had a shared services team which would do a part of the automations, like Terraform modules, for example.
They were built by this shared services team. And then the individual business units, they would consume this as a consumable. So like a team in Singapore would then build their infrastructure based on these modules. A team in Indonesia would do their own. Whereas on another side, which was a big banking organization, a big bank in Singapore, we had a slightly different model where there was an enterprise SRE team, which was taking care of all the tooling, in terms of what software governance tool was used, what tool was used for building the like Jenkins or if we were going with Bitbucket. So the tooling was managed by one team.
And then there was another team which was looking at the DevOps practices and this was within the business unit. So each business unit had their own versions of the DevOps team who would customize the DevOps patterns and practices as per their needs. So I think I've seen both these versions where you have a kind of shared plus a democratic model and in other players where it is much more structured.
So in my opinion, it really depends on the size of the organization. And again, it could go back to the Conway's law, which says that you structure your teams based on how the organization is structured. So it would very much depend, in my opinion, on the maturity and the size of organization, how they actually implement the DevOps teams and the patterns and practices.
Jon Scheele (12:50)
I think there are a couple of things there. So there's Conway's law, is organizations are constrained to build systems that reflect the communication paths within the organization. And I think you mentioned Parkinson's law, is a challenge in any work environment that whatever you estimate, work seems to expand to fill the time allowed for its completion. since the
Nilesh GULE (13:16)
Yeah.
Jon Scheele (13:20)
the term DevOps came about, there have been some additional nuances or movements within that. So the site reliability engineering movement that was just started by Google is about optimizing that practice of the engineering of how systems are operated and changed.
And then more recently, platform engineering is about making sure that developing the tools and processes so that developers can accelerate their productivity, which I think you hinted on that example that you gave about a team preparing tools for other people to use is an example of that.
So where do you see this heading in future?
Nilesh GULE (14:23)
I think it's an evolution as you rightly pointed out. I would say version one was about continuous integration. DevOps was version two. Then the SRE was version three. And now platform engineering is like version four of evolution of this automation and collaboration part when it comes to the developers and operators working hand in hand.
I see that the tools are becoming more and more mature and lot of these practices are integrated well within the tools. Earlier you might have had to use like two or three different tools to achieve the same. But now you see a platform like GitLab, for example, allowing you to build so many things just using one platform itself. You don't need a separate CI server. You don't need a separate continuous deployment tool.
This integration of various, I would say subsections of the platform engineering or SRE or DevOps practices, whatever you call it, into the tooling itself is one area where I see both the service providers as well as organizations maturing quite a lot. And it's become almost like acceptable nowadays that if you start a new project,
you would invariably start with DevOps integration and continuous integration, continuous delivery kind of pattern. Very rarely nowadays you see a project starting in 2024, which doesn't have a continuous integration enabled. And they're not trying to do something in terms of automating their infrastructure or using practices like the infrastructure as code and things like that.
So I think over a period of time, these things have just become normal. Like we don't think too much about them nowadays. It just assumed that you will start with at least these bare minimum things. So that's where I see as the tools mature and as the teams also mature in these.
different aspects, it's becoming just part of our life. Just like unit testing has become a part of developers life over the last few years. I see these platform engineering and DevOps SRE also becoming just a norm.
Jon Scheele (16:42)
Okay, well thanks very much for sharing that, Nilesh. So over the next few weeks, we're going to dig into some of these topics in a little more detail. Next week, it's automation. And so we'll discuss the CICD pipeline and automated testing and other aspects of that. And then we'll cover security and then observability. So I'll...
be talking to you about at least one or two of those topics in the coming weeks. But thanks very much.
Nilesh GULE (17:20)
Thanks, Jon.