DevOps The Agile Way
This is a blogpost of my session in AICON2018. For those of you who are new to AICON,
"Agile Impact is the conference to learn about agile and digital transformation. With speakers from all over the globe sharing wisdom during the 2 days, you’ll come out more agile than you came in.Following the success of the first Agile Indonesia conference in 2017, we have decided to broaden the theme this year."
Disclaimer: This is a long post as I dint want to miss any element that I spoke about. So keep Calm and read on :)
Last week I had opportunity to share my experience from 5 of my most successful client deliveries and talk about the 5 key aspects of "DevOps The Agile Way". The storyline for the 35 mins session was to understand the impact
When process meets technology.
Agile Impacts DevOps lifecycle.
Change is good and Transformation is sustainable.
Quality and Speed equally Matters.
We should Practice what we preach.
Let's us begin with a quicklook the
What is DevOps ? - A practice, a process , or is it just getting dev and ops together, write automation scripts, or is it a role, a team, a tool, so many thoughts right.
As can be seen in the image above it seems like a giant beast that can overrun your entire delivery if not driven in the right direction.
So my version of DevOps is defined as above and the key areas are highlighted for better ease of understanding. It means connecting all the dots & not just devs and sysadmins a.ka.a Ops , but all the units responsible for delivery including business stakeholders and thats when we achieve the complete lifecycle i.e "Continuous Delivery" of quality product and services. The obvious next question that comes to mind is "How we do that".
So without delaying much let us begin the journey of the 5 lessons learnt while doing "DevOps The Agile Way" and First Up,
While I was preparing the deck I came across above picture which said "Estimates: Why its not same as signing a blank cheque" and the author gave a real life example to explain this situation. He asked and I quote,
"Have you experienced this before:
You want to fix parts of your car and the service agent tells you that he cannot commit on the cost, scope and schedule of the repair.
Nevertheless, he assures you that he will deliver your car in excellent condition. Would you be convinced? The first thought to cross your mind would be the expense and then, the estimated time of delivery.”
Something similar happened in my first client delivery project. Requirement was to setup a Secure Virtual private Cloud On AWS. Sounds pretty straight forward right !!
Well that's exactly what the planner thought while creating the one story with that one statement and marking it a medium sized one (3 pointer, 1 week).
And, what we actually ended up delivering was an end to end pipeline that would launch an AWS VPC with Client VPN Access with Dual factor Authentication all enabled through one click deployment.
Everyone were happy and proud of the development work until when it came to get client sign off on commercials. And guess what we see, "3 points" to translate our 6 weeks worth effort of design, development and delivery of a client entire platform.
Good news was that the deliverables were well accepted but downside was the scope didnot convert to right commercials . Hence the lesson learnt,
"Never leave your stories with blank or low estimates"
Estimates becomes more essential especially for Infrastructure Development. Time based estimates makes most sense for infra development stories. If you don't know what to estimate time box it so you know when to stop and roll back. Always record and review your estimates during your sprint cycle. If the story is too big or takes too long to complete divide it into sub tasks. Whatever you do , don't leave black estimates and avoid underestimating. Second Up,
Recently I was doing this client project where they had all the key elements needed for a platform, you name it and they had it:
Latest tools and tech stack
Devs and Sysadmins
Top Management inclined towards Agile and DevOps
Yet, there were so many downtimes and issues, and the infrastructure had become so unreliable that every issue was pointed as an infra issue. There were less development happening and more of fire fighting.
And the real reason was lack of plan and long term vision. In short they had all the ingredients but did not have the right recipe.
No matter what you do or how quick you solve an issue if you don't know where you are headed to you will always be lost on the way and end up being the good old "IT Crowd".
If you wanna see some real results of being DevOps, what you actually need is
An OKR(Objective & Key Results) driven plan and initiatives.
Do regular backlog grooming, story analysis and acceptance criteria.
Be more proactive than reactive.
Sounds Familiar Right, More like a daily question.
And so this is my favourite story especially because of the lessons and learnings.
This was a project of building an on premise private cloud and migrating the data from public cloud to that. Things were going very well until one day when it started happenning. Even a small change in a script would break the whole platform and we would not know why. Things worked fine on a dev laptop but failed in the test environment. These issues heavily impacted day to day application development as the infra became unstable and we had no mechanism to check why it was happening
That's when we decided enough is enough, let's take a step back and focus completely on testing and quality assurance of the infrastructure code.
P.S: We were already practicing TDD but application development only , infra testing was still manual.
We tool the initiatives (shown in the image) and build a framework for TDD of infra code and finally what we achieved was gold.
Every developer had a replica of production environment and it was followed in each and every environment where the code was getting deployed.
Deployment time came down from 15 minutes to 93 secs.
Release cycle became weekly to daily which was earlier once in a monthly only.
Find out here How we did it in our articles.
If you are still with me, you will be curious to know
How was all these possible at all? Where did we find the people?
Did we just do any special course or certification of DevOps ?
(A question that pops up very much in any meet up or conference nowadays)
To answers all these questions especially the last one I am here to share you with story no. 4 where we started from ground zero and reached cloud within a quarter
Very often I have seen infrastructure projects gets dropped or de-prioritised the most common reasons sited are
"We don’t have DevOps engineers, what do we do ?"
"Its too costly and time consuming to hire them."
"We have never done it before.Its too risky"
This very situation of lack of people with DevOps skills happened to one of my projects , and let me tell you the story of how we got from Ground Zero to Cloud in a Quarter with an house team of 6 developers with no prior Ops/DevOps skills.
The management was about to give up that project as they dint have people and hiring would take 2-3 months. So I decided to step in and take this challenge. I identified 6 people who were very interested in doing something different and interesting in infra development. We presented a plan of take 1 product live in 4 months with 3 Dev pairs.
Did you notice that I mentioned "Dev pairs" and not developers !!!
Eventually those Dev pairs dissolved into the larger development teams in turn enabling other team members resulting in :
Reduced Dependencies Increased Reliability
Cross Functional Teams
Poly skilled Engineers
And this was possible because we focused on capability building and enablement that thus completing the stories and practicing pair programming in true sense where one dev works as a navigator and the other the driver, instead of two persons working on their own tasks but for the same story.
Thats why, "You cannot have a DevOps Team, You should enable the right skillset and capability for DevOps lifecycle. If you have the will , you can build the skills".
All of the above practices wont that effective if you ignore the last but not the least now
We always plan for a smoother ride but reality hits hard and all your plan goes for a toss. So despite doing all of the above have you even wondered "why your infra upgrades or restructure not a priority for the business?"
"Why your CTO always question about cost for infrastructure development and always ask if this is necessary".
Now if you think you are the special case , then you are wrong , the Business owners need to know what they are spending on to approve it and if they don't have visibility of whats happening on the ground they will ask question.
Thats the gap that we have to fulfilled. How many times have your client CTO asked
"Hey , when is the next new infra update happening"
"Long time no see, I miss your infra demos , lets catch up soon"
you see him tell his subordinates
"Infra Development is top most priority, get them whatever support they need"
Well I have heard all of these in my projects. If you ask me what I did to get that kind of trust, here it is
Review and showcase your weekly sprint progress with your business heads as well not just your product owners.
Run retrospective at least a month and find out initiatives to resolve pain points.
Give your CTO access to story board and put that in a physical card wall so that when he takes a stroll around the floor he can just visualise the process.
Setup build radiators and show your build and deployment pipeline live on the floor.
This will build more trust with your business owners and stakeholders and keep them all connected to the ground reality and look at the bigger picture of successful sustainable delivery.
For Faster, Efficient & Continuous Delivery Of Quality Products and Services