What skydiving taught me about automation

Back in the day, before I was a father, before I was a husband, back when I was more of an adrenaline ‘junkie’, one of my ongoing adventures was skydiving.  I wouldn’t necessarily call it a hobby as I never followed through with a certification, but I took my fair share of jumps.  When I first started I would go with either a ‘static line’, where a cord hanging from the plane would pull my chute, or go on what is called ‘tandem’. A tandem jump is where a professional skydiver or ‘guide’ is securely attached to your back and they take control of releasing the parachute and safely bringing you down to the ground.  

The lesson of over confidence 

After doing a couple of these initial jumps, I was in a specific skydiving club’s prep facility at a small airport in East Troy, WI. I was becoming a little too confident with myself, having now done some of these beginner jumps more than a couple of times. I was going to be performing another tandem jump, as some of my family members wanted to join. The professional jumper that was going to be attached to me started training me and showing me how it all works. My overconfidence kicked in and I let them know that I clearly knew what to do and none of this was necessary. I could tell they were annoyed with me, but I didn’t care, I just wanted to get up there and jump out of the plane. 

So our time came, and up into the air we went. It was eventually my turn and my ‘guide’ and I worked our way to the sliding door on the side of the plane. The door opened and as I stood at the edge of the open door I quickly took a look out into the clouds and down at all the farm fields below. All of a sudden I hear my professional guide say “Hey… that doesn’t look right”. I turned my head to look behind me and see him unrolling a small, maybe 3 inch piece of duct tape, ripping it off and putting it on one of the straps that connects me to him. Remember, he's the one with the parachute and I’m supposed to be connected to him…. And he’s taping it together?

I yelled “Wait… hold on… what are you doing…..” but it was too late, he cut me off with a quick “1..2..3…Here we go” and we rolled out of the plane. Down we fell, hundreds and hundreds of feet in the air, myself screaming… not even clearly thinking that this guy is still connected to my back. He pulls the cord, the chute comes out and we both hang there, dangling under the parachute, slowly floating towards the ground. At this point my connected ‘guide’ was just laughing and laughing as we sailed through the air. After he was able to catch his breath from laughing so hard, I asked “What was that???? Are we connected by tape right now???” And he calmly just says… “listen, you were getting a little too confident in your training. You can’t just glance over things in skydiving… I was just giving you a little scare”. Fair enough… I learned my lesson… or so I thought.

The lesson of an SAP Kernel upgrade

A mere two weeks later, I was in the office at the SAP managed services company I was working for. A high security threat came out for most SAP landscapes and it was suggested we run a SAP kernel upgrade across all environments to patch this vulnerability. I had notified my customers, and one of them came back immediately asking for the kernel upgrade as soon as possible. They wanted it in their entire landscape that upcoming weekend. They could only afford 30 min of downtime Sunday morning between 1:00am and 1:30am for production. I thought, no big deal, I’ve done a kernel upgrade a million times by now.  I’ll wake up at 12:45, do the kernel upgrade and be back in bed by 1:15am. I quickly downloaded the kernel, prepared it on the server and off I went with my day, thinking nothing of it.

Sunday morning came around and I popped right up at 12:45am, ready to go;1:00am rolled around and I started the kernel upgrade process. It looked like everything was going well… until the systems wouldn’t come back up. I immediately went into panic mode. I started reviewing everything, trying to figure out why the system wouldn’t come back up.  “It’s a simple kernel upgrade, the process isn’t that difficult, what could be going on???” I read every log, I tried starting/stopping the system more than once; I was lost. The last thing I wanted to do was call one of my co-workers in the middle of the night and embarrass myself by explaining that  I couldn’t complete a kernel update. But I had already wasted 20 minutes of a 30 minute maintenance window. Rolling back to the former kernel was not an option as this customer was very security conscious. I decided to give all components another glance and it was right there; I realized I had the wrong kernel downloaded for this system. It was a rookie mistake. I had, once again, become overly confident, didn't pay attention during my preparation and it came back to bite me…again. It was exactly what my skydiving guide had warned me about. I quickly went out to SAP’s support site, downloaded the correct kernel, completed the upgrade successfully and handed the system back to my customer just 5 minutes after I was supposed to.  
My lesson had been reinforced. Even the simplest of tasks, such as a kernel upgrade, can go horribly wrong if you don’t pay attention. As I went into the office the following Monday I notified my boss that I had cost the business about 5 min of unplanned downtime on top of the 30 min of approved downtime. He laughed when I told him and expressed that even the best of the best Sr. consultants (which I was not) make mistakes… It's human error.  “We can’t all be perfect all the time”.  

Looking back at how far technical SAP automation has come

Years later, I look at how far technical SAP automation has come. I can’t necessarily say that ‘We can’t all be perfect all the time’ is still a valid excuse for human errors when it comes to kernel upgrades anymore when you have automated solutions like Avantra managing and automating your kernel upgrade processes. Those like the former me, overly confident in their capabilities, can no longer get caught up in tough situations because we are no longer involved in these scenarios. You see, it’s not just the kernel upgrade itself that's automated, it's all of the pre and post processing steps, including the verification that you have the correct kernel, making it a foolproof plan.
  
Can SAP kernel upgrades become complex? Sure, in large high availability environments, but most of the time it’s a pretty straightforward process. However, with that said, human error can still take place. It could be like my situation, a legit mistake, or even a confused Jr employee..  Whatever the reason, it’s important to start thinking about an automation strategy if you have not already done so. Kernel upgrades are just the start, there’s a world of technical SAP automation out there. To find out more about how Avantra can help in your automation journey please reach out to us.