1. DevOps – A Journey into Open Source Tools

Once a year the team of Evangelists I work with gather together and spend two days engaged in a variety of different pursuits. This year we made our way to the Microsoft Research Labs in Cambridge.
Whenever I make the journey to this lab I do so with trepidation. I work in a large team of very clever people there are many degrees, several Doctorates and number of patent holders amongst them, but when we enter the hallowed ground that is the MSR Lab, we all feel a little awed in the presence of a large number of people who on the surface appear normal, ordinary types but actually have brains the size of a small planet and what’s more use those brains to develop useful technology for us all. These technologies include

Kinect
Bing Translator
Clutter (For Office 365 Outlook)
And many more.
So for our humble team to turn up and expect to produce something comparable in the inaugural UK DX DevOps Hackathon was always a stretch too far. One thing is for sure though the two days were going to be fun, noisy and full of problems just dying to be solved by the wonderful world of DevOps.

First – What is DevOps, why do we need it and how do we use it?

DevOps has a Wikipedia definition of

DevOps is a software development method that emphasizes communication, collaboration (information sharing and web service usage), integration, automation, and measurement of cooperation between software developers and other IT professionals. The method acknowledges the interdependence of software development, quality assurance (QA), and IT operations, and aims to help an organization rapidly produce software products and services and to improve operations performance.

Well that is a mouthful of long words.

I describe DevOps as common sense and the way that all organisations should always have been managing their business. Rather than spend many of my words (in a self imposed but small all the same word limit) explaining the theory, have a read of my colleague Susan Smith’s blog post on TechNet from January this year.

So what was the aim of the two-day event?

The department I work in is primarily staffed with Developer Evangelists with the aim of assisting Developers to use the Windows platform for their applications. There are a few of us that are IT Infrastructure evangelists and the exercise was devised to skill us up as a team to be able to help our customers to embrace the culture that is DevOps to allow for more agile and robust application releases and development.

Essentially we were to have an idea, create a solution and make sure the process used a number of the DevOps practices, with the aim of the solution being publically available in a Git Hub repository.

 

As a team we can then visit customers and help them to embrace the culture and practices in their own organisations.

That was the theory.

The first hurdle was getting to the Lab in Cambridge. If you have ever driven in Cambridge you will understand just how dysfunctional the transport system c

an be if you are unlucky enough to travel on 4 wheels. I rode in on my trusty BMW motorcycle but others used, trains, park and ride and even taxis from our hotel on the outskirts of the town. Huge queues and lots of road works led to timings being rather flexible!

 

 

 

 

 

 

 

 

 

 

After an excellent briefing from our external and very enthusiastic facilitators we got to choose our preferred project and team. This was surprisingly painless and I didn’t suffer the ignominy of being the last to be chosen, but only because each team had to have a mix of Developer and IT Pro evangelists.

I chose to work on a project to help our resident education / gaming Evangelist Lee Stott to produce a solution to enable greater use of a student DreamSpark Azure subscription, this would allow them to step through a manual process for creation of a website and a MySQL (ClearDB managed) Database. Once this was done the lessons had been learned and the student can then take advantage of an automated system to deploy future such solutions and learn the DevOps practices whilst doing so.

(DreamSpark is Microsoft's free offering to students of all ages to help them
gain access to software services and help to learn to use and develop with the 
Microsoft platform tools).

Other projects included a mixture of IoT gadgets from our resident IoT gurus Paul Foster and Bianca Furtuna and a project to allow easy updating of an ecommerce website by Martin Kearn. The final project was a Microsoft Band integration solution cunningly entitled Band on the Run.

I was slightly nervous working in the company of such awesome developer characters as Martin Beeby , Amy Nicholson and Lee Stott and this was borne out as the first suggestion in our team huddle came from Martin and went something like this.

“Why don’t we use as many non-Microsoft technologies as are available in 
delivering this as we can?”

So there we have it, the rest of the two days became filled with acronyms and products I had never heard of before and had almost no likelihood of understanding even if I had heard of them.

So what was our proposed solution and how did we plan on getting to that position.
Well we all (our team consisted of six evangelists) used a Visual Studio Online account to share the team room for our Student Continuous Integration project. We used VSO to manage the tasks and backlog for the project and also integrated it into our chosen method for team communication and integration with Git, VSO and other development tools. This was Slack, I had never heard of Slack and the phrase “Slack is the new black” was soon the team tagline. In short it is an instant messaging tool (which we already have plenty of, right) the big selling point (it is free) is the quantity and quality of the integration options available for your other tools.

We allocated tasks and did some whiteboarding of the architecture and since I am a OneNote fan I created a shared OneNote so that the team had somewhere to dump screenshots and other great resources, links etc. to be able to write this up and prove our DevOps chops to the facilitators at the end!

We then placed all our code and scripts and other interesting things in a public Git Hub repository.

This post was designed to be process based rather than technical but it will help to explain that we did have a few difficulties due to the limitations placed on the free Student azure account.

A student is given access to the new Azure portal and can only use the Azure Resource Manager features. This means that a number of our chosen methods and solutions simply would not work.

Martin chose to use a whole bunch of open tools to develop, deploy and amend the Website part of our solution.

These included

Gulp described as the streaming build system to automate your workflow

Grunt described as a JavaScript task runner – in one word automation

Yeoman described as the web’s scaffolding tool for modern web apps.

Slack  an Instant Messenger that integrate with everything
There are probably many more but I didn’t pick up on them due to my tasks. I was asked to deliver the PowerShell scripts to enable the automation of the solution to function. This was a pleasing task without a pleasing outcome. I failed in the time allocated to deliver anything that would work within our restricted environment.

At the end of day one we all set off across Cambridge on foot (the quickest way around town I reckon) to find the allocated restaurant. This was achieved much like the days tasks almost on time. After a great meal, great company and a comfortable night at rest we carried on with our DevOps hacking the next morning.

Our facilitators Damien and Alex who had travelled from deepest Europe to help us were seemingly impressed with our efforts and that we had come back to have another go! We even started half an hour early.

Sadly, for our team our restricted environment meant that we were not able to win the competition or to reproduce a solution we were proud of. What we did do was fully examine the tools available and the processes we used to manage a short fast project.

Just before lunchtime we had an opportunity to present our solution and listen to the others as well. My initial thoughts were that the four teams had delivered some really great results, all of which are still works in progress but will assist us and other customers to consider the DevOps culture in their future deployments.

I ought to mention that my colleague Andrew Fryer was part of the winning team and has or will write up their ‘victorious’ project in detail.

Also expect to see many more DevOps focussed posts here and elsewhere.

Watch this space.