DevOps Resources

I have been following the DevOps "movement" since its inception. Like any cultural meme that has value it has led to thousands of blogs, podcasts and now books on topics that are related directly and indirectly to it.

I've added a page on my blog linking to some podcasts that I have been following (on and off for some them), which might be of interest to somebody working in Technical Operations, Infrastructure, System administration or managing and architecting a cloud hosted product. It's linked from the top of my blog header - and also from here.

Multiuser SFTP server setup - the solution

I had to setup an SFTP server on an EC2 instance recently, with multiple users chroot-ed into their own directories (with access to only those directories), and a different set of ssh-enabled users, with key based authentication for sftp as well as ssh.

My first instinct was to do a Google search. Many links came up, none of which solved the complete problem. Some of them did not work (different Linux distro/version) and some ended up disabling ssh when I got sftp working.

I finally found this blog post -

http://blog.famzah.net/2011/02/03/secure-chroot-remote-file-access-via-sftp-and-ssh/

It's the only set of instructions that actually worked, with all the constraints mentioned above.
For the record, the OS was Ubuntu 12.04 LTS. An additional step you need to take on this OS is to disable apparmor, or the ssh stops working after a reboot. I am not a Linux wizard, so I don't know yet why this happens.

On a related note, it turns out that a common mistake that many make is confusing FTP over SSL/TLS with SFTP. FTP over SSL is just FTP over a secure connection, while SFTP is a completely different protocol, with the file transfer happening over an ssh connection.

Visualizing EC2

I hacked up a small application over the weekend to visualize EC2 instances at one place. Python/boto/web.py does the work of retrieving the data and JS (GraphDracula) displays it.

The need for this rose at work - where we use EC2 to host our infrastructure. The original intention behind it was an easier way to view things like

• Which availability zone has which instances
• Open ports between security groups
• Which security group has which instances
• and so on

Not all of this has been done yet, but will be.
Here it is on Github - https://github.com/talonx/ec2viz

The operational mentality in software development

A talk by Theo Schlossnagle spurred this line of thought. The description of the talk is "about the evolution of a career in web operations", but he talks more about the importance of thinking operationally by developers. In other words, he takes a different stance about the meaning of DevOps than what is prevalent. DevOps is usually described as increased collaboration between development and operations, with knowledge sharing between the two groups leading to better delivery.

Theo Schlossnagle is of the view that developers need to take the operational view when writing code. It's a very valid point of view and something that I suspect most of us overlook.

I'm going to expand on what he said and share my ideas on that.

Let's take the software you're writing now (assuming it's web software). Is it operable?
It probably does atleast these things

• Fulfills your requirements document
  
• Passes your unit tests and integration tests
  
• The UI is usable and responsive

But is it operable? Once it's deployed, can it survive unprecedented load? Fringe cases? Subsystems going down? Third party services it depends on becoming unavailable?

And present a front of graceful degradation as it does all this?


Selective Failure
Most of the time, we stress systems before deployment, using load tests to simulate real world conditions. That takes care of one aspect. But most of us don't think of failures of selective systems, especially when the system is distributed and its components interact in complex ways. The latter is true of most big web applications.Handling selective subsystem failures is not purely an operations responsibility. The application has to be written keeping selective failure in mind.

In the video, Theo brings up an analogy with security. Security is not a feature, but a way of thinking.
Sanitizing user input before putting it in a database query is not a feature.
Not allowing access to your internal web services is not a feature.
These are security restrictions you automatically think of when you develop.


In the same way, operational thinking should not be a something postponed till deployment while writing code. It should be de rigueur in the design and development process.

Some string matching

This puzzle from Facebook's engineering puzzles page is a good introduction to string matching.
A solution is possible using an algorithm called Levenshtein distance.

Let's take the example on the puzzle page -

TIHS SENTENTCNES ISS NOUT VARRRY GOUD

It's a munged up version of

THIS SENTENCE IS NOT VERY GOOD

The original problem statement involves minimizing the score (number of changes necessary) to transform each word in the munged up sentence into words that are in a predefined wordlist. It says nothing about grammatical correctness of the result.

So going by that, the score and the output sentence from my implementation are respectively, 8 and

TICS SENTENCES IDS NOT VAGARY GAUD

From the standpoint of the original problem, this solution is correct.
But not from a grammatical point of view. After some poking around, I realized it's not possible for this program to spew out the correct version of the sentence without it being aware of English grammar rules.


P.S. Without getting into grammar correction, of which I have no idea how to implement, I made a small change in the program as an experiment. The second version favours a smaller word whenever the scores for two are equal.
And the result is (Score remains the same)

TIS SENTENCES IS NOT VARY GOD