Dreaming in Code

I finished reading Dreaming in Code last week. It's Scott Rosenberg's account of a software development team's effort to build the ultimate Personal Information Manager (PIM). Led and funded by Mitch Kapor of Lotus 1-2-3 fame, the team goes through endless cycles of redesign, people issues and other upheavals.

Rosenberg follows the team very closely, participating in their meetings, interviewing them and filling the narrative with his own insights.

If a developer were to read this book, s/he would recognize the events in it this as standard stuff when developing a product, and s/he would also identify with the team itself, with the occasional shaking of the head :)

The software they set out to develop was based on Kapor's vision of a game-changing PIM. They called it Chandler, after one of Kapor's loved crossbred dogs.



This was the pre-Web 2.0 era, where most things, including product releases took longer to happen. The browser had not yet become the first medium for delivering any consumer facing application. Desktop applications for collaboration still looked as if they had potential.

Chandler's ultimate aim was to be a all-in-one tool where users could access their email, personal notes, calendar events and to do lists through one interface. Kapor visualized all these different data collections as isolated information silos. Chandler would break open and combine them. It would let you view and move around all these disparate things in a unified manner. You could tag a note with labels, and then put a date on it and make it a calendar event, or send it off as an email. Grand vision? Yes. It's something I would personally prefer to have around to manage things.

Due to numerous reasons, Chandler was late in delivering its promise. Gmail came in 2004, and opened the floodgates of what was possible using new Web technologies. Firefox arrived, and forced Microsoft to relook at IE (and make it better). Chandler was yet to release its 1.0 version. The browser was fast overtaking the desktop.

We have been spoilt by the seamless interconnectivity provided by the Internet - it's always there - home or office or a mountaintop - and with it, our data. Chandler did attempt a Web interface where you could access your data (which would be synced between your desktop and a central server), but it was too late.

The Chandler project is still on with many faithful developers. I tried out the latest version after finishing the book. My impressions? It still has a long way to go, going by the number of bugs I encountered, and assuming it can convince the Web 2.0 crowd to switch to a desktop application by the sheer force of its design and the things it is supposed to do.

The most interesting chapter in the book is "Methods" - where Rosenberg does a brief and very readable survey of the evolution of development methods over the years. This chapter can be read standalone even if you don't read the whole book.

Note: The Chandler logo image linked to here is used under a Creative Commons Attribution License 3.0.

Consistency in Development

Consistency in Development?

Simply put, it means following a set of basic guidelines in all development activities, from coding to deployment. This does not imply having rigid protocols and processes, because immutable rules don't help development but obstruct it. What it does imply is having simple, tried and tested conventions and some formal processes that people are comfortable with - 'Whatever works best for the team'. The key to getting the most out of consistency is to arrive at these rules by consensus and making sure everyone follows them.

It has to start at the very bottom.

Coding Standards
I cannot stress this enough. While there is no such thing as a perfect code convention, there should definitely be an agreed upon convention for a team - where everyone follows the coding standards agreed upon. This is true for any language. Imagine the plight of a developer who has to work on code originally written by someone else, and it takes him hours to figure out what the code does because the coding style followed by the original author is completely different. Junior developers often don't get this. The standards can be chosen democratically by involving every member of the team, freezing the conventions and applying it to everyone's IDE (Most IDEs support code style import/export). Ideally I would trust the developers to follow this, but it can also be enforced at the source control level where style checks can fail a checkin in case somebody messes up.

           Speak the same dialect so that others understand you and vice versa.

Maintenance becomes much easier.

ArchitectureDon't run and start to hack away the moment you get the requirements. Stop for a moment - think awhile. Put your thoughts on the whiteboard and discuss with your peers. Run through your design with somebody who knows the big picture. Come up with atleast two or three different solutions to the problem - that way you would know that you have looked at it from various angles and chosen the right one.

Application StructureYou might have multiple web applications in your project, differing in configuration files, HTMLs, images, css, libraries, server side scripts etc. Storing them in a consistent manner across applications helps keep them organized and makes it easier for developers to find things, especially new ones. You will not be surprised by the completely different disk layout of an application if it follows the same directory structure as all the others. Deep down it also appeals to the organized mindset that most good developers have.

Issue Tracking
New bugs/features come up every day. Small teams can probably manage these for sometime with sticky notes and paper and pen. Some developers have their own ways of keeping track of their ToDo lists - but without a unified interface, you are not going to scale. You will have chaos - files missed in checkins, people clueless about who is looking into a particular issue, wondering about the status of a critical bug. When your team grows large, you need some kind of tracking system to track milestones and open issues, tasks to do, assign issues to developers and prioritize them. There are lots of good bug tracking systems out there - get one which works best for you. 
                Track issues in the same way across people


Deployment
Web applications will need to scale somewhere in their lifetimes, especially if they are successful. Think about your initial deployment environment - one webserver and one database server (on the same machine). As time goes by and your app becomes popular, you add servers. And features. New features translate to new application modules and new databases. The simple script you used to upload and deploy your small app is useless now.

At this point, scaling has multiple meanings

        The ability to handle increasing load and maintain baselevel performance, and

        The ability to push code into deployment quickly.

Both of these are affected by having (or not having) a consistent model.

The first point is actual application scaling - the complexity of your infrastructural setup is going to increase hugely as it grows. The second point has to do with how your team scales to increasing demands.

They are related. Critical fixes and features might need to be pushed immediately. These demands would reach proportions where you cannot afford to spend time figuring out why Server No 6 in your cluster does not have the latest changes. Automation is the key here. Automation demands formal well defined processes (for making builds, uploading to production servers etc). Formal deployment processes imply consistency. They do not mean a bureaucracy – just well followed and automated rules about how to deploy a change into production. As your app and infrastructure grows, it becomes more and more important to be able to rollback changes if necessary. This is possible only if you have well defined deployment paths and scripts.

Automate wherever possible. Minimize the number of things you have to keep in your head. And in the process, lower your stress levels!

Note: These thoughts are not entirely mine - these are the culmination of what I feel about consistency after having read the experiences and opinions of many others in blogs, books and articles, coupled with my own experiences in developing products. Also, you might have noted that I have been talking of web application development in some of the sections above, but these apply to any kind of software development.