The process of software development is long and hard. It goes through a lot of stages. Development is usually done on your laptop/desktop and once you are done, it is hopefully moved to a production like machine where your changes are tested. If passed, it then is passed onto production. All these different environments are configured differently and possibly access and enable different areas of the application being deployed. All this is usually managed by configurations.

The scenario mentioned here gives rise to the possibility of two different types of configurations and that is going to be my rant for today.

There is application configuration and there is environment configuration. If you have to turn features on or off in your application in production, or release something that you want a subset of your users to see for some testing purposes or for whatever reason, you want flexibility, then that setting is an application configuration.

Any change in environment, whether machines, URL’s, databases etc when you move software from one system to another in the development lifecycle requires environment configurations.

Usually, the way to go about them is to put the configuration in a YAML file. For example

ThisReallyIsSomethingIHate: 1
ThisIsSomethingMeLikey: &AliasForIt
    Name: Gandalf
    Default: #98AFC7
    Update: #FFFFFF

This works fine for environmental configurations which rarely change and which do not need to be altered every so often. But for application changes, where toggling of features on and off, before and after deployment is a common thing, we need something that works, in my opinion similar to a JMX console. An admin console, which tells us the health of the feature/application, and allows us to toggle the features on and off.

This involves moving the application configuration from a flat file into a persistent storage, perhaps a database. This ensures that Operations do not have to worry editing files but actually have a console to do it in, something that is easier to start and manage. This would also help out QA during testing, because now they do not have to write automation tests to edit configuration files but a simple selenium step will accomplish that for them.

I do not know whether this is the right way of separating application and environment configuration, but I do know that we need to separate them.

A long time ago in a university not so far away, I was about to commence a subject that every body said was one of the most fundamental subjects available. Software Engineering; Processes and Tools. Before that, I had completed(barely) the prerequisite for that which introduced us to the basics of Software Development. So I was excited and very motivated to get this one down pat. I really wanted to do this right and learn as much as I could about the intricacies of designing and developing software in a team environment.

Apart from the usual load of assignments, tests, tutorials and final exam, the one thing that marked this subject out was a large project designed to be completed in a team environment. Whereas previously, in the prerequisite, I had worked with just one other person, this subject would involve teams of five and/or six. I was very excited and immediately approached a few of my friends in the same year and we all agreed to do this project together. I distinctly remember thinking at that point, this is going to be fun!

Alas, I was wrong. Very very wrong. And all the misery that ensued from that point on had one and one simple reason. I was not objective! Don’t get me wrong, my friends were very talented and committed, but they had different scales of working. They all worked in a different manner, some at different times of the day! Some worked at home while some preferred to pull night shifts in the university! The only thing that bound us, was that we were friends! And therein lies the problem. Had I approached my friends on the basis that they were very talented people with whom I could objectively work and finish this project and my friends, in specifically that order, we might have developed very good software. As it turned out, my priorities were skewed. I wanted friends to work with, not to work with friends.

Hence the point of this post. One must always, always, be objective. If you are working with friends, in a professional environment, you must be able to tell the truth to them, in front of others and in private. If you are leading a team, then you must be able to tell people to pull their weight, first by example and then by actions. Being friendly with your friends only works when all of you are clearly working towards a goal and objectives are clear. Trust me, friendships are strengthened when you are brutally objective towards each other professionally.

Postscript: We managed to pass the subject, but ended up not learning much. And thanks to ashridah for the objective talk after my results.

A very good article on the what a Cyber War entails. Taken from here

Using cyber attacks to spy on another nation is a grey area, which gets greyer still when a country penetrates information networks, just to see if it can do so. Penetrating such networks and leaving a back door open, or even leaving logic bombs behind to be used later, is a harder case—yet the US and China are doing this to each other right now.

On the second and final day of the YOW 2010 conference, there were a few management oriented sessions in the evening, a few technically descriptive ones in the morning and two very nice focussed ones in the middle. These were the DevOPS session presented by Michael .T. Nygard and one about taming large builds by Chris Mountford

Chris started talking about how they worked at Atlassian, particularly on JIRA and how they have problems testing and managing the tests. He showed us a few screen shots from many of the builds and frankly they were monstrously large, especially the one about JIRA, which between the time he had made his presentation and presented it, grew larger. He mentioned some very nice techniques that could be used to prevent and/or maintain monstrously large builds as they are very bad. Why ? Because builds are there to give you better feedback and quick. A build that takes 10 hours is of no use in an Agile world. Another thing, which might interest the business owners more, is that large builds have large turn around time and consequently more time needed by the developer to fix the build, which is less time she or he is spending on the work that the business wants done. The one thing I figured that Atlassian does that we don’t do are Canary Builds. But soon, I think we will. A very good session that was very informative, and slightly reassuring that we are not the only ones wrestling with huge monstrous builds.

It was refreshing to have another session which based on actual events and the conclusions and/or lessons that could be and were drawn from them. I have seen many speakers over the last two days conjure up scenarios to fit the example on the upcoming slide, but this was very nicely structured around real world events, with pictures as proof. The whole session was basically an operational and development postmortem of a few events and why those events demonstrate the invaluable aspect of no barriers between developers and operations.

Then Michael went on about how they diagnosed incidents along with developers after chaotic incidents that led to downtime. How they went step by step through any causes and how they actually constructed a KanBan wall without realising it. How process went out of the window and how that had both good and bad consequences. He also reflected on how that defined, to an extent, the intersection of the roles a system administrator and a software developer.

All in all, of all the talks in these two days, these two are the ones I have enjoyed the most. Practical, simple, and extremely well presented.

On the first day of the YOW conference, I was pretty excited about the post lunch session (even after eating a delicious lamb kebab). The two topics that I was very interested in were NoSQL by Justin Sheehy and Strategic Design by Eric Evans.

So off I traipsed into the NoSQL talk, eager about some new news on things like CouchDB, MongoDB and maybe a bit about Mongoid. It certainly was not what I expected. The first half was passionate analysis of what exactly NoSQL is, and more importantly, how we should choose it! The second half of the talk focussed on Riak and I confess uptil today, I had no idea what it was. From there it was a bite more interesting to see actually how Riak worked and how scalability and extensibility were inbuilt into it. Not sure I will be using it anytime soon, but good to know me thinks.

Out I came from this one and went to hear the Strategic Design talk. It was a very well constructed talk on how the best of motives could lead the best of developers astray by setting wrong goals and the drive to never make a compromise. And also metaphors are slightly evil . But the talk was worth listening to, as it demonstrated how refactoring needs to be targeted and how to get business on your side to sponsor the things you think they won’t agree to. Eric demonstrated how models can be extracted in different forms by using a poem, a map and a quote as example domains. It was pretty instructive. Looking back at some of the work I have been doing at personally and some work we have been doing at REA, I would definitely say that we have implemented a fair number of said strategic ideals, especially for the latter body of work.

And that pleases me 🙂

Tomorrow, is the last day of YOW 2010 and I am excited about a devops session amongst others.

I am currently sitting in the foyer of the Jasper Hotel, sipping a cup of tea, thinking about the two sessions I have just had and the one I am waiting for. The sessions got kicked off by Obie Fernandez about the evolution of Rails 3. My first assumptions on listening were that I do not know why I want to know how Merb and Rails merged and who was involved and what was said to whom. But I was clearly in a minority. I put this down to the fact that I have not been as involved with the rails community as much as the majority of the people in the room.

Towards the latter half of the session, Obie began showing the new features in Rails 3 and I have to say that they do look very promising. I have not yet used Rails 3 myself (I began installing it in a separate gemset during the talk) but things like routing(in routes.rb), deploying(using rack), persistence handling and querying (using ActiveRecord etc have been made very very and easy to understand. I particularly like the lazy execution of DB queries and the subsequent chaining. I am told that upgrading from 2.X to 3 is a bit of a pain, but I guess I will find out.

The second session was delivered by Neal Ford about how he was involved a 3 year (and possibly ongoing) long project using Rails that is very enterprisey[sp?] in nature. It was interesting about the philosophy of tests should do but a lot of talk was bout the tools used. (A lot of them happened to be ThoughtWork tools 😉 )

One interesting point that did come up was that they had 15 pairs sometimes having one stand up and that the stand up was still quick! I would have loved to see how that was enforced, as I know that from first hand experience that that is not easy. They created things to make software development easy and they made life of people as easy as possible so that people could concentrate as much as they could on their core task.

But what I liked most of all is that they had 4 work day weeks. 🙂 10 hours a day, sure, but small price to pay for a 3 day long weekend.

I am currently waiting for the next session, which is titled “Why NoSQL” and I am really looking forward to it considering I am thinking of moving my personal projects over to Mongoid