I just came into office, a few days after the last release of our product, feeling relaxed and thinking about the new release. Just then my colleague, who was working on introducing a new LDAP backend, and was testing the migration code which he had written came up to me and reported a flaw in the upgrade/migrate of the just released version. As I have been sharing with you guys, we have a well defined process of making a product release with clear steps and forms which are filled up and recorded to help track back when required. I looked up the release records and found that the particular test which would have trapped this flaw was performed and the tester had reported a success on that. Surprise...Surprise...Surprise. Was it an oversight by a tester or was it a flaw in the process. So lets get to the bottom of things...
I've been very inspired by the program "Nat Geo investigates" on the National Geographic channel, where they do stories on investigations carried out for plane crashes, space shuttle crashes, and other such life threatening disasters. Last week, I saw how the investigators (combined from several countries) looked into a Swiss air crash in the Atlantic, spent over $50 million and 4 yours of painstaking work to recover the splintered millions of pieces of the aircraft to get to the bottom of the cause of the crash. Amazing...Its no wonder that the airlines industry is six sigma.
We have been following similar principals, where we analyze each issue, waste, problem and see how it can be fixed by technology, processes and training (in this order) In this particular instance, it turned out that the process form had an item to confirm that the logs are clean after upgrade....Hello...Which logs, how to check if they are clean...How much time has elapsed since the test before you check the log (logs rotate due to automated cron jobs and leave behind an empty log file which will have no errors). Improved the form immediately and retrained the QA team.
Every problem is an opportunity to improve the system which drives your business. Don't waste it.
Friday, April 07, 2006
Friday, March 31, 2006
Ideas - The light bulb...
Everybody goes through this phase (some stay in it forever) where maximum amount of time is spent in conceptualizing, ideating, and discussing the new thoughts. We as a team also used to do a lot of it previously. Undoubtedly an idea can be powerful, can provide breakthroughs, and we as humans are lucky to be the rare gifted species who can imagine things and conjure up images about the future. There was a time when just discussing an idea would provide immense excitement.
However, of late I realized that my attitude towards a good idea (or ideas in general) had become quite subdued. In my job, I am grappling on a daily basis with the just GTD (Getting things done)/execution. We learn daily about how complex it is to convert an idea to reality which is marketable, usable, scalable. For me an idea will excite me no end if I see it executed at least partially if not fully.
In my sphere of influence, I find that many people are caught up in idealogical world where they spend a lot of time thinking about the future (read dreaming), marveling at the creation of other people/companies (grass is greener on the other side of the fence), reading a lot about new technologies (mostly out of context and more than required) and doing precious little themselves to get any substantial work done.
Mind you, I am not disputing the power of an idea...but nothing can beat the power of a well executed idea. So what am I saying?
1. Think and learn in the context of the work in hand. (Lean thinking, lean learning, lean execution)
2. Execute it with demonic focus (read my previous blogs for some tips on how to do this)
3. Marvel at your own creation (Let other lazy people also marvel at your creation)
OK so lets get back to work now :-)
However, of late I realized that my attitude towards a good idea (or ideas in general) had become quite subdued. In my job, I am grappling on a daily basis with the just GTD (Getting things done)/execution. We learn daily about how complex it is to convert an idea to reality which is marketable, usable, scalable. For me an idea will excite me no end if I see it executed at least partially if not fully.
In my sphere of influence, I find that many people are caught up in idealogical world where they spend a lot of time thinking about the future (read dreaming), marveling at the creation of other people/companies (grass is greener on the other side of the fence), reading a lot about new technologies (mostly out of context and more than required) and doing precious little themselves to get any substantial work done.
Mind you, I am not disputing the power of an idea...but nothing can beat the power of a well executed idea. So what am I saying?
1. Think and learn in the context of the work in hand. (Lean thinking, lean learning, lean execution)
2. Execute it with demonic focus (read my previous blogs for some tips on how to do this)
3. Marvel at your own creation (Let other lazy people also marvel at your creation)
OK so lets get back to work now :-)
Tuesday, March 21, 2006
Being thorough
It was close to 10 P.M., with a release in progress. All the issues pertaining to the current development/checkin had passed the QA. But as part of the elaborate QA process, one of the elements is a smoke test of the entire product (recently introduced step, previously we would smoke only the changed areas of the code since we have a very dense automated testing system). During the 'full' smoke test the QA team started reporting issues unrelated to the current checkin and as I kept analyzing each issue to see where it originated, I discovered that these were a result of our old bad habits (quick development where the developer declares the code complete as soon as the file has been saved, cursory reviews with no review checklist/framework, hurried checkins, hurried or incidental QA with inadequate support via test cases, or automation etc.).
Of course we have come a long way and now (thanks to continuous and relentless improvements) can boast of a formal and very elaborate process for development, reviews, documentation and QA. As described in my previous blog posts, we have introduced templates (forms) for each stage and formalized the process to prevent slippages. As a result each checkin goes through heat treatment and typically results in almost no kickback in terms of reopened issues. In fact our lean production system with one piece flow allows us to make a strong release almost every week.
But last night, I realized something very important - after working with many developers during my tenure as a product manager, I can recall only a handful who were really thorough. This handful of guys, without any formal process, could produce high quality stuff, checkin all by themselves and get that code released. Today some of the reviews I do make me wonder, as to what would happen if I would let that get checked in as is (used to happen before and would cause Murphy's law to apply all the time - if anything can go wrong it will). Mind you, we are talking about really smart, intelligent and capable programmers who can design and construct good code but miss on the details/or are too distracted to be thorough in completion of their work or dont know what is completion.
I am sure if you are a product manager, project lead or development lead, you might have faced this type problem in your work...So how do we tackle this issue. I feel that being thorough is a habit/style of work that can be cultivated. Its not a genetic encoding.
For you the manager, you have to foster a culture of thoroughness:
1. Formalise the processes to define completion. Have forms which prompt and trap information which would otherwise be incidental. This makes quality consistent and not 'one of'.
2. Record the filled up forms with the checkin/release to help do post mortem analysis in case a problem is to surface later. This will help you identify process improvements, or genuine lapses in following the process.
3. Don't fix the code given to you for review - return it for rework with honest feedback to the developers on the problems, good points and recommendations. Use this as a way of mentoring. If you tend to take a shortcut and fix the code, you are actually leaving the developers with the impression that they have done a great job.
4. Stop when you face any problem and immediately fix the process, templates, infrastructure or whatever is required.
For the developer, be a hawk with an eye for detail:
1. Just coding is not enough: Please understand that just writing and compiling code successfully is not enough. You need to ensure it integrates well, all connections to other code are maintained, test cases are thorough, documentation of the design and code is done to promote easy maintenance, coding standards are followed, an honest self review is done etc.
2. Be thorough: Develop this habit (was the motto of my school) to get a grip on the details (remember god lies in the details and BTW the devil also lies in the details :-), make notes, search through code to handle changed connections (many times a product has modules/components written in different programming languages, legacy code, and a certain change may not get directly caught by a compilation e.g. dropping a database table from the storage but having the code still refer to it, will not get caught by a normal build of the code and if you are unlucky may never get caught unless it is too late).
3. Stay focused: Keep disturbances away (refer to my earlier blog posting)
4. Throw a challenge: for the lead/reviewer/qa to find bugs in your code (For this you have to be really confident...comes from following my tips above)
I hope the above tips help you grow as an individual and also contribute better to the company where you are working...All the best (P.S. I am toying with idea of rating developments and also reviews on a scale of 1-10)
Of course we have come a long way and now (thanks to continuous and relentless improvements) can boast of a formal and very elaborate process for development, reviews, documentation and QA. As described in my previous blog posts, we have introduced templates (forms) for each stage and formalized the process to prevent slippages. As a result each checkin goes through heat treatment and typically results in almost no kickback in terms of reopened issues. In fact our lean production system with one piece flow allows us to make a strong release almost every week.
But last night, I realized something very important - after working with many developers during my tenure as a product manager, I can recall only a handful who were really thorough. This handful of guys, without any formal process, could produce high quality stuff, checkin all by themselves and get that code released. Today some of the reviews I do make me wonder, as to what would happen if I would let that get checked in as is (used to happen before and would cause Murphy's law to apply all the time - if anything can go wrong it will). Mind you, we are talking about really smart, intelligent and capable programmers who can design and construct good code but miss on the details/or are too distracted to be thorough in completion of their work or dont know what is completion.
I am sure if you are a product manager, project lead or development lead, you might have faced this type problem in your work...So how do we tackle this issue. I feel that being thorough is a habit/style of work that can be cultivated. Its not a genetic encoding.
For you the manager, you have to foster a culture of thoroughness:
1. Formalise the processes to define completion. Have forms which prompt and trap information which would otherwise be incidental. This makes quality consistent and not 'one of'.
2. Record the filled up forms with the checkin/release to help do post mortem analysis in case a problem is to surface later. This will help you identify process improvements, or genuine lapses in following the process.
3. Don't fix the code given to you for review - return it for rework with honest feedback to the developers on the problems, good points and recommendations. Use this as a way of mentoring. If you tend to take a shortcut and fix the code, you are actually leaving the developers with the impression that they have done a great job.
4. Stop when you face any problem and immediately fix the process, templates, infrastructure or whatever is required.
For the developer, be a hawk with an eye for detail:
1. Just coding is not enough: Please understand that just writing and compiling code successfully is not enough. You need to ensure it integrates well, all connections to other code are maintained, test cases are thorough, documentation of the design and code is done to promote easy maintenance, coding standards are followed, an honest self review is done etc.
2. Be thorough: Develop this habit (was the motto of my school) to get a grip on the details (remember god lies in the details and BTW the devil also lies in the details :-), make notes, search through code to handle changed connections (many times a product has modules/components written in different programming languages, legacy code, and a certain change may not get directly caught by a compilation e.g. dropping a database table from the storage but having the code still refer to it, will not get caught by a normal build of the code and if you are unlucky may never get caught unless it is too late).
3. Stay focused: Keep disturbances away (refer to my earlier blog posting)
4. Throw a challenge: for the lead/reviewer/qa to find bugs in your code (For this you have to be really confident...comes from following my tips above)
I hope the above tips help you grow as an individual and also contribute better to the company where you are working...All the best (P.S. I am toying with idea of rating developments and also reviews on a scale of 1-10)
Friday, March 17, 2006
Keeping disturbances handy
It was noon and I sat down to do a design/code review as part of our development process. I had just wrapped up my sundry tasks of responding to mail, checking the status of ongoing projects, going to the loo, drinking water before settling into the review. Yes, for me a review of design and code before it goes further down the pipe is the most important part of the process. (Refer to my earlier postings about the importance of reviews, or where it is cheapest to fix bugs). Just as I started reading the specifications and design, my cell rang. It was one of our deployment engineers onsite at a customer location asking for some help in using one of the new features. After I finished the call and struggled to revert my attention to the review (yes its very difficult to switch your mind amongst activities easily), one of my team members called out to me for some quick help on some design decisions (Yes I don't have a cabin and sit in the middle of my group and am easily accessible - a boon and a bane). Once done with that, I again struggled and restarted the review. But by now my initial focus was disturbed and the two discussions (during the disturbances) were coming back to me to think about. It was like not being able to work at work.
I stopped what I was doing to reflect on this. I looked around and observed what other people in my team were doing. I also walked around to other divisions to see what those teams were doing. I saw one set of people intently doing their work (coding, diagramming, documenting, invoicing, selling etc) and another set of people talking to friends, chatting with friends on IM, browsing unrelated sites, and generally appearing to wile away their time. This was of course a sampler at a given moment but I also realized that these are the same set of people who keep their "disturbances handy" like I did when I started my review. i.e. Make it easy to be disturbed ( I am assuming that they are committed to the job; there are a set of people who wait for the day to end... this article is not about them). I also realized that there are two basic types of activities one where you create (a document, code, design, review...) and another were you maintain (run a process like front desk, recruitment, etc). In the case of the former, being disturbed is sure to bring down the quality of the work since the very nature of that work demands undivided attention and laser like focus to get it done (no wonder that Bill Gates takes two weeks off by himself on an island to figure out the future of Microsoft). As for the latter, the mind is not creating and is not in a stressed state trying to retain connections, make logical links, theorize etc so it can afford to be disturbed)
My recommendations while creating:
1. Isolate yourself so you will not be disturbed during that time.
2. Schedule such work at times when you are least likely to be needed urgently.
3. Turn off the phone, email and chat pop ups since these are like the door bell (Humans are inadvertently trained to respond to the doorbell and phone ring, dropping whatever they are doing, literally).
4. Focus on the task at hand.
5. Divide your time into sections where you are available for discussions, you respond to mail, return phone calls etc and sections where you are invisible.
Even as I write this, I was disturbed thrice...hope you will excuse any break in the flow due to that :-). I am going home now to complete the review...at least there only my dog will disturb me.
I stopped what I was doing to reflect on this. I looked around and observed what other people in my team were doing. I also walked around to other divisions to see what those teams were doing. I saw one set of people intently doing their work (coding, diagramming, documenting, invoicing, selling etc) and another set of people talking to friends, chatting with friends on IM, browsing unrelated sites, and generally appearing to wile away their time. This was of course a sampler at a given moment but I also realized that these are the same set of people who keep their "disturbances handy" like I did when I started my review. i.e. Make it easy to be disturbed ( I am assuming that they are committed to the job; there are a set of people who wait for the day to end... this article is not about them). I also realized that there are two basic types of activities one where you create (a document, code, design, review...) and another were you maintain (run a process like front desk, recruitment, etc). In the case of the former, being disturbed is sure to bring down the quality of the work since the very nature of that work demands undivided attention and laser like focus to get it done (no wonder that Bill Gates takes two weeks off by himself on an island to figure out the future of Microsoft). As for the latter, the mind is not creating and is not in a stressed state trying to retain connections, make logical links, theorize etc so it can afford to be disturbed)
My recommendations while creating:
1. Isolate yourself so you will not be disturbed during that time.
2. Schedule such work at times when you are least likely to be needed urgently.
3. Turn off the phone, email and chat pop ups since these are like the door bell (Humans are inadvertently trained to respond to the doorbell and phone ring, dropping whatever they are doing, literally).
4. Focus on the task at hand.
5. Divide your time into sections where you are available for discussions, you respond to mail, return phone calls etc and sections where you are invisible.
Even as I write this, I was disturbed thrice...hope you will excuse any break in the flow due to that :-). I am going home now to complete the review...at least there only my dog will disturb me.
Friday, February 03, 2006
How much testing is "good enough"
Yesterday I had a fruitful discussion with my colleague (Kul) on the above subject. We were debating the selection of test cases, which ones will provide maximum impact or which are not likely to occur in a normal working scenario or to ignore cases which even if they fail will not damage the integrity of the data or working. Although we didn't reach any conclusion yesterday, I reflected on it later and this is what I came up with:
I believe that if you have taken the pains to encode a behavior (and validation of data input is a behavior), then there must be a good enough reason for it. In its simplest form, it could mean that if a program accepts a domain name as a parameter, and if we pass a wrong domain name, the program will do no harm (who knows ?) but may not inform the user about it via a user friendly message. So we as programmers build in data input validation checks and report neat failure messages and stop proceeding further. BUT if we don't test this case (may occur rarely, or is not harmful as we understand), we stand a chance of hitting a potential failure (could be as trivial as an embarrassing error message or worst still some damage which wasn't envisaged.)
Testing is the science of validating ALL assumptions and behaviors of a program. If you have taken the pains to encode a behavior it needs to be tested OR else the behavior need not be encoded at all. Also remember that code is never one time; it is maintained by other programmers and a regression suite of COMPLETE COVERAGE automated suites will help the maintenance programmer make changes freely.
Hence my mantra
1. Make code easy to test. Write compact and simple code with easy constructs with only necessary checks and behavior. (In my experience I have seen that this is possible even for very complex code, if it is broken down into units. This constraint will help you write better code in fact.)
2. Write data validation test cases for invalid and valid data input covering all possible (necessary and sufficient) test cases to cover all encoded behavior. You could refer to code and its flow for designing the cases. Remember if a program accepts junk input, it is more likely to excrete junk output. Control input to control output.
3. Write test cases to check the functioning of the code.
4. Automate aggressively to allow for easy regression.
5. Put in a process to maintain the test cases when the code changes.
Happy bug free coding...
I believe that if you have taken the pains to encode a behavior (and validation of data input is a behavior), then there must be a good enough reason for it. In its simplest form, it could mean that if a program accepts a domain name as a parameter, and if we pass a wrong domain name, the program will do no harm (who knows ?) but may not inform the user about it via a user friendly message. So we as programmers build in data input validation checks and report neat failure messages and stop proceeding further. BUT if we don't test this case (may occur rarely, or is not harmful as we understand), we stand a chance of hitting a potential failure (could be as trivial as an embarrassing error message or worst still some damage which wasn't envisaged.)
Testing is the science of validating ALL assumptions and behaviors of a program. If you have taken the pains to encode a behavior it needs to be tested OR else the behavior need not be encoded at all. Also remember that code is never one time; it is maintained by other programmers and a regression suite of COMPLETE COVERAGE automated suites will help the maintenance programmer make changes freely.
Hence my mantra
1. Make code easy to test. Write compact and simple code with easy constructs with only necessary checks and behavior. (In my experience I have seen that this is possible even for very complex code, if it is broken down into units. This constraint will help you write better code in fact.)
2. Write data validation test cases for invalid and valid data input covering all possible (necessary and sufficient) test cases to cover all encoded behavior. You could refer to code and its flow for designing the cases. Remember if a program accepts junk input, it is more likely to excrete junk output. Control input to control output.
3. Write test cases to check the functioning of the code.
4. Automate aggressively to allow for easy regression.
5. Put in a process to maintain the test cases when the code changes.
Happy bug free coding...
Thursday, January 05, 2006
Formal reviews
Sometimes back I wrote about informal code reviews, since that is what we were following as a practice. However I found that the number of bugs in the checked in code was not reducing and the qa team had to work hard to not just validate the product but also to locate defects. We were practicing informal reviews as a way to "speed" up the development cycle. Unfortunately the reverse was actually happening - the code was checked in rapidly but the product languished in the qa stables for longer and also carried the risk of some bugs being discovered after release. As I wrote in one of my previous blogs, it gets more expensive to fix bugs as the code goes further away from you.
However, we recently introduced a formal and detailed process for review of not just the code but the entire design, test cases, documentation guidelines, and the code (peaceful reviews in the solitude of your cabin) with a defined checklist on what all to cover, including using the features of a developer build before checkin. We have had some amazing results...did some rough measurements and found that we trap about 90% of the problems before checkin especially framework plugs missed, silly errors, code connections, basic design flaws, messages etc. This is against trapping about 50% of the problems by the older method of review. On the face of it, the process seems to take longer but actually it outputs better quality code almost right first time reducing the rework thrash. Also such detailed drill down reviews provide benefits like
- Another perspective on the design, code thus bring it closer to extreme programming.
- Helps others to understand the code/design building in the redundancy into the team.
Yes, I know what you guys are thinking...that all the software books talk about formal reviews in which the developer sends code and lets people review it or the code is reviewed by multiple people as an event rather than "by the way", so why werent we doing it earlier...Like anything else, all practices if done in a context, adapted to your specific needs and learnt the hard way :-) bring about better results. Try for yourself, it works.
However, we recently introduced a formal and detailed process for review of not just the code but the entire design, test cases, documentation guidelines, and the code (peaceful reviews in the solitude of your cabin) with a defined checklist on what all to cover, including using the features of a developer build before checkin. We have had some amazing results...did some rough measurements and found that we trap about 90% of the problems before checkin especially framework plugs missed, silly errors, code connections, basic design flaws, messages etc. This is against trapping about 50% of the problems by the older method of review. On the face of it, the process seems to take longer but actually it outputs better quality code almost right first time reducing the rework thrash. Also such detailed drill down reviews provide benefits like
- Another perspective on the design, code thus bring it closer to extreme programming.
- Helps others to understand the code/design building in the redundancy into the team.
Yes, I know what you guys are thinking...that all the software books talk about formal reviews in which the developer sends code and lets people review it or the code is reviewed by multiple people as an event rather than "by the way", so why werent we doing it earlier...Like anything else, all practices if done in a context, adapted to your specific needs and learnt the hard way :-) bring about better results. Try for yourself, it works.
Thursday, November 24, 2005
Get professional in your attitude
I am writing this in response to facing the brunt of purely childish emotional behavior by certain class of people whom I have worked with. No matter how talented, skilled and capable you might be, but if you have a bad attitude, are insecure professionally and lack confidence, you are a misfit wherever you work. You will always be disgruntled and untrusting of your colleagues, thinking that the next guy you meet is going to take you for a ride. I believe that as professionals, we have to deal with situations impassionedly and rationally. These are some of my tips for being a good professional:
1. Believe in yourself, be confident and be hungry to learn. Get involved in your work to widen your horizon.
2. Everyday, do an honest days work. Get connected to the overall vision/goal of the company and follow through with actions. Don't vile away your time...There's lots to do, just go do it.
3. For you to make a significant contribution in any sphere, and also for you to gain something worthwhile from this engagement, you need to spend substantial amount of time in the company. I feel that 2.5 to 3 years is a bare minimum. If you retire from the company, that's great.
4. Trust the company you work in or else quit. Don't hold resentment within yourself towards the management or hold feelings of being cheated or short-changed of opportunities or start indulging in politics like forming them v/s us groups. If such feelings arise, discuss with your manager candidly and sort them out. After that if discomfort still persists, it is better to leave. By continuing, you are creating more harm than good to yourself and to your company.
5. If you must leave, do it with grace and dignity. (If you are leaving for better opportunity, it's perfectly all right. Be forthright about that, rather than hide behind some farce of blaming the company for non-issues just to avoid feeling guilty. I feel that you don't have to feel guilty if you have adhered to points 1 and 2 in spirit.) Complete all unfinished work, discuss with the manager a suitable leaving date so that you don't leave in a lurch (we've had people who come in the middle of a product release and desire to leave that very day with no concern about their commitment to completing the job) and leave behind your contact details in case you are needed in future for some help. Also remember that this globally connected world is getting smaller, with technology enabling easier and wider communication (e.g. this blog to share my thoughts), good as well as bad references can make or break your career.
6. Stay in touch, since you never know when you may need to ask for some help/references from old contacts.
Wish you all the best.
1. Believe in yourself, be confident and be hungry to learn. Get involved in your work to widen your horizon.
2. Everyday, do an honest days work. Get connected to the overall vision/goal of the company and follow through with actions. Don't vile away your time...There's lots to do, just go do it.
3. For you to make a significant contribution in any sphere, and also for you to gain something worthwhile from this engagement, you need to spend substantial amount of time in the company. I feel that 2.5 to 3 years is a bare minimum. If you retire from the company, that's great.
4. Trust the company you work in or else quit. Don't hold resentment within yourself towards the management or hold feelings of being cheated or short-changed of opportunities or start indulging in politics like forming them v/s us groups. If such feelings arise, discuss with your manager candidly and sort them out. After that if discomfort still persists, it is better to leave. By continuing, you are creating more harm than good to yourself and to your company.
5. If you must leave, do it with grace and dignity. (If you are leaving for better opportunity, it's perfectly all right. Be forthright about that, rather than hide behind some farce of blaming the company for non-issues just to avoid feeling guilty. I feel that you don't have to feel guilty if you have adhered to points 1 and 2 in spirit.) Complete all unfinished work, discuss with the manager a suitable leaving date so that you don't leave in a lurch (we've had people who come in the middle of a product release and desire to leave that very day with no concern about their commitment to completing the job) and leave behind your contact details in case you are needed in future for some help. Also remember that this globally connected world is getting smaller, with technology enabling easier and wider communication (e.g. this blog to share my thoughts), good as well as bad references can make or break your career.
6. Stay in touch, since you never know when you may need to ask for some help/references from old contacts.
Wish you all the best.
Friday, November 18, 2005
Engineering is simplicity
In a very old Hindi movie, I had once come across a scene in which a textile machine in a plant breaks down, bringing the whole plant to a halt. Unless the service engineer for the machine from the company comes in, there will be no production. Enter the unemployed hero (engineer who is not getting a job…in olden times unemployment was a big issue) who offers to fix it. The boss gives the go ahead and observes that the hero hammers a certain portion of the machine, gets it running and asks for Rs. 100 (big money in olden times). The boss was furious, saying that it is unjustified to ask for so much money for just hammering the machine once. The hero justifies saying that Re. 1 for the hammer blow and Rs. 99 for knowing where to strike.
Of course this story has been oft repeated in textbooks with different characters and situations but all point to the same basic thinking of what engineering is all about. Minimum effort = maximum output. Our job as engineers involves
- Continuously improving and simplifying the production system to deliver top quality products with minimum effort, minimum waste and a fast response.
- Present simple & accurate interfaces for the end user to use the product to minimize the learning curve, increase the adoption rate and reduce operator errors.
- Simplify the serviceability of the product to ensure quick turn around for customer support.
Any of the above mentioned items could take years to achieve and 1 or all of them can easily become the source of competitive advantage for the company (in today’s time I think we need all 3 in place to even survive)
I believe that any problem that you face will have multiple alternative solutions and in your process of choosing the solution to be worked on, please put simplicity of the solution on a high score. A simple element has fewer connections, less deviations, is cohesive and is simply easier to produce, use and maintain.
Saturday, October 01, 2005
Evolutionary delivery perfected as a fine art
It was 7:00 P.M and my brain felt like noodle soup. I had just been through 3 grinding reviews before a checkin and was feeling totally exhausted. At Mithi we have small and frequent checkins with as frequent product releases to keep "inventory" as low as possible. Inventory is described as code developed but not released to the customer. At this stage of our existance, we are catering to enterprise customers one on one, understanding their needs/requirements, feeding the product specifications for a "general" product (to eventually be usable by the masses) and developing the product in steps. Due to this model, we can cater to customer demands rapidly, the product shapes up very close to the market needs (without any 'out-inventory', which is described as features never used by customers) and we get feedback on the feature developed before getting too far into the feature maturity.
Traditionally product development plans a release, draws up specifications, developers work on these specifications, build and use the product inhouse continuously (hopefully) before releasing the pruduct after a few months/years.
Most companies trying to reach the elusive version 3 (Version 3 is where the product is deemed to have sufficient maturity to cater to a vast set of users - Maturity shouldnt be confused with stability, reliability, which is a given even in version 1 of the product) of the product (where it starts a snow ball effect and can pick up steam with its capabilities), cant afford the long cycles of product releases.
At Mithi we set up an architecture, build on it with short frequent releases of features required by our direct in touch customers (always trying to get in the feature at version 0.001 and building on that in next releases), while another team creates the next level architecture (to allow for easier development, easier testing, easier addition of features, better integration etc) in parallel. Typically we try to never go beyond 3 months of work for the next nevel architecture (again with the same idea of improving the architecture incrementally).
Whats the flip side of this? There is never any respite for the team...They are always on their toes since releases are continuous and deliveries to customers are continuous. But, yes I think that we all enjoy the noodle soup...
Traditionally product development plans a release, draws up specifications, developers work on these specifications, build and use the product inhouse continuously (hopefully) before releasing the pruduct after a few months/years.
Most companies trying to reach the elusive version 3 (Version 3 is where the product is deemed to have sufficient maturity to cater to a vast set of users - Maturity shouldnt be confused with stability, reliability, which is a given even in version 1 of the product) of the product (where it starts a snow ball effect and can pick up steam with its capabilities), cant afford the long cycles of product releases.
At Mithi we set up an architecture, build on it with short frequent releases of features required by our direct in touch customers (always trying to get in the feature at version 0.001 and building on that in next releases), while another team creates the next level architecture (to allow for easier development, easier testing, easier addition of features, better integration etc) in parallel. Typically we try to never go beyond 3 months of work for the next nevel architecture (again with the same idea of improving the architecture incrementally).
Whats the flip side of this? There is never any respite for the team...They are always on their toes since releases are continuous and deliveries to customers are continuous. But, yes I think that we all enjoy the noodle soup...
Tuesday, September 13, 2005
Do we need 'experts' in a business?
The answer to this depends a lot on the type of business or the objectives. E.g. if you look at a specialty medical clinic, a specialty fitness center, etc, these places need full time experts to handle the inflow of business on a one to one basis. To scale, these business will need more experts at the different locations. If the expert leaves, it is possible that he would leave a void n the business. Whereas when we talk of a software product business, do we need experts? The answer is of course YES. These experts provide the domain knowledge, they bring in their experience from other jobs, they can crack through problems quickly and they are extremely useful in providing the breakthrough ideas. This is especially very true for a startup trying to bring out its idea to the market. But be warned, working with experts in a startup company is a double-edged sword. For a startup in version 1 or 2 of its product, it can get very easily carried away by seeing something working in the R&D lab (read a very early 'prototype') and be tempted to offer it as a solution to the early customers (in an effort to gather the mind share of maximum customers quickly). In such a situation, depending on the complexity of the software, you may need the very expert, who 'prototyped' the solution, to go and deploy it at the customer location.
This model may work if you limit this exercise to learning from a few deployments (real life scenarios, with the customer's consent) and feeding it back into the prototype and then towards completion. However this very model can backfire if you do too many of these 'prototype' deployments without 'heat treating' the solution within a controlled environment. For all such live deployments, you run a risk of discovering bugs on the customer server (some of which could be dangerous), you always need the expert to maintain these deployments (God only knows what else he did on the customer's server while he was onsite doing the deployment), and you run the risk of overloading the customer support group with emergencies which could well have been avoided in the first place (customer's expectations are not set right and for the customer this is a finished product).
My advice to startups in this mode is to use experts in the research lab to discover solutions, break it down into small & complete deliverables (you don’t want to put out a very mature solution which may not have been tempered enough by customer feedback), do a thorough in house test of the solution or at least the first deliverable (any product you make should be used in house from day 1 - eat your own dog meat), test in a live situation for a few select customers by interacting with them on a one to one basis (don’t put up these solutions on your marketing material unless they are heat treated) and then incorporating it in your product for replication.
So, YES we need experts to provide answers BUT we need systems and processes to capture their output in the business flow for replication and scale.
Sunday, September 11, 2005
I didnt get much work done today!
This 'post' is coming after a very long time. Its not that I havent been thinking about continuing
this series but its just that I have been caught up in two back to back product releases which kept me away from my blogger (and mostly on my desk). BTW I am still caught up in these releases, but I am just squeezing this one article in for now...I had a gentle reminder from a few readers about my promise to expand on my previous blog "It's not just about the code".
I have learnt over the years that besides code, there are a lot of considerations (technical and
non-technical) that can affect a project (positively and negatively). We are not just talking about
schedule, but also about the density and quality of a release. Some one wisely said that "a project schedule slips a day at a time right from the time that the project starts (extend that for quality slippage also)".
So what should be done and what should not be done in order to deliver more than required? I am not going to try and structure these thoughts...like in a book...but just let my thoughts flow in any order that it occurs to me. My suggestions/advice about issues (tech and non-tech) come from my experience and learning on the job and in life. Please use discretion while following my tips and if necessary get help from a consultant about the subject. So here goes...
Recently while I was dropping a programmer colleauge of mine after work, he mentioned during our conversation, that he had a pretty unproductive day and was a little depressed since he really couldnt pinpoint the reasons for the same. In my attempt to help him overcome his depression by finding out the root cause of his unproductive day, I reviewed his activities during the day. I found that he was fine with the tech side of the work on that day...clear specs, simple direct issues, also got a checkin done...etc.
From my own experience, I decided to probe about his health and lifestyle habits. I realized that it would be getting personal, but I know that a lot of people in this world are oblivious of their own body and mind condition. They just go from day to day like robots and feed their physical and mental self with "toxins". They live in what I call a state of "unawareness". Many people dont stop to reflect on what they are getting out of life and whether they are putting their health at a risk to "achieve". I have found that even a distant nagging headache, fatigue, pain, simple stomach ailments, not reading good soul stirring stuff, etc can make you perform at sub-standard levels and you may not even realize this.
Sure enough, my friend had consumed a fair amount of fried goodies at the lunch party, followed by a cola, and cake. Also during the day, he had had more his normal share of tea with sugar and milk. Bingo, his body was full of toxins, making him dull, sleepy and probably a little acidic. All said and done, to "consistently" perform at your peak with a razor sharp mind and a fit body to endure the stresses of todays hyper-competitive world (in any field/profession), one of the most important ingredients is to be in good health, eating well, sleeping well and being moderately active. So is this possible in todays pressure cooker world or fitness and health is a phase in life which comes only after we "achieve"...Watch out for my next blog for some answers...
this series but its just that I have been caught up in two back to back product releases which kept me away from my blogger (and mostly on my desk). BTW I am still caught up in these releases, but I am just squeezing this one article in for now...I had a gentle reminder from a few readers about my promise to expand on my previous blog "It's not just about the code".
I have learnt over the years that besides code, there are a lot of considerations (technical and
non-technical) that can affect a project (positively and negatively). We are not just talking about
schedule, but also about the density and quality of a release. Some one wisely said that "a project schedule slips a day at a time right from the time that the project starts (extend that for quality slippage also)".
So what should be done and what should not be done in order to deliver more than required? I am not going to try and structure these thoughts...like in a book...but just let my thoughts flow in any order that it occurs to me. My suggestions/advice about issues (tech and non-tech) come from my experience and learning on the job and in life. Please use discretion while following my tips and if necessary get help from a consultant about the subject. So here goes...
Recently while I was dropping a programmer colleauge of mine after work, he mentioned during our conversation, that he had a pretty unproductive day and was a little depressed since he really couldnt pinpoint the reasons for the same. In my attempt to help him overcome his depression by finding out the root cause of his unproductive day, I reviewed his activities during the day. I found that he was fine with the tech side of the work on that day...clear specs, simple direct issues, also got a checkin done...etc.
From my own experience, I decided to probe about his health and lifestyle habits. I realized that it would be getting personal, but I know that a lot of people in this world are oblivious of their own body and mind condition. They just go from day to day like robots and feed their physical and mental self with "toxins". They live in what I call a state of "unawareness". Many people dont stop to reflect on what they are getting out of life and whether they are putting their health at a risk to "achieve". I have found that even a distant nagging headache, fatigue, pain, simple stomach ailments, not reading good soul stirring stuff, etc can make you perform at sub-standard levels and you may not even realize this.
Sure enough, my friend had consumed a fair amount of fried goodies at the lunch party, followed by a cola, and cake. Also during the day, he had had more his normal share of tea with sugar and milk. Bingo, his body was full of toxins, making him dull, sleepy and probably a little acidic. All said and done, to "consistently" perform at your peak with a razor sharp mind and a fit body to endure the stresses of todays hyper-competitive world (in any field/profession), one of the most important ingredients is to be in good health, eating well, sleeping well and being moderately active. So is this possible in todays pressure cooker world or fitness and health is a phase in life which comes only after we "achieve"...Watch out for my next blog for some answers...
Monday, August 22, 2005
Where is it Cheapest to fix bugs?
While I was working with my team on the latest release of our product, we had a small percentage of reopened issues or one can say 'last minute bug discoveries'. While I was analysing the issues and trying to fix the root cause (and not the symptoms), I articulated the following important stages of adding anything to the code base (it is possible to introduce or skip bugs at any stage)
1. Issue entry (incomplete, unclear, wrong specifications leading to the wrong development and therefore the mother of all bugs i.e. useless development. In some cases, after careful analysis, we have observed that the issue is best fixed in a non development area like devliery, or sales commitments etc. No code=no bugs)
2. Issue design (Plan and details of how the issue will be tackled. The seeds for bugs are sown here, if the design is lacking clarity, simplicity, proper integration & is improperly or not reviewed by multiple people for perspectives)
3. Issue development (Bugs are introduced here, due to improper code maps, missing or incomplete dependancy charts, improper reviews, carelessness, etc)
4. QA (Bugs are missed here, due to improperly documented test cases, insufficient automation, etc)
As part of any product development model, it is imperitive to find and fix bugs as early as possible and definitely before it reaches the customer. Its a no brainer (and a mantra in software development books) that we have to move towards a right first time build. Our endevour should be to nip the bug in the first two stages where it is cheapest to fix. So to all developers, I suggest a more concentrated effort to Perform aggressive specs and design reviews with more than one person.
1. Issue entry (incomplete, unclear, wrong specifications leading to the wrong development and therefore the mother of all bugs i.e. useless development. In some cases, after careful analysis, we have observed that the issue is best fixed in a non development area like devliery, or sales commitments etc. No code=no bugs)
2. Issue design (Plan and details of how the issue will be tackled. The seeds for bugs are sown here, if the design is lacking clarity, simplicity, proper integration & is improperly or not reviewed by multiple people for perspectives)
3. Issue development (Bugs are introduced here, due to improper code maps, missing or incomplete dependancy charts, improper reviews, carelessness, etc)
4. QA (Bugs are missed here, due to improperly documented test cases, insufficient automation, etc)
As part of any product development model, it is imperitive to find and fix bugs as early as possible and definitely before it reaches the customer. Its a no brainer (and a mantra in software development books) that we have to move towards a right first time build. Our endevour should be to nip the bug in the first two stages where it is cheapest to fix. So to all developers, I suggest a more concentrated effort to Perform aggressive specs and design reviews with more than one person.
Saturday, July 30, 2005
Its not just about the Code
The title & article is inspired by Lance Armstrong's book "Its not about the bike" where he recounts his experience of battling with cancer, recovering and going on to win the worlds most gruelling physical event - "Tour de France". In this book he talks about all what makes a winner (besides the bike). A recommended read.
Last saturday, I happened to get visited by an ex-employee of Mithi who used to be a developer on my team. I was asking him about their development processes, systems and tools since they were serving international fortune X clients with their product. In the course of our discussion I discovered how much 'code' focussed their team was. They didnt have any formal processes to track issues, resolve issues, perform reviews, test case documentation, feed the documentation team, perform releases, automated testing, etc. The whole system was working on 'heroics' with ad hoc processes and arbitrary checkins. The QA team also always started from scratch without any documented test cases & reported issues via email. If an employee left, he left behind a legacy of unmaintainable code. Basically kind of chaotic. Of course they have managed to get some order from the chaos by "systemizing the chaos/arbitrariness".
But it got me thinking. I analysed that their user base is very small (their niche product is sold to a few large corporates where a few users will use the product, totalling maybe 30-40 users), hence they are dealing with a small end user world. Its possible for them to get away with their current way of working and even handle error/bug reports from these customers one-to-one. But when your product starts to reach out to larger number of customers and is used by an even larger number of end users, then the need for a formallised process and system kicks in.
Without a formal issue tracking information system, templates for design/fixing bugs/closing issues, a process for review, a process for documenting test cases for regression, an automated testing system, a source control system, a release engineering team, a formal release process... one is asking for serious trouble in the quality area. Even if the code is perfect, the process and systems around it can break the whole delivery and support of the product. If the inflow of issues from the customers is high, the whole team can get locked into support, letting the future growth of the product languish.
In my future blogs I would like to share with you how we handled rapid growth in customers and users. I will share how we put some of these systems in place, at a very low cost to up our efficiency, consistency and reliability. Putting in systems/processes also reduces dependance on human resources, so the company is better equipped to handle attrition.
BTW, the latest news from my ex-employees company is that their latest release wasnt taken too well by their customers. So much for a systemised chaotic system breaking down.
Last saturday, I happened to get visited by an ex-employee of Mithi who used to be a developer on my team. I was asking him about their development processes, systems and tools since they were serving international fortune X clients with their product. In the course of our discussion I discovered how much 'code' focussed their team was. They didnt have any formal processes to track issues, resolve issues, perform reviews, test case documentation, feed the documentation team, perform releases, automated testing, etc. The whole system was working on 'heroics' with ad hoc processes and arbitrary checkins. The QA team also always started from scratch without any documented test cases & reported issues via email. If an employee left, he left behind a legacy of unmaintainable code. Basically kind of chaotic. Of course they have managed to get some order from the chaos by "systemizing the chaos/arbitrariness".
But it got me thinking. I analysed that their user base is very small (their niche product is sold to a few large corporates where a few users will use the product, totalling maybe 30-40 users), hence they are dealing with a small end user world. Its possible for them to get away with their current way of working and even handle error/bug reports from these customers one-to-one. But when your product starts to reach out to larger number of customers and is used by an even larger number of end users, then the need for a formallised process and system kicks in.
Without a formal issue tracking information system, templates for design/fixing bugs/closing issues, a process for review, a process for documenting test cases for regression, an automated testing system, a source control system, a release engineering team, a formal release process... one is asking for serious trouble in the quality area. Even if the code is perfect, the process and systems around it can break the whole delivery and support of the product. If the inflow of issues from the customers is high, the whole team can get locked into support, letting the future growth of the product languish.
In my future blogs I would like to share with you how we handled rapid growth in customers and users. I will share how we put some of these systems in place, at a very low cost to up our efficiency, consistency and reliability. Putting in systems/processes also reduces dependance on human resources, so the company is better equipped to handle attrition.
BTW, the latest news from my ex-employees company is that their latest release wasnt taken too well by their customers. So much for a systemised chaotic system breaking down.
Monday, July 25, 2005
Programmer maturity
On a Saturday afternoon, I & Aditi (my colleague and co-founder at Mithi) were interviewing a candidate for the post of a software developer, and I as a matter of routine asked him (this is after a tech evaluation, which he had cleared), how long he plans to stay in Mithi. He was prompt in his reply - '6 months'. I was quite taken aback by his ‘clear’ thinking (considering that he was fresh out of college). When I mentioned that it takes around 3 months before a fresh join is productive (to be really useful), he very casually said that ‘OK then I will stay until 9 months to do justice to the job’. Such people are very clear that in small and startup companies, they will learn the most and very fast. They believe that such small companies should be used as a stepping-stone to their personal growth. Amazing, this clarity of thought….
I feel that very large organizations are primarily process and system driven, to the point that any body (satisfying basic criteria of intelligence, academic qualifications, aptitude) can come in produce almost immediately and be of value. These people basically run the system/process (created by others), part of which is monitoring and reporting to ensure that these guys are doing a good job. The system ensures that they produce. Such organizations can recruit in large numbers, get work done, and it doesn’t affect them too much if there is a regular and heavy turnover. In fact many such organizations have perfected the art of selection, recruitment, induction, and exit. (Infosys recently had 730 people join on a single day…to get so many people through the security itself is evidence of how process driven the organization is.)
Unfortunately for small companies & startups, who have to first struggle to find the right product, strategy and business model before putting corresponding processes/systems in place, they need to find people who will go through this struggle and set the system/processes, which will be ‘run’ during the growth phase by the type of people described in the previous paragraph. In such an environment, a heavy turnover causes slowdown in the companies progress and can push the start of the growth trajectory. For such companies the primary selection criteria could be people with the right attitude & character, having desire to create something of lasting value, risk taking ability, have deep belief and confidence in their own abilities and possess a natural drive. Such people know that even in the worst-case scenario of the company failing, they will be only be better off personally.
I have to admit that such people are rare…No wonder that only 1 out of 100 guys make through our (Mithi’s) selection process. BTW, we took a tough call and let the 9-month guy go.
The pain and gain of running a business.
I've been following the Tour de France closely for several years now and have watched how Lance Armstrong has improved year after year to win the worlds most demanding race 7 times in a row. At age 33 he is peaking as an athlete. One would think that a younger, stronger, energetic and more aggressive athlete would have bettered him. And there were many in the race and several of them even won individual stages (battles) but the war was won by the wholistic athlete. Its the cumulative learning from successes & failures, controlled aggression, sustained exposure to competition, and team work (read trust in others) that are required beyond physical strength, genetic ability, and a deep desire to achieve. Thus it has been observed that athletes peak in their early thirties.
Subscribe to:
Posts (Atom)
