Skip to main content

$400 million to Bankruptcy in 45 mins :(

While googling for devops case studies, i found this very interesting blog post by Doug Seven on how Knight Capital Group (an American Global Finance) failed code deployment on one of the eight production instances lead to the bankruptcy of KCG in August 2012.
At the first read I felt it may be a hypothetical case or something that is shown too aggravated. But with minimal digging I could correlate some of the facts even thought I am not totally sure about the the whole story. Anyways here the point is not about the story but about Dev & Ops nightmare.

Ignorance is not a bliss

Blog may be talking about 2012, but even today we just have a below 40% adoption of devops which seriously means lots of organization out there still doing manual heavy lifting of syncing of code between the dev/qa/prod environment. With manual intervention there is always a possibility of mistakes. If not today… tomorrow. Now the question to be asked is When or Why? Yes, with so much at stake organization cannot leave it to chance because things are fine at this point.

Human mistakes are inevitable. It may be in any form on the PDCA (Plan Do Check Act) cycle. So better to alway automate the processes which is repeatable. Definitely high risk processes like software deployments so that we have more reliability with each repetition.

Comments

Post a Comment

Popular posts from this blog

Infra store – the next IT marketplace

We are all familiar with the Apple App Store or Google Play Store we visit every day to download apps, games and necessary updates for our phones and tablets. The app store model revolutionized the marketplace idea, making it easy for both software vendors and consumers to publish and install software without the hassles of software building, distribution and deployment. Read further on CSC HyperThink
Post a Comment
Read more

Do we know the enterprise IT challenges...???

Last night during the dinner chat with one of my old school pal, we stumbled on the topic of current issues that enterprises are stuck with. It went on almost for 30 mins. But what made it less interesting to me is that whole discussion was around cost cutting, our sourcing, rationalization etc., It is really boring, we are still taking about the tip of iceberg. But the question is due we really know what the real challenges are. I am not talking about a laundry list with 30/40/50 items. I am looking why we really have those items? (whatever the count is). I could not get this out of mind and started listing, order, consolidating, prioritizing those items to make sure I am completely confident that as a consultant I am doubly sure about them. Of course, it is debatable. But this is what I think are core problem and rest of list is the symptoms. 1. Dynamic market conditions are forcing business to adopt rapidly while IT is able to respond to this 2. Day by day IT is becoming exp
Post a Comment
Read more

Machine learning challenges with Imbalanced Data

For many real world machine learning problem we see an imbalance in the data where one class under represented in relative to others. This leads to mis-classification of elements between classes. The cost of mis-classification is often unknown at learning time and can be far too high. We often see this type of imbalanced classification scenarios in fraud/intrusion detection, medical diagnosis/monitoring, bio-informatics, text categorization and et al. To better understand the problem, consider the “Mammography Data Set,” a collection of images acquired from a series of mammography examinations performed on a set of distinct patients. For such a data set, the natural classes that arise are “Positive” or “Negative” for an image representative of a “cancerous” or “healthy” patient, respectively. From experience, one would expect the number of noncancerous patients to exceed greatly the number of cancerous patients; indeed, this data set contains 10,923 “Negative” (majority class) and
Post a Comment
Read more