Blogs

   The Dangers of Complex Adaptive Systems

Posted October 3rd, 2009 by alshall

There has been some talk about CAS systems in software.  My own experience (albeit limited and I am wanting to learn more) is that CAS is mostly useful at the team level.  I believe there are some dangers at higher levels in most organizations.  Furthermore, I tend to view CAS as a good explanation of things - but not necessarily a good predictive model. In other words, can you use CAS as a model to create a new approach with regular success? BTW: If you can do this, please leave a comment describing when you did that. I have done this with the theories of flow within Lean thinking which is why I am so keen on Lean.  However, I digress.

I wanted to discuss CAS here by providing two examples of it that may provide insights into how it might be used and how it might be dangerous. Actually, I'll take the dangerous one first.  

Dangerous CAS - A School of Fish Trying to Avoid Being Eaten

Most of us have seen a picture of a school of small fish bunching up into a small ball when predators (sharks, bigger fish, ...) attack them.  I've heard this stated as being a survival mechanism.  It was interesting to see how individual fish could follow a simple algorithm (if fish on right and no fish on left - turn right, ...) and keep the ball formed. I had always thought it worked.  I mean, they did call it a survival mechanism - didn't they?

Then one day, I was at an IMAX - Wonders of the Seas - or something like that.  Anyway, this huge school of mullet was being attacked by about 10-12 dolphins (mammal kind of dolphins, not fish kind of dolphins).  Most of the dolphins surrounded the mass of fish while one or two would circle through the fish eating their fill.  The dolphins rotated their positions.  The mullets just kept reforming and the ball (school) got smaller and smaller.  When it was all over, there were 12 happy dolphins and 0 (yes 0, nada, none, all eaten) mullet left.  CAS didn't seem to do a good job here.  A simple strategy of every fish for himself/herself would have worked much better. If the school had merely disbursed, the dolphins would have gone away hungry.

Now why did CAS fail?  I suspect because there was no big picture - no guidance - no leadership.  The dolphins had a big picture - keep the school intact while we eat it.  The mullets had a local phenomena they were trying to keep in place.  Which they did for a while.  

Useful CAS - Geese Flying in Formation

We've all seen geese flying in formation.  It is beautiful to behold. It is aerodynamic - even the fact that one side is longer than another.  It has a purpose and it is efficient.  It is another example of CAS and one that works. Actually, don't need to say much here - just a beautiful thing.

Where Does CAS Work?

When contrasting these two examples of CAS, I am left with - where does CAS work and where does it fail?  Well, probably need more than two examples, but I can see one difference right away.  In the first example, beings were in a competitive situation with some form of intelligence and that situation was changing dynamically.  In the second, beings were working against nature and that situation wasn't changing. 

In the world of software, I suggest we're more in the mullet case than in the geese case.  We are in a competitive situation and our world is changing. Leadership is needed.  Someone who can see what is going on and how to react.  This is what makes me nervous about CAS as a model for Agile - except for the team - where it is probably fine.  But team Agility is not a very big problem to solve anymore.  I did that in my first Scrum implementation 8+ years ago.

Agile at the enterprise is much more complex. Scaling CAS may just mean more balls (schools) of fish are eaten by bigger competitors.

Alan Shalloway

Technorati Tags:
tag with del.icio.us

Comments

Where to begin...

On October 18th, 2009 marcrix says:

First, thanks for the post, Alan. I am always intrigued to know how others are applying system theory in their professional pursuits, and there are many great thought-provoking concepts in this article. I'll try to avoid diving down the rabbit hole, though, and just offer a few ground-level tips that have worked for me.

I agree with Pablo that CAS is not a roadmap or methodology. It is a set of scientific observations about the natural world that can offer insight into the dynamics of social organizations, such as businesses, but it is not prescriptive. In my view, CAS is the desired result, the item on the menu, and it is up to us to forge the recipe. So I do not agree totally with Pablo's statement that "the enterprise is a CAS." Rather, the enterprise is a complex system--a whole composed of many interacting parts--but the "adaptive" part needs to be earned and maintained. Most successful companies do begin as adaptive systems--innovative and opportunistic--but get locked into rigid, almost mechanistic, processes as they become larger and more established, making it harder and harder for them to adapt to the changing world around them. I believe one of the more significant challenges facing businesses today is how to retain (or regain) that agility in the face of rising global competition. CAS theory puts you in the right mind set but doesn't push any buttons or turn any gears for you. So how to become adaptive?

When it comes to adapting, all complex systems have one thing in common: evolution is a bottom-up process. CASs are distributed systems with decentralized control. There is no lead goose, head mullet, or central brain directing the behavior of all others in the group. Any member of the system can make independent decisions on behalf of the whole. Most companies don't operate this way, though. New strategies are established from on-high and new behavior mandated through top-down policy. Change does occur this way but I would argue that it is not adaptation. So, the most important lesson I have learned about applying system theory to business is that rigid, top-down, command-and-control style governance models are extremely counter-productive.

We don't need more leadership, we need more openness. In order to effect agility at the enterprise level, we need to address the flow of control.

I believe we need leadership

On October 25th, 2009 alshall says:

I believe we need leadership and openness.  We are not trying to do evolution - we are trying to have a directed journey of improvement.  Can you show me one enterprise that has positively transformed itself without leadership?  Bear in mind, that by leadership I mean creating a vision of a better space and then coaching/training people on getting there.  Not how to do it, but rather helping people get the tools/knowledge to do it.

Too many agile teams think that if we just work together we'll figure it out.  Not likely if you are a big organization.  For that you need to understand how big organizations work.  Yes, CAS is a part of it, but systems thinking, flow and other concepts are incredibly useful to get the components of the organization a common vision so they can work together and avoid local minimizations.  All too often teams solve problems that aren't the real problem - that which is keeping the organization from moving forward.  You need to see the whole.

Alan Shalloway, CEO Net Objectives

CAS and Natural Selection

On October 5th, 2009 ABlair says:
In addition to thinking about the contrast between the geese and the mullets, it is worth thinking about the contrast between the behaviour of the mullets in situations where it does work and in the situation you describe. My guess is that most of its predators are non-social, and that the bunching behaviour works against them. If dolphins were their only predator, there would be strong selection pressure to develop the behaviour of just swimming off in all directions. I'm not sure what conclusions might be drawn about this for software development - maybe something along the lines of: some Agile practices work in some situations, but not in others.

Another thing to think about is: Is there an analogue to natural selection in software development methodologies? If all its predators begin acting like dolphins, then either there will be the odd mullet who swims away to survive and reproduce another day, or else they're going to go extinct. Does something similar occur in the way software development methodologies get reproduced, or else go extinct?

Andrew Blair

CAS and the enterprise

On October 4th, 2009 pablo.emanuel says:
Alan, actually, at the enterprise level, the point is to realise that the enterprise is a CAS, and act accordingly. At the enterprise level, we (managers, process thinkers) are not the fish, we're the dolphins. We need to understand that just pushing the school to a direction won't make it go there, we need to find cleverer ways to make it move the way we want it to, like circling it. Regards, Pablo Emanuel

dolphins as CAS

On October 5th, 2009 alshall says:

Pablo, thanks for this.I'll think about it. But it seems the dolphins had a strategy they all aligned to. Where did that come from?

My biggest issue with CAS is that it appears to be a way of explaining what happens.  As a consultant and practitioner I need a way that fosters the proper behaviors to emerge.   The notion of flow, limiting work, optimizing the whole, ...  enables me to do that. CAS can explain things - but don't see how it helps me decide what to do.  Do you (or anyone else) have an example of where it is?

Alan Shalloway, CEO Net Objectives

CAS in practice

On October 5th, 2009 pablo.emanuel says:
Alan,
there are some practical principles I inferred from CASs that I use in my day-to-day work. I divide them in two categories:

Dealing with CASs (usually large groups such as companies, big departments, cities, countries, etc):
  • CASs have homeostasis, and move through jumps - it doesn't help to try to push a CAS little by little, either you cause an avalanche or they'll stay where they are.
  • CASs won't react linearly. Sometimes the best way with CASs is to act on the effects, not on the causes (since there's so much feedback going on). It's the technique of force a smile to actually get happier.
  • CASs doesn't have a natural scale, i.e., sometimes you have to act globally, sometimes you have to act on the detail to get the result you want.


Inducing on systems the good features of CASs:
  • Short and abundant feedback loops
  • To achieve robust efficiency, CASs must have:
    • Redundancy: Each feedback loop must have a secondary feedback loop, no individual can be irreplaceable, etc
    • Variability: The distributions must have fat tails, instead of focus on just what happens to be optimal right now, allow for some different sub-optimal behaviour to exist, which must be exactly what you need in the future. For instance, new ideas must have room to grow, parallel projects must be allowed to drain a small percentage of the resources, people should be trained in skills other than the ones you need now.

In general, I agree with you. Lean is a set of principles that translate very directly to a set of tools that give measurable results. CAS theory can give you insight, but it's definitely not a roadmap, so, comparing CAS and Lean is indeed comparing apples and oranges.

Regards,
Pablo Emanuel

thanks

On October 6th, 2009 alshall says:

Pablo:
Thanks. This helps tremendously. Gives me a completely different perspective of the value of CAS.

Alan Shalloway, CEO Net Objectives