framework for decision making - understand complexity

Harvard Business Review has an article by David J Snowden and Mary E Boone in their November issue titled "A leader's framework for decision making".

The theme (in short) is the need to shift management methods (they use the word science, but I'll leave that alone for now) from the usual simplification of an assumed structured reality to accept and assimilate a complex reality.

Good is that, have noticed that complexity exists, and am reminded about that frequently when discussing enterprise systems with some of the big enterprise software vendors. That said I mostly see such uttering as an alternative to arguing.

Allow me to quote a part from their article:

"Today, advances in complexity science, combined with knowledge from the cognitive sciences, are transforming the field (scientific management) once again. Complexity is poised to help current and future leaders make sense of advanced technology, globalization, intricate markets, cultural change, and much more. In short the science of complexity can help all of us address the challenges and opportunities we face in a new epoch of human history."

Promising much, lots of science involved I see.

Then they add some of the characteristics of complex systems (to which I agree fully, here the short version):

• Large number of parts.
• Large number of relationships.
• Non-linear relationships, ripple effects, dynamic.
• System has a history.

Good. Or?

Although I agree heartily that assimilating complexity is of utmost importance, there is a small bump in the road:

Today we represent reality's parts and relationships in any recording method (paper or IT) by a roster of event and/or transaction reports.
Want to "know" a widget in the warehouse? Prepare to flip through folders - order sheet, shipping papers, transaction reports, reports, more reports. Parts represented by a stack of event reports, relationships by transaction reports (aka accounting) - in other words increasing the parts in the model (or record) far beyond the parts and relationships that exists in the reality. Bad move.

Allow me to put it more succinctly:

"How in Earth's name do anybody really think they can get a grip on complexity when the add complexity to complexity, usually called complicating things in the worst and most elaborate way??"

Sorry to say, quite the normal way to approach an issue - do not check the fundament just add "science" on top of any wobbliness. Start with the fundament first please.

(If you wonder, thingamy represents one-to-one for parts of any complex system and have precise multidimensional dynamic relationships. It's all about singular things as we say...)

p.s. Next time meeting my friends at the big enterprise systems developers and hear the "remember that business operations are very complex, leave that to us big chaps" I'll smugly retort with "selber schuld!" (your own fault!).
Will behave though when here at SAP's influencer summit in Boston, they again graciously cover my travel costs and they are really nice people.
But after all, what is not friends for if not to tell the truth? ;)

Comments

[Charlie Wood]

Great post, Sig. But the argument can be made that to deal with complexity, one must "fight fire with fire" and apply complex management systems.

I suppose the question is whether the application of such systems improves clarity into and control of the total system or not, both immediately and over time. I suspect there's a net gain in clarity and control in the short term and a net loss in the long term without continued, increasing investment in the management systems themselves, which starts to sound like a great business plan for a vendor of such systems. :-)

-c

[phil jones]

I started working on stock control / materials planning software about 15 months ago. I came from thinking in terms of rather idealized database design. And one thing that shocked me was the amount of copying information around and how un-normalized everything looked. It was explained to me, and I started to perceive, that a lot of this exists to cope with *history*.

So, when you order 10 widgets, you make copies of the widget price and unit of measure that you buy widgets in etc. in transaction files because all of these might change. Next month widgets may cost 5 pounds for a box of 500 instead of 12 pounds for a box of 1000, but you'll need to know the details at the point when you bought them if you want to verify the delivery or have accurate accounts.

If you switch from storing boxes of 10 to storing boxes of 20, and your stock-control counts the number of boxes, you can't allow it to imagine that your existing stock suddenly doubled overnight.

If you didn't have to worry about history, and were only concerned with modelling the state of your enterprise here-and-now, then I think you could have your radical simplification to one-to-one parts-in-software to parts-in-world. But a lot of that transaction-oriented representation exists because of this need.

Another way of looking at it, a Thingamy-type system needs to be able to "abstract away from" history or fold it out of sight in some way in order to be able to escape the complexity of transactin files.

How well are you managing that? Seems some clever high-level Lispy magic might be able to manage it if anything can.

[sig]

Thanks Charlie!

Thing is that a managing system has to manage some "things" (ehem :) that represents something real out there - so how smart or not, how complex or not that managing part is will always depend on how the "things" in the systems represents the real "things".

In other words, that's where the design should start, on how to represent the real things (parts per complexity theory).

To put it like this: 2 parts gives 2 relationships, 4 gives 6, 8 gives 28 etc (think I got that right).
One widget is represented by order sheet + shipping paper + reception report + umpteen transactions in accounting by the ubiquitous method of event/transaction reporting.
One widget is represented by one unique data-object in an object-driven singular object based model.
That's 4,5,12 parts instead of 1 single one - just imagine the increase in numbers of possible relationships! That is complication (increase in complexity) for you! :D

Phil,
another good point, and a resounding yes to what we have thought of (heh, if good enough, well... we hope so, if not we fix!) - history.

Every singular data-object that represents a real world object will capture all that happens to it and keep that forever - time, who, what. That's the only way we can use these for creating accounting reports later.
Add that we never allow any object ever to be deleted or changed without it being captured forever - that way the History browser can be used, or logs checked out - practical but also a must to handle SOX, Health requirements, Basel II and whatever etc. thrown at it...

Excellent, now off I am to see the SAP'ies with a glass of wine in hand... bet the theme will be brought up at some point ;)

[Dave Snowden]

I think you are confusing complex, with complicated and the article advocates a series of simple measures to deal with complex environments (shift from fail-safe design to safe-fail experimentation for example). Taking an SAP type approach for a complex environment (treating it as complicated) makes life worse not better. Naming and cataloging things can be useful, but if take to excess to create a static system which cannot change.

[sig]

Dave,

apologies for what can easily be seen as a tad negative to your approach! Certainly not meant as such, so all my fault if it gives that impression!

I do humbly differ on whether it's better to "patch" and soothe a problem or to dig down to the problem's root and go from there. I, as you can see, suggest there is more, perhaps even far more to fetch from that last approach. Admit that it is not an easy one and spans more areas of expertise than what normally is termed management theory.

As to complex and complicate - I'm leaning on Oxford American Dictionary which agrees fully with your list of many parts and all their many relationships for the term "complex".

Now, "complicate" as verb is therein defined as "making (something) more difficult or confusing by causing it to be more complex".

And that is the whole point of my post - using a model of multiple objects (event reports) to represent one real object increases complexity extremely fast which as we know makes everything more difficult and confusing, and thus makes it more complicated.

And I do agree that taking the approach that all current enterprise systems take is exactly that, event reporting, with the inevitable result of much unnecessary complication. The thing is that is no different from how we handled the same reality by record keeping pre IT with pen and paper. Actually that was the only way and, sadly enough, the model for the IT systems.

By the way, as you will see in other postshere - any kind of categorising (cataloguing, taxonomy) is no friend of mine ;)

[phil jones]

OK, so can you give us an example of how you query the histories of objects in Thingamy?

For example, imagine I have a simple stock-control for a warehouse in which I keep hammers in bin 1.

So would I represent each item-type-per-location with one object? So I would have one object that represents "Hammers in bin 1"?

Or would I be more likely to represent each box of Hammers with a different object, each related to the "Bin 1" object?

Those hammers got bought at different times for different prices. And this information is presumably folded away in the history of the object, so what would it be like to write a query across all the hammer-purchases in order to find the average price?

Is it more like an SQL query? Or something you construct in the GUI? How can I *refer* to the historical "parts" in my query if we're trying to hide those?

Actually, maybe that last question raises what I imagine is the hardest thing to get right in Thingamy. You want to simplify, which means hiding unneccessary complexity (like, say, history) when you build your model, but you're also going to have to make that complexity available when you *do* need to engage with it, eg. in defining your queries.

[sig]

Phil,

currently we have a "Historical browser" interface as part of the navbar - more as a working example.

It's rather simple, set a time and apply historical view and the current view will change to what it would have looked like at that precise moment, including layout and even deleted objects reappear.

For your "query" we can thus use the reports that are user defined templates that when invoked picks up objects following template rules to display as wanted.
Thus you could have a report that shall display all objects of type hammer for location warehouse, and perhaps even some more user real-time-input (say producer shall be [user input]).

Click on that report, add any more filters if requested and it displays for this point of time. Then choose another point-of-time in the Historical browser and apply - suddenly the report changes to what it would have looked like at that point-of-time.

As said, historical browser is currently more of an illustration of the ability and power but can easily be as method in any number of features when we know what such should be with user experience. Obviously also for the display of a report - choose report date and time.

Hope that gave you an idea :)
(There is a screenshot under 4. at http://thingamy.com/screens.html)

[David]

Thoughts on definitions:

Complexity ... the objective quali/quan-titative analysis or value.

Complication ... the subjective 'impression'.

Something that is complicated to one, may not be so to another. Both, though, or equally complex.

***

I agree wholeheartedly with the need/desire to simplify, and to do so by digging deep and getting to the heart ... through design. It is the design that should embody the complexity in order to simplify for the user: which is what makes the design(ing) so difficult.

Your work (and intentions) with Thingamy are greatly appreciated. As a designer/entrepreneur (Toolmaker, with no IT experience), I am most interested in it's application to my own work in designing a corporate structure/system ... itself, designed as a complex system, to facilitate dynamic interactions. Long story. The point is, your work holds immense potential.

I appreciate all I learn through your postings. Very much (both the learning and appreciation!).

Thanks.

[sig]

Thanks David!

Complication is indeed a subjective one, good observation! And in a sense it follows from the dictionary as it being "making things more difficult by increasing the complexity" - and "difficult" is highly subjective!

And from that, and as we mostly understand it, it's a negatively laden word, nothing that we'd like - making things more difficult than it needs to be never would :)

Complexity though, at least for me, is a positive, something spurring interest, the stuff that some of us at least thrive on when we try to get our heads around. Understanding and learning is always fun! :D

That said, when you raise "simplify" - sometimes complexity is good and need no simplification as long as it reflects reality, but the simplification is required when it's complicated, back to reality so to speak, but not beyond!

[phil jones]

Hmmm ...

If "complication" is subjective (ie. it's not a real fact about the world) then the problem is in us, rather than the thing we're looking at?

Are you sure you want to say that?

Doesn't seem like that's compatible with saying that some software is "objectively" too complicated and we can / should design easier software.

I'd say that certainly, whether we're overwhelmed by the real complexity of something, whether it's too complicated *for us* - is subjective. But that sometimes that's the fault of us not having good ways of thinking about it. (We haven't learned or noticed the right abstractions that can help make sense of it); whereas sometimes it's the fault of the designers who layered an overly complex interface on top of a less complex underlying situation.

[sig]

Phil, you're right, came over wrong perhaps - think I'll stick to the complex - reflects many parts and many relationships: That should be objective.

Complicate - make things more difficult by adding complexity. And that happens for sure. The subjectivity lies I guess in the "more difficult" - as adding complexity in itself should be objective, measurable, possible to handle. But not very smart of course.