For months I’ve been reaching out to colleagues to explore barriers to collaboration, a key tool in the social innovator’s toolbox. Among those queried (and in spite of diverse backgrounds), virtually all had experienced significant barriers to collaboration over the years including silo-thinking, dated and inefficient problem solving models, cultures of control, and a strong, prevailing lack of trust.

Consensus? The barriers to innovation seem to be as universal as they are frustrating.

So something is broken. What is the root cause?

Beth Noveck and David Johnson have published important research on how new Social Media collaboration technologies can change the game. Their perspective on a New Science of Complexity is summarized in this People & Place blog post and explained further in an excerpt from their research. Their focus was the U.S. EPA (including the Federal process for environmental research and legislation) but their conclusion, which I agree with strongly, is that the principles are applicable in business (#e20) and broader social venues (#gov20) as well.

My primary takeaway?  I now believe that INNOVATION IN COMPLEX ECOSYSTEMS will depend on an improved collaboration process – a new middle ground for problem solving – that balances large-scale central organizational approach with grass-roots contributions by individuals. It is about finding the “sweet spot” between rigid structure and adaptive, organic sourcing of ideas. In a new and somewhat uncharted public collaboration space, it means that the forces of organizational scale and leverage can be networked – connected – with discrete centers (or hubs) for contribution to produce more rigorous solutions.

At the core of this thinking? A realization that traditional large-scale organizations (with their central thinking, hierarchical layers, and silos of functional experts) are generally ineffective when dealing with complex situations. Quite literally, they are too rigid. Without the ability to adapt to new variables or to coordinate across silos, grid-lock ensues. And complex social ecosystems are impacted, since “sending in experts” is how we tend to attack these issues. On the list? The well known structural challenges in energy, sustainable food and water sources, public education and healthcare.

What’s needed is an outright paradigm shift in problem solving models that are fundamentally more interactive and cross-functional. And focusing on complexity theory is key, because it begins to unlock some new doors. For one, there must be an organic aspect that allows solution teams to learn, self-correct and grow. And to meet the requirement of connecting people more dynamically, Social Media is the ideal technology. Some examples? Think about experts engaged in live chat. Acceleration of thought synergies. Tools to merge and re-mix knowledge. Ability to leverage and extend dynamic repositories.

With focus and coordination, we can work to find the elusive “sweet spot”.

In terms of naming and framing the problem, the above research makes significant strides. The next step is critical as well, and is just as exciting: in pockets across the internet, the new collaboration is already starting to appear.

Are you seeing it too? Let’s talk, I’ll show you where and how.