Lead your virtual organization from behind, and they will drive you forward

The back of the bus

In March, 2013, David Reid at Palantir tweeted a quote: “Drupal’s community is a bus driven by the collective screams of its passengers.” At first glance, this seems really unfortunate. But on reflection, there are some great things going on here. First, it’s a bus, not a limo. Second, it’s filled with passengers. Third, they are passionately engaged in the destination of the bus. And fourth, the bus is responding to them. In short, the Drupal community is doing a lot of things right. 

This tweet took me back to a sunny afternoon in Berkeley at BADCamp. I was in front of Dwinelle Hall (where I first met my wife way back before there was an Internet) talking with Jacob Redding, who, at the time, was the Executive Director of the Drupal Association. I had recently finished Jono Bacon’s book, The Art of Community, taken from his experience helping to lead the community of developers who support the Ubuntu distribution of Linux. I mentioned to Jacob that, compared with Ubuntu, Drupal did not seem to be grounded in a common vision that was written down as a statement somewhere.

Jacob said that the Drupal community doesn’t have a single vision statement but the leaders of the project each may have their own. The Drupal project does have a mission statement (drupal.org/mission). In the project there is a lot of non-visible leadership. The Drupal Association is a good example of that. The Association does not guide or direct the project, but it certainly does watch what is happening with the project and provides the necessary support and services to either sustain momentum on areas in the project or encourage more involvement. “You can say the Drupal Association is leading from behind,” he said.

There are visionaries across the Drupal developer network, dozens of active people who are building new tools and pushing for better code. There are occasions in the year (at big Drupal Camps or at DrupalCon) and moments in the development cycle (sprints and telecons, in discussions on Drupal.org and elsewhere) where these leaders would come forward with their visions, and, at some point, a decision would happen. But the Drupal Association, which runs DrupalCon and hosts Drupal.org, is not out in front breaking trail for these decisions.

I called Jacob last week to get some clarity on this “leading from behind” technique. I might be involved in a grand experiment on governance that the NSF has fostered. It’s called EarthCube, and it proposes to corral all of the geosciences and environmental sciences under a new organization that can manage and promote the multi- and transdisciplinary sharing of data and knowledge. To bootstrap this effort, a seed of governance will be funded. In the first months, this governance kernel will need to get out in front and open up online opportunities for thousands of scientists to step into new leadership positions and get to work. In the subsequent months, the funded governance kernel will need to step back and lead from behind as the community finds its own governance footing. In the final phase, the funds for governance will be passed directly to the community, and kernel will become a front office. But this transition from leading from the front to leading from behind is truly experimental.

“Part of it is allowing chaos to reign,” Jacob told me. The point, I would propose, is to not spend a lot of effort (and time) establishing and then defending a set of top-down decisions. He continued,“a lot of things go wrong when you have a benevolent dictatorship that says ‘this is the way’.” When the community feels the need to make a decision, the lack of a top-down process enables people step up to offer solutions.

Behind the scenes, Jacob noted, the leaders of the Drupal project pay very close attention to all of the blogs and the threads on discussions and comments, and keep track of who was working well with others and who was influencing their peers. It would be vital to have these people in the room for important decisions. This internal eco-system is fostered through support for travel to Drupal events, and through email lists and other back-channel communication. Out in the front channels, the Drupal bus riders are shouting out their visions and their peeves.

The other eco-system to which the Drupal project and Organization pays close attention is the larger CMS world, where big corporations and government agencies are looking to make long-term investments. “This way we can counsel corporate users, and use their feedback to counsel Drupal developers,” Jacob noted. One big part of leading from behind is also watching the road ahead.

You can find Jacob Redding at: http://jredding.info/

Photo Credit: by Guerilla Features | Jason Tester used under CC license on Flickr.

The next generation of environmental software needs a vision and some help

ISEES1

At a three day workshop, a group of scientists explored a vision of the “grand challenges” that eco- and earth science face in the coming decade. Each of these challenges, if answered, would provide invaluable new knowledge to resource planners and managers across the planet. And every challenge contained a workflow that called upon software capabilities, many of which do not currently exist: capabilities to handle remote and in situ observations and environmental model output in order to incorporate multiple data layers and models at several resolutions, from a prairie to the planet. Water cycles, pollution streams, carbon sequestration, climate modeling, soil dynamics, and food systems—achieving the next plateau of understanding these processes will require a massive investment in computing and software. The reason for this workshop was to help inform a new institute that can provide key services to make this investment pay off.

Much of this software will be built by research teams that propose projects to solve these grand challenges. These teams will be multi-institutional and are likely to be more focused on the science side of their project, and less on the value their software might acquire by being built on standards, using best-practice coding, and ready for reuse by others. The history of federally-funded science software is crowded with abandoned ad hoc project-based software services and products. I’ve helped to author some of these. One of the federally-funded products (a science education software package) I helped produce had its home-page URL baked into its user interface. After the project funding ended, the PI did not renew the domain name, and this was picked up by a Ukrainian hacker, who used it as the front end of a pornography portal. So the software UI (distributed in hundreds of DVDs) now pointed students to a porn site. A far more prevalent issue is that of software built with 3rd-party services (remember HyperCard?) that have subsequently changed or died, breaking the software after the funding is gone and the programmer has moved on. The point here is that there are dozens of lessons already learned by science software developers, and these need to be assembled and shared with the teams that are building new software.

There is still more value to be added here. A software institute can offer a range of services that will make funded software more reliable, more reusable, and more valuable to science. Much of the federally-funded software development will be done by university staff scientists and graduate students. Most of the latter are in the beginning stages of learning how to program. A crash course on agile programming and Git, or other basic programming skills, could help them get up to speed over a summer. An up-to-date clearinghouse of data and file format issues and recommendations, a help-desk for common CMS and data access problems, and particularly, personal (Skyped) help when the grad student hits a wall: these services can save a funded project from floundering. All together, these services can save the project’s software from an early grave. Research into extending the lifecycle of science software is needed to help science maintain the longer-term provenance of its methods and findings.

Isees2

This workshop was organized by the team that is looking to build the Institute for Sustainable Earth and Environmental Software. Here is their website: http://isees.nceas.ucsb.edu

Who says that “agile” is just for software? Architect an agile online community!

If you’ve familiar with object-oriented coding using any agile methods, you understand that the customer is at the center of the software development effort. The customers problems, her needs and goals, laid out in stories that the programmers return to every day: these keep the software from feature creep and UX failure. Delivering working software in small increments helps the customer reveal the moment they don’t need or understand something. The programmers can toss out that new feature and go back to the customer stories again with fresh insight.

They are testing all the code every day and releasing every week. So there’s never a time when the changing the software costs more than the last time. They are free to pivot toward some new capability that just might provoke delight in their customers; something they would never have tried if the cost of change was growing. Finally, they are solving their customers’ problems and building into the software new opportunities that might not have existed anywhere else in this way ever before. OK, so it rarely works out that well. But the philosophy of agile code development is sound and it offers valuable lessons beyond writing code.

Virtual organization community leaders would do well to consider how the Manifesto for Agile Software Development might be tweaked to be a working Manifesto for Agile Community Development:

Individuals and interactions over processes and tools

Working software (working volunteers) over comprehensive documentation

Customer collaboration (member collaboration) over contract negotiation

Responding to change over following a plan.

from http://agilemanifesto.org/

Let’s look at these one by one. Individuals and interactions are the daily work of the community leader. Staying in touch with members is more important than whatever tool (email, forum, listserve) is picked to accomplish this. Working volunteers build value and direction for the virtual organization. Having a grand, detailed volunteer guide is less important than having a team of members who want to move the organization ahead. Member collaboration means giving the membership as much ownership as possible. Staff are around to take care of day-to-day business, but budgets, planning, and the real work of the VO belong to the members. The staff are the sails, the members are the wind. Responding to change means having a full double-loop governance system and pivoting this to stay ahead of changes surrounding the organization. If you’re only following a plan, you are already lost.

Agile: it’s not just for software anymore!

photo credit: From flickr user magia3e  http://www.flickr.com/photos/magia3e/6236962059/sizes/z/in/photostream/

Sharing Creativity: Talk given at 2012 ESIP Federation Summer Meeting.

Here is a talk I gave at the recent (Summer 2012) ESIP Federation meeting. Sharing Creativity:

I am hoping that this talk will lead to some conversations over the potential for virtual organizations to achieve, with more efficiency and effectiveness, a capacity for creativity and predictable innovation. This capacity—in large part due to Internet-enabled capabilities for coordination and collaboration— can, I believe, rival (at various scales) the capabilities of dedicated R&D facilities/programs such as Bell Labs and Xerox PARC on the corporate side, and the Manhattan Project and the Apollo program on the government side.

Bohr, Oppenheimer, Feynman, and Fermi: Key requisite variety of knowledge assembled for the Manhattan Project. http://en.wikipedia.org/wiki/Manhattan_Project

Large research and development operations such as these were built as national laboratories, with hundreds or thousands of employees and forefront facilities. They were designed to assemble a critical mass of talent and direct this toward innovation. They were also enormously expensive: the Apollo program had cost more than $25 billion by 1973 (in 1973 dollars). The successful ones are rightfully famous.

Today’s top technology companies (Apple and Google, for example) often add to their innovation potential by buying forefront start-up companies, as much for their talent as for their technology. Their goal in a highly competitive market is to own enough talent, enough intelligence, enough creativity, to stay ahead of their rivals.

The basic business-school rule for improving the odds for successful innovation is to assemble a requisite variety of knowledge: a range of knowledge at least as large as the problem being tackled. The three ways to do this are the following: Hire it (add to your team); Grow it (reeducate your team); and, Buy it (purchase a rival company/team). All of these methods assume that you need to own the requisite variety of knowledge.

Science, on the main, has only one rival: the unknown. Scientists are relatively free to seek out new collaborators from anywhere. And, through Internet-based services, they are now enabled to become collaborators everywhere. This is one reason why the NSF has been promoting virtual organizations and research networks as the future of science collaboration (instead of building new centers at institutions). A good part of the potential that virtual organizations offer government and private funding agencies comes from a new logic for innovation: assemble and share the requisite variety of knowledge.  With the right sort of organizational governance and funding, a virtual organization can achieve what the older “think tank” R&D centers could: predictable, successful innovation.

There are some social aspects of the ESIP Federation that might be key to this capacity for creativity. These aspects are not secret, however, and can be fully copied and applied in other arenas. They are also not expensive (the Federation budget is remarkably small), but they are of great value, in that they have been worked upon by dozens of volunteers over the course of more than a decade.

Virtual organizations (VOs) come in many forms and sizes. The science of building and managing VOs is still being explored. There are many examples of early failures, and only a few examples that herald their potential success. Members of virtual organizations need to be sufficiently engaged to build collective intelligence. Take a look at the YouTube video and let me know what you think.

From carrots and sticks to donuts and heroin: what academic software producers need to learn from their commercial counterparts.

Carrot and Stick

I’ve spent much of the past decade managing software development projects. These projects can be sorted into two types. One type involves collaboration with academic organizations, mainly with government agency funding. The other type is with commercial partners and an eye toward the open marketplace. Software project management for both types is similar in most ways. Both types used the same agile software development process. The agile project management process includes a conversation about user experience and engagement. In fact, it starts with user problems and stories and use cases.

The notion of customer-driven design is a central feature of all good software development. So too is the goal of creating something of immediate use and widespread need. There are some differences that, when teased out, suggest arenas where academic (and other, open-source) software developers might want to learn something from commercial software development practices. The reverse is not as obvious at the software development level, but is more evident in the user licensing and IP level.  At the code level, the process of development and design for academic/government agency software can be quite different than commercial software. This difference is mainly a matter of user expectations. As Doc Searls noted, “…Microsoft needed to succeed in the commercial marketplace, Linux simply needed to succeed as a useful blob of code” (Searls 2012, Kindle Locations 2262-2263).

A couple of conversations in the academic software code arena can illustrate how far apart these two types are. In the first, I was told that “we can deliver this with the warts showing, as long as it works.” And in the second, someone noted that some combination of “carrot and stick” could be applied to make sure people used the software service. Compare this to the goal that Guy Kawasaki promotes for software: enchantment. “There are many tried-and-true methods to make a buck, yuan, euro, yen, rupee, peso, or drachma. Enchantment is on a different curve: When you enchant people, your goal is not to make money from them or to get them to do what you want, but to fill them with great delight” (Kawasaki 2011, Kindle Locations 185-187). No warts or sticks allowed if your goal is enchantment. In fact, not that many carrots, either.

I countered the carrot and stick suggestion with one of my own, “How about donuts and heroin?” In commercial software development, it’s not uncommon to ask “So, what is the heroin in this software?” The idea is that the customer would be so enchanted with the software that they would gladly use it every day. Even the worst experience should still be a donut, and not a wart, and certainly not a stick.

Certain realities do intrude here. Academic and agency software developers work on the tiniest of budgets. They tackle massive problems to connect to data resources and add value to these. They commonly have no competition, which means they solve every problem on their own. A “useful blob of code” is better than no code at all. But still, they might consider imagining how to enchant their users, and provide a few dimples and donuts along with the worts and the carrots. Because their users spend most of their digital lives on the daily heroin supplied by Apple and Google and Facebook, being handed a carrot may not do the trick.

Kawasaki, Guy (2011-03-08). Enchantment: The Art of Changing Hearts, Minds, and Actions. Penguin Group. Kindle Edition.

Searls, Doc (2012-04-10). The Intention Economy: When Customers Take Charge. Perseus Books Group. Kindle Edition.

Photo credits, CC licensed from Flickr:

carrot and stick: bthomso

carrot on plate: malias

donuts: shutterbean

eating donut: Sidereal

The Intention Economy: a window into the next phase of the Internet

Doc Searls

Doc Searls: The Intention Economy. used under CC license from dsearls on flickr.

I just finished Doc Searls’ latest book. This book is several things, all of them good. This is a knowledgeable look at the future of being a customer in a world where the Internet realizes its potential as an information commons (instead of a storefront). The book is simultaneously about being a consumer and a customer (not exactly the same), and about big data and little data (the data you should be in control of), and about the Internet and the economy. Doc introduces a new (5 years old or so) effort to create software services that enable customers to announce to the world their intentions, and to then receive bids from vendors who wish to sell the products and services that might be some value for those intentions. This is a reversal of rules and roles which currently lock customers into the loyalty silos that companies use to corral their wallets.

Every chapter in this book is a revelation on an important topic, from the coming collapse of the advertising bubble, to the need for customer-based contracts instead of the current lopsided boilerplate contracts of adhesion, to the Internet as a managed commons, which can support individuals owning their own data and negotiating with an open market for what they need: based on their own intentions, rather than from some expensive (in money and effort) algorithm devised to mine their data and ferret these out. Who knows their intentions better than the customer?

The new economy, based on fourth-party brokers that act on behalf of the customer —not the vendor—will be open (newcomers welcome, no silos allowed), efficient (no more guessing intentions, transactions are knowledge-full), effective (allowing vendors to work together), and it will bring the Internet closer to its potential as a free exchange of knowledge that can also support innumerable transactions and contracts. In the end, this is also a story of a work in progress, as Doc and others have already started to build software services to explore this new economy. This is an important work, that announces what could, and I would argue, should be a new direction for an Internet enabled economy.

As a bonus, the work is extraordinarily well written at the prose level, and is not simply a blog-to-book. Each chapter adds substantially to the overall argument. I cannot recommend this book too highly. I am encouraging friends and strangers alike to give it a read.

I would also submit that there are corollaries to the commercial vendor/customer relationship that Doc’s logic and services would help improve. How much better would civil society be if the intentions and the capabilities of citizens, and the problems they face, were announced in this fashion to their local governments? How much more effective would continuing education be if the student could announce the skills they require to the world and have multiple offers for training? The Internet as a managed commons (Doc does a great job of advancing Lewis Hyde’s work on the commons) extends to many facets of our social interactions, not just those that involve transactions for money. Doc does talk about micro-transactions, but there are also new efforts to enable a sharing economy that would benefit greatly from these services.

Doc Searls: The Intention Economy: When Customers Take Charge. Harvard Business Review Press

http://www.amazon.com/The-Intention-Economy-Customers-Charge/dp/1422158527

Hulk want Negroponte Shift in publishing now

It’s one of those weeks where the clanking chains of the ancient devices of academic publishers have been more than a bit annoying, and it looks like no amount of WD40 will smooth the transition into digital delivery without first demolishing these anachronistic machines and their devilish DRM schemes.

This morning on NPR there was a bit on how public libraries need to subscribe each year to access the same digital files for eBooks, in order to provide these in serial increments to individual users. No overdue books here, the narrator notes, the digital files simply disappear from the user’s device, forcing them to queue up (and wait for weeks or months) until the digital file is again available. The more popular the book, the longer the wait.

I also had the opportunity to search for and find a book published by an academic press in Europe, and was informed by their website that I could download a digital copy for only $100+. Makes the iTunes bookstore seem cheap.

And then, with a link from William Gunn, I made myself read a response to the open access demands by scholars, students, and libraries, by a (IMHO fatuous) mouthpiece of the publishing industry in the Guardian. At one point he justifies the bloated profits of the industry by noting that they pay taxes on these (not if they can help it) and the government uses these taxes to fund (wait for it)… research. For profit academic publishers are the “research producers” that keep the wheels of science rolling. Lord help us if the (socialist) open access lumpen masses get their way.

On the more hopeful side, John Wilbanks and the Open Access Gang of Four are most of the way to a successful open access for research petition on the White House petition site. And if only the intern who programmed the Drupal user authentication for that site had hooked in a better module, then it’s likely that the necessary 25,000 signatures would have already been accomplished.

At last look, over at The Cost of Knowledge, 11,923 scholars have pledged to not give their services to Elsevier.

And, today we will find out if Redditors can oust Texas representative Lamar Smith, who co-authored SOPA.

We are all waiting for everything to digital and available and searchable and browsable, and linked, and curated, filtered and yet without the bubble, semantically rich, and, of course, free. We are not there yet, plenty of cruft to clear away. Time to point the Hulk at the entire academic publishing enterprise and say “SMASH.” It couldn’t hurt.