Tag: software

The next generation of environmental software needs a vision and some help

ISEES1

At a three day workshop, a group of scientists explored a vision of the “grand challenges” that eco- and earth science face in the coming decade. Each of these challenges, if answered, would provide invaluable new knowledge to resource planners and managers across the planet. And every challenge contained a workflow that called upon software capabilities, many of which do not currently exist: capabilities to handle remote and in situ observations and environmental model output in order to incorporate multiple data layers and models at several resolutions, from a prairie to the planet. Water cycles, pollution streams, carbon sequestration, climate modeling, soil dynamics, and food systems—achieving the next plateau of understanding these processes will require a massive investment in computing and software. The reason for this workshop was to help inform a new institute that can provide key services to make this investment pay off.

Much of this software will be built by research teams that propose projects to solve these grand challenges. These teams will be multi-institutional and are likely to be more focused on the science side of their project, and less on the value their software might acquire by being built on standards, using best-practice coding, and ready for reuse by others. The history of federally-funded science software is crowded with abandoned ad hoc project-based software services and products. I’ve helped to author some of these. One of the federally-funded products (a science education software package) I helped produce had its home-page URL baked into its user interface. After the project funding ended, the PI did not renew the domain name, and this was picked up by a Ukrainian hacker, who used it as the front end of a pornography portal. So the software UI (distributed in hundreds of DVDs) now pointed students to a porn site. A far more prevalent issue is that of software built with 3rd-party services (remember HyperCard?) that have subsequently changed or died, breaking the software after the funding is gone and the programmer has moved on. The point here is that there are dozens of lessons already learned by science software developers, and these need to be assembled and shared with the teams that are building new software.

There is still more value to be added here. A software institute can offer a range of services that will make funded software more reliable, more reusable, and more valuable to science. Much of the federally-funded software development will be done by university staff scientists and graduate students. Most of the latter are in the beginning stages of learning how to program. A crash course on agile programming and Git, or other basic programming skills, could help them get up to speed over a summer. An up-to-date clearinghouse of data and file format issues and recommendations, a help-desk for common CMS and data access problems, and particularly, personal (Skyped) help when the grad student hits a wall: these services can save a funded project from floundering. All together, these services can save the project’s software from an early grave. Research into extending the lifecycle of science software is needed to help science maintain the longer-term provenance of its methods and findings.

Isees2

This workshop was organized by the team that is looking to build the Institute for Sustainable Earth and Environmental Software. Here is their website: http://isees.nceas.ucsb.edu

From carrots and sticks to donuts and heroin: what academic software producers need to learn from their commercial counterparts.

Carrot and Stick

I’ve spent much of the past decade managing software development projects. These projects can be sorted into two types. One type involves collaboration with academic organizations, mainly with government agency funding. The other type is with commercial partners and an eye toward the open marketplace. Software project management for both types is similar in most ways. Both types used the same agile software development process. The agile project management process includes a conversation about user experience and engagement. In fact, it starts with user problems and stories and use cases.

The notion of customer-driven design is a central feature of all good software development. So too is the goal of creating something of immediate use and widespread need. There are some differences that, when teased out, suggest arenas where academic (and other, open-source) software developers might want to learn something from commercial software development practices. The reverse is not as obvious at the software development level, but is more evident in the user licensing and IP level.  At the code level, the process of development and design for academic/government agency software can be quite different than commercial software. This difference is mainly a matter of user expectations. As Doc Searls noted, “…Microsoft needed to succeed in the commercial marketplace, Linux simply needed to succeed as a useful blob of code” (Searls 2012, Kindle Locations 2262-2263).

A couple of conversations in the academic software code arena can illustrate how far apart these two types are. In the first, I was told that “we can deliver this with the warts showing, as long as it works.” And in the second, someone noted that some combination of “carrot and stick” could be applied to make sure people used the software service. Compare this to the goal that Guy Kawasaki promotes for software: enchantment. “There are many tried-and-true methods to make a buck, yuan, euro, yen, rupee, peso, or drachma. Enchantment is on a different curve: When you enchant people, your goal is not to make money from them or to get them to do what you want, but to fill them with great delight” (Kawasaki 2011, Kindle Locations 185-187). No warts or sticks allowed if your goal is enchantment. In fact, not that many carrots, either.

I countered the carrot and stick suggestion with one of my own, “How about donuts and heroin?” In commercial software development, it’s not uncommon to ask “So, what is the heroin in this software?” The idea is that the customer would be so enchanted with the software that they would gladly use it every day. Even the worst experience should still be a donut, and not a wart, and certainly not a stick.

Certain realities do intrude here. Academic and agency software developers work on the tiniest of budgets. They tackle massive problems to connect to data resources and add value to these. They commonly have no competition, which means they solve every problem on their own. A “useful blob of code” is better than no code at all. But still, they might consider imagining how to enchant their users, and provide a few dimples and donuts along with the worts and the carrots. Because their users spend most of their digital lives on the daily heroin supplied by Apple and Google and Facebook, being handed a carrot may not do the trick.

Kawasaki, Guy (2011-03-08). Enchantment: The Art of Changing Hearts, Minds, and Actions. Penguin Group. Kindle Edition.

Searls, Doc (2012-04-10). The Intention Economy: When Customers Take Charge. Perseus Books Group. Kindle Edition.

Photo credits, CC licensed from Flickr:

carrot and stick: bthomso

carrot on plate: malias

donuts: shutterbean

eating donut: Sidereal