by Kent Garber

A funny thing happened when Steven Chu arrived to head the Department of Energy last January. Chu, a physicist, got his start at AT&T Bell Labs, once a dynamo, a real powerhouse of technology innovation. That's where he did the work that won him his Nobel Prize, and he has talked ever since about what a great place it was and how quickly things got done there.

When Chu got to Washington, he wanted DOE to hustle. He wanted to move quickly, for example, to give loans to clean energy projects. He asked some of the career employees how long it would take to start handing out money. He was told 24 to 36 months. Chu was shocked. "That's not acceptable," he said, according to someone familiar with the exchange. "We are going to get it done in three." (The first loan went out that March.)

People who have heard this story chuckle because they say that it's a good reminder of how slowly the government moves. But it's also a tale of how slowly the energy landscape in America is changing and how people like Chu, who are itching to change it, are frustrated with the pace. The United States, as pretty much anyone will tell you, is racing China to develop clean energy technologies--the wind turbines and solar panels and advanced batteries and energy storage devices of the future--and there are a lot of people who are worried that the country is lagging, even losing, when it comes to innovation. Over the past 20 years, the amount of money that government and private companies have spent on energy research in the United States has tanked. Meanwhile, scientists and engineers increasingly fret about what they call the energy "valley of death," a figurative place where promising ideas go to die because they simply can't get the funding they need to advance.

Chu knows this is a problem. At the center of his effort to remedy it is a radical new program called ARPA--E, or the Advanced Research Projects Agency--Energy. The idea for the program came from a report Chu contributed to in 2005, and Congress authorized it in 2007 but never gave it any funding. Then ARPA--E got $400 million in President Obama's stimulus package. The name doesn't reveal much, except that it's modeled after the military's wildly successful Defense Advanced Research Projects Agency, known as DARPA. It was created in 1958, a year after the Soviets launched Sputnik into space, which sent Americans into a state of soul-searching panic about not being technologically superior. Vowing never to be caught from behind again, the U.S. government launched DARPA "to prevent technological surprise to the U.S." (The mission has since been expanded to creating "technological surprise for our enemies.") DARPA gave birth to the Internet and is credited with making major strides in advanced aircraft, missiles, and surveillance, just to name a few fields. In other words, it turned panic into dominance.

Much of DARPA's success reflects its structure and the way it approaches innovation. Its annual budget is about $3.2 billion. By design, almost all of its research is undertaken at universities and private labs, not government facilities. The Defense Department puts out calls for proposals on a certain topic--say, shape-shifting materials--and funds the ones that look most promising. The ventures don't last long, perhaps a couple of years, and then the Defense Department moves on to something else. If a project fails, that's OK. If it succeeds, the government can buy it. The whole point is to foster breakthroughs quickly, to recruit the brightest talent, and to avoid all the impediments that tend to make "government innovation" seem like an oxymoron.

Energy innovations. Can this approach do for energy what it has done for the military? Can it help usher in a new industrial revolution, this time in clean energy? Chu chose Arun Majumdar, a former colleague at Lawrence Berkeley National Laboratory, to lead the agency. Majumdar now has a staff of 12 working under him, including some of the sharpest minds poached from the private sector. "ARPA--E is sort of this infant baby that's coming out and needs to be nurtured," says Majumdar. "I am like the baby sitter, and as with any baby, there are many issues."

ARPA--E's first big move was in October, when DOE announced that the program was awarding $151 million to 37 projects. They are spread across the country and range from schemes to produce biofuels from algae to a new battery design that could make storing energy from renewable sources a good deal cheaper. Majumdar has since put out a second call for proposals, with $100 million up for grabs. That will leave about $150 million to spend.

The proposals all have two things in common. First, they're far enough along to be promising, but they're still too risky and have too many unknowns to be attractive to the business community. The ARPA--E folks like to talk about innovation as occurring along a spectrum from 1 to 10. A 1 is an idea, a concept, that still needs a lot of basic research; a 10 is a product that's ready to be sold. And here's the fundamental problem that energy researchers often face: There's funding at the low end of the spectrum, and there's funding from venture capitalists at the high end, but there's not a lot of funding in the middle, in the 4 to 6 range. Hence the moniker "valley of death."

The projects' second commonality is that each could be transform- ative if it panned out. Take 1366 Technologies, a Boston-based company that got $4 million from ARPA--E. It has proposed a new method for making silicon cells for solar panels, which convert sunlight to electricity. Traditionally, silicon is hardened and cut into pieces, a process that results in a lot of silicon being lost as dust. 1366 Technologies would bypass the cutting. If the idea works, it could slice the cost of making solar panels in half, a potential game changer for the industry.

ARPA--E is closely monitoring the progress of its funding recipients and coaching them for a set period of time (1366 Technologies' contract runs 18 months) But the agency doesn't grow too attached. "Something could be a good idea on paper, but it may not work out," says Majumdar. "They may fail, and we'll learn quickly from it and move on." If a concept does work out, Majumdar's team will help market the product to buyers.

By most accounts, the innovation "valley of death" is only growing bigger. Venture capital has been hit hard by the recession. But more fundamentally, there's been a decades-long trend of declining investment in energy research, both from the government and from private companies, in part because energy was cheap and not a national priority. "The last big surge in energy research happened in the Seventies," says Majumdar. "That's not to say nothing happened in the Eighties and Nineties, but frankly, energy research and development was not a focal point for our country for about two and a half decades."

Long odds. Peter Adriaens, a professor of entrepreneurship at the University of Michigan's Ross School of Business, says most of the technologies that get developed today at U.S. research universities will never see the light of day. "Of all the inventions, the fraction that gets venture capital is about 2 percent," he says. And the fraction still in business after five years? A minuscule 0.01 percent.

In the old days, successful innovators followed a well-defined path. Greg Yurek, the CEO of American Superconductor, a Massachusetts-based company that designs wind turbines and high-voltage transmission wires, is a good example. In the 1980s, Yurek was a professor at the Massachusetts Institute of Technology. Around that time, scientists discovered ceramic superconductors, which transport high currents of electricity. Yurek and his colleagues invented a way to make flexible wires from this material, which they could bend into coils for motors. In 1987, he got a patent; within a month, he had formed American Superconductor, "literally in my kitchen," he says.

To get the company off the ground, he found venture capital. Yurek's company is now publicly traded, and business is good. But he admits that if he were starting out today, funding might be more difficult to find. "Venture capital is really taking it on the chin," he says. "Lots of groups have folded up." That doesn't mean Yurek is sold on the idea of the government taking over part of the job, however. "I think the jury is out on this right now," he says of ARPA--E. "You have some pretty enthusiastic guys within ARPA--E who want to do the right thing. They've got money to spend and throw at the wall and see what sticks. It could serve a purpose of seeding some good things."

The United States isn't the only country playing around with the question of innovation. So is Europe. So is Brazil. So are Russia, Indonesia, and, perhaps most important, China. Each country is trying a slightly different approach. Europe, for example, is trying to grow companies through university labs. What's interesting, from an American perspective, is that ARPA--E, as much as it draws upon DARPA's example, actually has a lot in common with the Chinese model. "They've set up all these incubation technology parks, all seeded with capital from government," Adriaens says. "They call it venture capital, but it's not really because they are not seeking returns."

The Chinese model has its flaws, which should serve as something of a cautionary tale for ARPA--E. "It's really a bunch of people sitting in a room in Beijing saying, 'Where do we want to go? What's important?' and deciding to put the money there," says Yurek. "The bad news for them is that they are not always right. They don't always pick the right thing."

Majumdar says he's trying to avoid that fate by funding different teams working on the same goal and letting the best approach win. "We don't know ahead of time which one is going to work in the market," he says. "So competing technologies are developed to the point that business can take over and someone makes money out of it."

That last point may be the most important one. Profit and risk-taking have long been the bedrock of American innovation, with high-quality products rising to the top. The lack of that is one reason why China, though it has a knack for producing low-quality goods cheaply, so far hasn't surpassed the United States at innovation. But it's trying. ARPA--E, with its focus on risk, with its embrace of failure, may end up being what the country needs. There's still one problem.The agency has $400 million from the stimulus, but it has never gotten a penny through the regular budget process. And stimulus money won't last forever.

Available at Amazon.com:

The Next Hundred Million: America in 2050

Jolt for Energy Innovation: Government Investing | Kent Garber