Government R&D Spending
Understanding that (and a few other things), let's you know why government R&D spending is such a waste.
Technological innovations has certain speed limits.
One is diminishing returns. Pouring 10x the money may get you twice the returns, and there are certain barriers that just won't budge.
In 1946 Dick Tracy showed the two-way radio wrist-watch (upgraded to two-way TV in 1964), and everyone wanted one. But we didn't have cell phone networks and smart phones yet. And no matter how much we wasted on government R&D, you weren't going to get it before we invented integrated circuits in 1958, and 40 more years of generational evolution to get the transistor count high enough that you could do that with a cheap app on your smart phone.
You just can't advance faster than the supporting technology can advance. And politicians and their constituencies aren't good technologists at knowing what's possible (ready) or not.
For example, Let's say you wanted to build a self-driving car in 1980. Sounds like a good idea, many people had thought of it by then. Let's imagine this person can convince the government to sponsor it to the tune of $1T/year (or our tax dollars) to make this a reality. What happens? You waste $1T/year. You hardly dent the speed of innovation at all.
Self driving car
What you find out that in order to solve that problem you need: Really big and fast computers (parallel processing), lots of software (and new languages/methodologies and tools to support them), sensors (Optical Cameras, LIDAR, etc), something like GPS and maps of everywhere you want to drive, and a public that's comfortable with trusting technology.
None of those supporting technologies is ready. So you can't get there for here, even with trillions of dollars in subsidies, you need time (it'll take 30+ years at that burn rate).
But if you wait until 2020:
- Moore's law is fixing the computer problem -- by doubling transistor count every 18-24 months, in 40 years, the computer hardware you need drops from $100M per car (and would take a semitruck to drive around, along with its own power station) to $100/car and can be run off the battery.
- Software invented new tools and processes to take advantage of parallel computing (GPU's), and is now free.
- Consumer imaging is solving the camera sensors, first in digital cameras, then in smart phones, cars, and many other things. The sensor technology that didn't exist in 1984 are widely available and so cheap, it's near free.
- The government created GPS for weapons systems, and it went operational in 1995. It became consumerized, and chips to know where you are, became virtually free.
- The Internet developed, as did networking, and Google gave us a national online maps, and after years it got debugged to the point of being usable. You now have maps you need, for free.
- The public got used to computers and technology and trusting it far more than they were ready to in 1980.
So the point is something that was impossible in 1980, is not impossible in 2020. All the money in the world, couldn't make it a solvable problem in 1980 -- or worse, the ways you solved it (like only allowing special lanes with magnets in the road to guide the cars, and computer systems outside to manage the cars, more like trains), would have crushed your economy to implement, and most would have been completely throw-away money once the technology advanced enough to not need all the gimmicks and hacks you used to avoid waiting a few more years.
And that's the problem with Government R&D -- it's based on politics, not science. It's based on what you can tell people they want, and not based on what's technologically ready (or near ready), and they will buy.
Government R&D is for people who can't do math
There was a whole big fiasco of "GM Killed the Electric Car?" -- but I'll let you in on a secret. No one in 1997 wanted an electric car that cost $55K, sat only 2 people and had a 70 mile. Well, no sane people. A handful of early adopters and granola munching earth-first types liked them -- but the public at large wasn't going for it. The Tech wasn't ready, and people didn't want them when gasoline was $1.19/gallon. No matter how much government poured into subsidies, people weren't going to widely adopt them.
Now in 2017 Tesla is going to try to give us a car that seats 4, costs 25% less (entry), has 10x the performance and 3x the range, comes with far more amenities, Gasonline is $2.50/gallon, and it's still iffy on whether it'll be a commercial success yet.
The problem is the electric car was mostly dependent on battery innovation. No matter how much money you pour into the car development, until batteries are ready to give it a manageable range and a manageable price, it's not a viable product, no matter how much that hurts the feelings of the activists who wish otherwise.
Some people will say, "well the money can be diverted to battery innovation" -- but that's a waste too.
R&D has these critical mass levels. Computers, cell phones, tools, electronics, (now cars), they all are dependent on battery improvements. So there's already $50B+/year in battery sales, which means huge investments in battery R&D -- with or without government subsidies. If you know a battery innovation could be worth $10B dollars, government making it worth $15B isn't gong to motivate you any more. Once investment hits a certain threshold, you're not waiting on it directly, you're waiting on things around it to innovate. In this case, Battery development is dependent on research in material sciences, or some happenstance breakthrough in some other field that lets as make big leaps -- all the money in the world, isn't going to change the rate of that secondary and tertiary innovation.
In fact, governmental subsidies and dollars, not only does help, it can hurt. Government money can displace private investment, since who wants to compete with government? This is the Treasury View in economics. Or the other thing is they get people focused on the wrong thing. So some politician thinks Lithium Polymer is the right way to go and subsidizes it -- and everyone chases those dollars. That pulled money out of other research they would be doing. And it turns out the real breakthrough was in Lithium-Glass (if John Goodenough's claims pan out). Or vise versa. The point is that the politicians aren't basing their investment on knowing the technology or good business cases, but emotions, politics or what someone told them. Their odds of success are lower than average: as proven by decades of planned economies under performing free'er ones.
Economics of the past
Look at something like the Internet. The government poured half a billion dollars per year (starting in the 1950's) up to $3B/year today. Doing some napkin math, they spent $30B to created"the Internet" --really just a network protocol (TCP/IP) that replaced other private ones. People who don't know better, think that's a bargain, but a private company could have developed a similar protocol for maybe $500K on the high end. So the naive are bragging that the government "gave" us the Internet, for only 60x the price the free market could have done it? And without it, a few other competing private protocols like SNA, DECnet or OSI would have won, and it wouldn't have mattered at all to consumers. Oh boy, such a deal.
Government/ARPA research gave us the Internet the same way they invented the car or airplane. By 1976 (founding of ARPA) we had hundreds of computers networked, by 1993 the Internet only carried 1% of the information traffic (and we had plenty of traffic). But by 2007 that had flipped and most traffic was TCP/IP based, because it was free, standard and good enough. However, without TCP/IP, one of the other protocols would have become a standard, and we’d still have had everything we have today (in some areas, more). The government gave us nothing that we didn’t already have (or wouldn’t have). Politicians (as usual) took credit for other people’s work.
And that's the problem. Non-economists will look at the rare returns and ignore the frequent costs. Smarter folks can think what were the costs versus the returns? How much money did ARPA/DARPA throw at things that didn't pan out, that have to be added to the cost of the successes? How much would the alternative have cost if there had been no DARPA to fund it? Our job as rational humans is to always think, "compared to what?"
It's like this story of the Space Shuttle. Some folks love it, it sounded great a reusable space plane that cost 10x what a rocket costs to refurbish it between flights, and it's fragile and less reliable and more expensive in every way than the alternative. Deeper thinkers imagine what we could have had, if we hadn't poured so much into that boondoggle.
In the future, I'd love to do research that shows the rate of solar panels advancement before and after Obama poured billions of OUR dollars into R&D. Did they even nudge the needle at all over prior innovation rates? Can we show any rate of change at all, or was that money just wasted?
I doubt forcing us into investing into politicians good intentions had any positive returns, but I haven't done the research yet. What I do know is that historically, and from everything I've seen in the past -- politicians aren't where America should invest it's faith and money. But trust the invisible hand, free markets, and that most things will innovate on their own, without government. And the very few government successes (as measured by ignoring all the invisible costs) can never make up for the many more government R&D failures. When the private sector makes those bets and loses, only the investors pay. But when the government forces us to invest, then we are the ones who pay. And I don't think poor coal miners should be forced to invest in Solar Power boondoggles like Solyndra, if they don't want to.