At school I was taught that the advantage of the private sector over the public sector is that it has the profit motive pushing firms to become more efficient. But this isn’t really true; when firms have monopoly power they can and often do charge high, and pay their management and owners more. They price higher and sell a lower quantity than the allocatively efficient outcome.Furthermore they can become complacent and fail to invest in reducing their costs. The result is inefficiency, at least party caused because the firm is profit maximising. Also, the public sector can fairly easily use incentives to become efficient; for example, performance based pay or allowing government owned firms to retain and reinvest their profits.
The real difference is the market gives more opportunities to mess up and fail. If 500 firms entered the phone industry, only the best 10 survived the next five years. There are some good ideas and many bad ones; the good ideas flourish and the bad ones die out quickly. The market essentially provides feedback on the quality of ideas, methods, products, and so on. By allowing itself to fail, the market can make progress; experiment with new risky but potentially effective production methods for example, and when one works, the other firms in the industry are forced to adopt it to stay competitive. When a firm goes out of business the market as a whole doesn’t suffer; another firm will expand to take its place.
The way firms in markets adapt is similar to biological evolution. In both cases, solutions arise from failures. An unguided process leads to extraordinary complexity. But biological evolution has many selection pressures; the free market has just one, maximising efficiency. (In fact it’s not quite that simple. Firms can aim to maximise profit, minimise costs, maximise market share, or service to the consumer.)
On the other hand, the government in a traditional command economy has no such opportunities; it has a single plan such as Mao’s great leap forward. There are few opportunities to experiment and fail, so there will be few good innovations as well as few bad ones. In a (healthily competitive) free market, inefficient and ineffective firms quickly go out of business and are replaced. But the lack of feedback or information on much of government spending means that it can be extremely ineffective or inefficient, yet still carry on for a long period. Even if the government’s provision is initially effective, it will fail to adapt and develop over time.
But there is no reason why this has to be the case. Just because governments have been run in a low risk, low reward way so far doesn’t necessarily mean it has to be this way. Could the government be run more like a market? It needs to retain the welfare of its citizens as its main objective, rather than profit. However, it is possible that different branches could trial different methods and innovate, and then the most successful approaches are universally adopted.
One difficulty is how success would be measured. Whenever you start measuring something, the incentive is to optimise for the measure itself, rather than the thing it is trying to measure. This is the reason why schools can become like exam factories as they try to move up the leaderboards. Using profitability as a proxy for success would essentially be a return to the profit motive; in which case it might as well be privately provided. While no measure is perfect the best one will probably measure the value to the public. Successful prisons are the ones whose prisoners have low reoffending rates and low chance of escape. Successful hospitals have high success rates and low waiting times. Clearly the cost needs to be considered, but better results should be valued even if there is no obvious, tangible way to measure them.
Drawing all of this together, government provision could become far more effective in the long term if there are several different government providers who are in competition with each other. It is also important to foster an environment that is friendly to innovation, risk taking and new ideas. If the competition and culture of innovation both exist, then good ideas will be developed and adopted, while bad ones will be quickly trialled and discarded. However, this depends on the success of the providers being measured accurately enough to determine which ideas are good and which are bad.
All of this is done in a healthy competitive market anyway; competing firms innovate in the hope of getting ahead, and successful innovations make the firms that adopt them successful, so it is natural that those innovations will be fairly quickly adopted by most firms in the market. The difficulty is organising government provision in a way that artificially reproduces these conditions. Government provision needs to become more like a competitive market, but crucially while retaining the welfare objective over the profit objective. (Read: serving the public instead of itself.) While firms in free markets ‘evolve’ towards optimising profitability, government providers would ‘evolve’ towards optimising welfare. If this is possible it could capture the best of both the public and private sectors.