Grey. Whenever my daughter asks me “What’s your favorite color?” my answer is always the same. Grey. My daughter asks me this often, maybe once a week, expecting my favorite color to change, maybe based on my mood, or the weather. But, no, my answer is always the same. Grey.

Why grey? First, grey is simply a cool color. Contrast is defined by shades of grey. Think of drawing with a pencil. All of the texture is conveyed by shades of grey. But, really, grey is my favorite color because of what it represents. Grey reminds me that the world is not black and white. It is nuance. It is not absolute. It is shades of grey. Our knowledge is not black and white. Science is not black and white. It adapts with new knowledge. It evolves. Science is, in some real sense, grey.

That’s, in some sense, the message of Nate Silver’s The Signal and the Noise. Nate Silver is most famous for running the FiveThirtyEight political forecasting website. He initially gained fame by contributing to the wave of analytics applied to baseball and his tools for forecasting baseball games and player performance. In The Signal and the Noise, he describes the inherent complexities involved in forecasting in a range of fields, from baseball and politics to the stock market, weather and hurricanes, earthquakes and, ultimately, national security issues such as terrorist attacks. He provides perspective into when we have proven successful in our ability of prediction, when we haven’t, and why some of these problems are so complex.

As a scientist, my job, in a real way, is about forecasting and prediction. I personally am not so involved in real forecasting, per se, but about understanding the phenomena that might be important to account for in forecasting. Others actually try to use that information to make real predictions. That said, the ultimate goal is to take knowledge we have today and make predictions about how materials would respond if the conditions were changed, either pushed further in time, or in slightly different environments, or for new applications. Thus, whether the results I uncover are useful is to be judged by if they help us understand and make predictions of materials that are better than they were yesterday. So, Silver’s treatment, while not delving into the my field per se, provides a nice overview of the idea of prediction more generally.

Silver takes the view that the world is inherently Bayesian. I’m certainly no expert on Bayes statistics, but, as far as I understand, the basic idea is that information doesn’t exist in isolation. Rather, there are certain biases or preconceptions or prior knowledge that we have and our experiments or experience modify those priors to give us new knowledge or allow us to make a better prediction. As one negative example of this, our intelligence agencies didn’t prepare for 9/11 not so much because they had no intelligence but because they had an implicit assumption that such an event could not happen. Their implicit probability that a group would hijack multiple planes and crash them into a building with no demands was effectively zero. So, all the intelligence of the world wouldn’t indicate an increased likelihood of such an event as it was simply beyond the realm of possibility for them.

Similarly, using what we do know to estimate probabilities and make predictions enhances our ability to make good predictions. As Wikipedia describes it, if we want to estimate the probability that someone might have cancer, it is important to account for the fact that cancer risk depends on age and that the probability is not uniform for all people. Knowing someone’s age lets us better predict if they might have cancer. Or, as Silver discusses, it allows us to better interpret test results.

Thus, the world isn’t black and white. It is a range of grey and the shade of grey depends on our experience and prior knowledge. Silver believes in an objective truth, but realizes that we will never fully be able to describe that truth. All of our models of reality are approximate, to varying degrees. None are fully faithful to reality itself. Realizing this, helps us make the best possible use of what we know and make the best possible prediction.

Silver delves into many examples and discussions of the state of prediction in several fields. Weather and hurricane prediction are success stories as we have improved our abilities significantly over the last several decades. In contrast, earthquake prediction has not advances at all. We are no better at predicting when an earthquake will happen than we were 20-30 years ago. Some of this is related to our ability to make relevant measurements — it is much easier to measure surface and atmospheric phenomena than the state of stress deep in the earth. We can’t directly measure the conditions that might allow us to make a good prediction.

Along the way, Silver makes a number of interesting points, a few of which I thought were worth noting:

• “Human beings have an extraordinary capacity to ignore risks that threaten their livelihood, as though this will make them go away.”
• “We forget — or we willfully ignore — that our models are simplifications of the world. We figure that if we make a mistake, it will be at the margin.”
• “The key is in remembering that a model is a tool to help us understand the complexities of the universe, and never a substitute for the universe itself.”
• A lot of pundits, such as the political talking heads on TV, are what might be termed “hedgehogs”: their predictions are not better than random and their views do not change with new evidence but rather remain entrenched, regardless of how bad their predictions have been in the past.
• “If you have reason to think that yesterday’s forecast was wrong [about anything you might have forecast], there is no glory in sticking to it.”
• A particularly difficult aspect of economic forecasting is that the prediction itself can influence the system. If a forecaster predicts that Facebook stock will tumble, that prediction itself may lead to a frenzy in trading Facebook stocks, impacting the price of those stocks and the market as a whole. So, while in weather and earthquake prediction, the forecast is outside of the system, that is not true of economics and other similar fields such as the prediction of epidemics. “The most effective flu prediction might be one that fails to come to fruition because it motivates people toward more healthful choices.”
• In the era of Big Data, predictions might become worse rather than better, as much of the data might be noise that confounds our models rather than signal that leads to enhanced predictive capability.
• “The need for prediction arises not necessarily because the world itself is uncertain, but because understanding it fully is beyond our capacity.”
• “What I do know is that there isa  fundamental difference between science and politics. In fact, I’ve come to view them more and more as opposites… In science, one rarely sees all the data point toward one precise conclusion.”
• “The dysfunctional state of the American political system is the best reason to be pessimistic about our country’s future. Our scientific and technological prowess is the best reason to be optimistic.”
• “Whatever range of abilities we have acquired, there will always be tasks sitting right at the edge of them. If we judge ourselves by what is hardest for us, we make take for granted those things that we do easily and routinely.”

In the lead up to the Iraq War, Donald Rumsfeld gave a speech that was widely criticized. Not for the argument that Iraq had weapons of mass destruction (though that was certainly a big criticism) but for this statement: “[T]here are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — there are things we do not know we don’t know.” He was criticized for this phrase “unknown unknowns.” I certainly don’t agree with his politics, but I never understood the backlash he got for this. The view of the world he espoused in this simple statement is one we should all have. It is precisely the “unknown unknowns” that led to our intelligence agencies being unable to fathom a 9/11-like attack. There are always things we haven’t even thought of that will impact our day, our job, our health, our life. We can’t know what these are, by definition. But, we have to at least acknowledge that such things are out there and that they can severely disrupt what we predicted might have happened. “By knowing more about what we don’t know, we may get a few more predictions right.”

Overall, Silver advocates for a view of the world that is probabilistic, one in which we don’t make black and white assertions about the world, but one in which we acknowledge what we know, how uncertain that knowledge is, and what we don’t know, and use it to make estimates/predictions/forecasts about the world around us. Only by admitting that the world is grey — or better said, at least our understanding of the world is grey — can we hope to make better sense of it. Silver’s book is an initial step in arguing for this view of the world. The next step is trying to take that view to heart and use it in our every day lives. As Silver concludes: “Distinguishing the signal from the noise requires both scientific knowledge and self-knowledge: the serenity to accept the things we cannot predict, the courage to predict the things we can, and the wisdom to know the difference.”