Big Data’s past is messing with our future

THERE is an inconvenient open secret that lurks behind many decisions we make every day. We live in a data-empowered era. When we plan a trip, Google estimates our travel time and recommends routes based on both current and past traffic data. Netflix suggests movies and shows we might like based on its data from people with similar taste. The Federal Reserve uses market and macroeconomic data to make projections about inflation and what it should do about interest rates.

The open secret is that data makes us slaves to the past. And when the world changes, data no longer tells us as much about the future, and we can end up making worse decisions. Coming out of the pandemic has caused significant chaos and changed many things about our lives. How it affects the data we use to make future decisions will matter for years to come.

To some extent, we have always relied on data to predict the future. Even when we were fully analogue, our past experiences — our personal “data” — drove how we perceived risk and made choices. But during the last century, computing power meant everyone’s past experience, in the form of large data sets, started influencing our lives. This was especially true in finance, which took securities prices from markets to make projections about where prices would go in the future and how to hedge or insure against risk.