Limitations of Data

A few days ago I set out to write something about the information overload, and ended up with just that: The very basic entry branched from Information Exposure to Data Crunching, and there is still more to cover - still within the "very basic" review. This present entry is concerned with the sad fact that even with the near-infinite amount of data accessible to us, the data is still limited, and often we don't achieve the understanding we seek.

Beyond relevance and reliability of data, critical thinking, as a skill and a habit, can help us identify when we are given incomplete information, rendered useless by its incompleteness. For example, a common news item: "The water reserves went up 2 centimeters over the last 24 hours." Without knowing the exact level right now, and the normal level at this time of year, and the level where we are in danger of dying of thirst - this information is just pseudo information. It leaves us at the same level of ignorance it found us. If anything, it clogs our thinking with a hairball of meaningless trivialities.

Another use of critical thinking in the context of looking for information, is knowing the difference between what one knows and what one doesn't know. Some explanation: Suppose I think - based on hair color - that some celebrity is dumb. I must be aware that this notion is a total guess. Suppose that I then hear a rumor confirming the dumbness of that celebrity. I may feel better about my guess, but should keep in mind the fact that I still don't have reliable information. Suppose further that I read a scientific survey of the IQ of different celebrities, confirming that the particular one is dumb. Maybe I can start marking that notion as reliable, but I need to look critically at the survey: Who performed it? Who financed it? What methods were used to measure IQ? And - not least - do I think that IQ has much to do with dumbness? The interesting bottom line here is that we need to keep in mind, together with each notion, how sure we are of it, and preferably exactly what sources we have supporting or refuting that notion. Academics already require this kind of discipline from themselves, taking care to make accurate and traceable quotes. Laypeople increasingly need it, too. This reminds me a time I was in London, before there were GPSs, so the cab drivers had to know the street names. I was looking for a certain street, and realized that the cab driver remembers full street names - Brook Road, Brook Street, Brook Lane, etc. are different places. Those of us living in smaller towns, think it's enough to remember where "Brook" is. But humans are adaptable, and we can remember the full name. Similarly, it is becoming increasingly useful to remember facts together with their associated source.

Knowing what we know is not limited to "knowledge". It is useful for skills, too. "Can I fix the car in my own garage?" Not anymore. Cars have become too complicated, and I didn't keep up. When humanity's knowledge and skills did not accumulate so quickly, it was expected from a well educated adult to know all that's important. This is no longer the case. It is the norm that people don't know everything, and even if one does know a lot about a specific subject, in a few years they will not know enough anymore, because much more knowledge has accumulated on the subject. Of course one could actively keep up their knowledge on a certain subject that might interest them, but then they will fall behind on other subjects. Dirty Harry said "A good man always knows his limitations." Bottom line, we should stop denying what we don't know. This will give us the opportunity to decide what we want to know, and - again - to keep learning.

So, we have too much data, and we are aware that we don't have enough. How do we make decisions based on imperfect knowledge? How do we avoid Analysis Paralysis - a state of continually looking for information. We must realize we live in a state of imperfect certainty at best, and almost-perfect uncertainty at worst. We, the graduates of the education system, must have the ability to decide at some point that it is better to take a decision now than to look for more reliable information. It's a risk that has to be taken - we never have indisputably sufficient information. Some tools for managing that risk can be borrowed from the world of statistics. Layman's statistics: Maybe the kind of statistics insurance people know - not high mathematics, just risk-management.

Whatever the technical or mathematical skills required to make risk-managing decisions in an uncertain world, these are not the main obstacle for most people. The main difficulty is emotional and intuitive. To function in a world swamped by information that is nevertheless imperfect, we need to get used to the uncertainty. To be comfortable with this uncertainty, and to be comfortable with the mistakes that come with it. Probably develop the intuition for knowing (or at least deciding) when we have "enough" information. This reality of insufficient information even forces us to use some intuition in place of missing information. A difficult notion for those of us who like to think of ourselves as rationalists, but maybe its time has come. Trying to rely solely on hard information in a swamp of imperfect information, is not a rational approach.