The illusion of knowledge

I was writing a simple app to present a surf forecast for my local spot some time ago. I began by introducing the swell size, wind speed and direction, and all relevant data from the MSW API that I believe is crucial for predicting whether the surf will be good or not. Then, after looking through all the basics, it came to the tides. Simple enough, I thought, there must be some equation that takes time, latitude and longitude and spits out the current tide for a given location. The gravitational forces of the moon cause tides, and this is predictable. I was stunned to discover that there is no simple way to calculate the tide. We created the tide tables based on observations in a particular spot, rather than relying on a simple calculation of the moon’s position. Something that I was sure was straightforward and I would talk about with confidence if asked, all of a sudden, became very blurry. It turned out the tide depends on plenty of different conditions, including the moon, sun, big objects on the Earth’s surface such as mountains, the weather and many more. It is almost impossible to predict the tide in particular locations without actually measuring it over a period.

One day, while surfing during high tide, I saw the moon on the horizon on the opposite side of the ocean. Again, before that, I was sure the moon pulls the water, so it would be low tide when it is on the side of the ocean and high tide when on the other side. Wrong again. We actually have four tides. A high tide occurs when the moon is in line with the Earth, at 0 and 180 degrees, and a low tide when it is at 90 and 270 degrees. That was somewhat mindblowing to me and made me think how often I take some knowledge for granted and, without even examining it, spread pseudoscience as if it were the ultimate truth.

Because we don’t know what we don’t know, we often fall into the trap of an illusion of knowledge. We frequently become so convinced of our own correctness that we might ignore evidence that proves us wrong. Even though having an impression that we are brilliant can be pleasant, it can cause some severe damage at the same time. I believe it is essential to act based on our best judgment, but always be prepared to face the possibility that we might be wrong.

Thinking about this topic reminds me of the Dunning-Kruger effect. In short, when we acquire basic knowledge or skills, we tend to overestimate our abilities. However, once we overcome the initial confidence peak, we start to doubt our abilities. Only after we begin to understand the actual complexity of the field do we step into the expert path, and the confidence starts to align with what we know or can do. I can see this pattern in every aspect of my life, from my professional programming career and surfing to expressing opinions on random topics. Having heard something or catching a few waves makes us think that we understand the subject very well and behave as if we were experts, judging other people and giving tons of advice to the beginners who haven’t managed to climb the first peak yet.

It is hard to behave correctly in facing this dilemma. On one hand, we don’t want to doubt ourselves and act as if we don’t know. On the other hand, we don’t want to spread pseudoscience and behave as if we are experts, even though we are far from that. I firmly believe that we should act with confidence based on what we know, but always seek to judge our abilities and knowledge objectively. We can do it by comparing our capabilities to those of people we know have mastered the field. There is nothing that helps me understand my place better than rolling with a black belt in jiujitsu who plays with me like a doll, staying in a surf lineup with guys who catch amazing waves, or getting an excellent, eye-opening review of my code. So act with confidence and always be ready to be wrong and corrected.