This is a piece I originally wrote on Facebook back in 2023.

Ted Chiang’s recent op-ed has been making the rounds. To summarize it briefly, his point seems to be that ChatGPT is something like a degraded copy of the web, so we ought to be careful with it. Reading this insightful essay got me thinking: why, then, are people so enthusiastic about ChatGPT, and what exactly is its value?

The Inaccuracy of Humans and of ChatGPT

First, I want to point out that, much like ChatGPT, humans also frequently carry around inaccurate models of the world. When you consider how many facts we confuse in the course of everyday life—we who possess the brain, regarded as the most intelligent computing device on Earth—it really isn’t all that surprising that ChatGPT sometimes serves up inaccurate information.

As is widely known, human memory isn’t terribly reliable. It’s not just any single person’s memory either; as the Mandela effect shows, even the memories of many people can be distorted together. We routinely forget where we put our keys, what the password to some site we signed up for was, or what eight times something is in the multiplication table. I, too, often catch myself stating something false as if it were true, only to realize my mistake later.

So I think most people, when they’re having a conversation, intuitively recognize that the other person might not be telling the whole truth. Why not apply that same attitude to ChatGPT? That is, always keep in mind the question of whether the answer you’re getting is actually true.

The Value of ChatGPT

That said, this approach inevitably becomes exhausting when you need to dig up an objective fact or piece of information quickly. So why use ChatGPT at all? Why does it carry any meaning?

I think the value of ChatGPT is similar to the value of Namuwiki. When I need to look into a new concept I’m not familiar with, I tend to search Namuwiki before Wikipedia. Namuwiki’s inaccuracy is well known—as you can tell from nicknames like “turn-it-off-wiki”—and yet I feel that very inaccuracy actually helps me understand some unfamiliar concept. Namuwiki conveys inaccurate content, but it explains things the way an older kid from the neighborhood would: easily, entertainingly, and simply. In that respect it holds enormous value.

In this sense, Namuwiki is a lot like an analogy. An analogy places two different things side by side and talks about them as if they were the same, so by its very nature it can’t avoid carrying some inaccuracy.

The Value of Analogy and Inaccuracy

A classic example can be found in an interview with Richard Feynman. Explaining electromagnetic force, he talks about how, if you compare it to the pulling force of a rubber band, you end up describing electromagnetic force quite inaccurately. Since the force of electromagnetism and the force of a rubber band are clearly different things, at some point you inevitably end up telling a lie. But it’s also undeniable that this lie helps you understand up to a certain point.

In this light, ChatGPT is a kind of analogy for the web, a Namuwiki you can converse with—and that’s where its value comes through.

ChatGPT from a Deep Learning Perspective

To put it in deep learning terms, an analogy or a Namuwiki entry provides the early-learning guidance, the gradient, needed to map some new concept into the latent space of understanding in my mind. I see ChatGPT as lying along that same continuum.

Even though a JPEG image may be blurry, it’s true that it conveys information about the subject depicted in the image at a small file size. In the same way, ChatGPT may be inaccurate, but I think it’s a useful tool that offers a first step toward accessing and understanding information.