Organizations That Approve Risk

It has emerged that Anthropic bought millions of books to train its LLMs, cut off their bindings to scan them, and then discarded the originals. The operation, called “Project Panama,” reportedly cost tens of millions of dollars. Separately, copyright litigation also revealed that the company had obtained vast numbers of books from piracy websites.¹

Unsurprisingly, this has drawn enormous criticism. From using copyrighted works to train AI without the rights holders’ consent to the revulsion at physically destroying books on such a scale, the case has attracted considerable attention even among the general public.

With so many people already rushing to criticize it, I spent some time thinking about a perspective other than criticism.

What interested me most was that someone proposed a project this risky, an organization approved it, and it actually carried it out on an enormous scale. I am by no means trying to defend these actions. Still, I can’t help being impressed by this organization’s ability to execute.

At some point, someone inside the organization must have proposed acquiring and digitizing vast numbers of books. A company full of very smart people could surely have anticipated the risks of copyright disputes and public backlash. Given the project’s scale, it seems likely that they decided to proceed despite recognizing those risks. Remarkable.

What if someone had proposed a similar idea at a large Korean corporation?

The first questions would probably have been whether there were legal risks, whether other companies had set a precedent, whether the benefits justified the cost, who would take responsibility if something went wrong, and whether this might get the CEO hauled before a National Assembly audit hearing. These are all reasonable questions. They are also issues that absolutely need reviewing before undertaking a project like this.

But whether each question is reasonable and whether the organizational decisions that emerge from those questions are always reasonable are two separate matters. If a project can only be approved once every uncertainty has been resolved, highly uncertain projects are hard to start at all. You can only discover the effect by trying something new, yet you need to prove the effect in advance before you’re allowed to try.

When a project succeeds, the achievement belongs to the organization. But when it fails or provokes public controversy, the burden on the individual who proposed and pushed it forward can be enormous. If the personal downside of failure is disproportionately large compared with the personal reward for success, not making risky proposals is a rational choice.

This is less a question of individual employees’ creativity or willingness to take on challenges than of how the organization distributes risk and responsibility. However brilliant someone’s idea may be, if they have to bear the risks of realizing it alone, there is little reason for them to act on it.

I suddenly thought of The Tale of Heo Saeng, a satirical story I read in a school textbook. Yi Wan, who is pursuing a northern expedition against the Qing, asks Heo Saeng for advice. Heo proposes measures such as recruiting capable people, making the privileged surrender their wealth, and adopting Qing dress to send people into Qing territory. But Yi Wan replies that each is difficult, citing the king’s authority, the dignity of the scholar-officials, vested interests, and ritual propriety. Furious, Heo asks what on earth they can accomplish if this is impossible and that is impossible too. He’s right.

It’s not that I can’t understand Yi Wan’s position. Imagine bringing proposals like that to the king. You’d be lucky to keep your head. The target of the satire in this scene is clear: an organization that proclaims an enormous goal, a northern expedition, while refusing to accept the changes it would have to endure to realize it.

Many organizations want disruptive results. Whether they are prepared to shoulder the accompanying risks is another matter. If an organization demands results that could overturn the existing order, yet its decisions must follow established precedents, the possibility of failure must be minimized, and frontline employees must take responsibility when things go wrong, what does that organization actually want? Does it want innovation? Or just the results of innovation?

I worry about being misunderstood, so let me emphasize again: I am in no way defending Anthropic’s actions. Not every American company would approve a project like this, and there is no basis for declaring that a Korean company would necessarily have rejected it. Whether Anthropic’s judgment actually produced results commensurate with the risks must also be evaluated separately.

Even so, this case makes me think about an organization’s ability to execute. However good an individual’s idea may be, carrying out a project on an enormous scale requires someone to approve the budget, accept the possibility of failure, and ensure that when unexpected problems arise, the organization can share responsibility rather than leaving it to the individual.

We pay attention to how talented and enterprising the people in high-performing organizations are. But we talk relatively little about the organizational decision-making structures that made those achievements possible. Demanding disruptive results is easy. Accepting the uncertainty and responsibility involved in producing them is harder.