[Creative Writing Contest] The Legend of the Goldseeker

post by Aman Patel (aman-patel) · 2021-10-21T21:31:23.105Z · EA · GW · 5 comments

This is a link post for https://amanjpatel.notion.site/Short-Story-1-The-Legend-of-the-Goldseeker-bbae80899f7f419180c698cd4aad595f

In the land of Kunus, there is a story that everyone knows, and everyone must know.

This story supposedly tells the history of Kunus in days long past. Maybe that is true, maybe not—it doesn't matter. The people of Kunus treat it as such.

Parents use it to warn their misbehaving children, politicians use it as a rhetorical tool, and comedians avoid it at all costs. No matter the context, a mere mention of it is enough to make a rowdy room serious.

Each year, this legend is passed down to the youngest generation, to ensure that the people of Kunus never forget its gravity. All the town gathers in the central square for a performance put on by the Legend Committee, who dress up in costumes and act out the story. Being named a Committee member is considered one of the highest honors a Kunusian can receive, and nearly every prime-aged citizen covets the job.

Accordingly, Legend Day is considered the most important holiday of the year, albeit not a festive one. Businesses are closed, children have no school, and candles are lit in somber observance.

The mood is quiet, solemn, and slightly apprehensive, and for two hours, an entire people's attention is given to the actors on stage.

This is how the story goes:

A long time ago, a man and a woman gleefully announced the arrival of their son. He was a beautiful boy: round cheeks, big eyes, and a warm smile.

He was a kind boy, too. In school, he was known for assisting his classmates, even at personal cost to himself. His parents instilled in him the value of helping others, and he embraced the role of doing so.

As he continued exploring his place in the world, the range of his beneficence grew wider, and his parents continued rewarding this behavior with praise—so his desire to be beneficent grew, too. He volunteered much of his time to public service, learning more and more about the problems that his neighbors faced.

By the time he came of age, he was well aware that a lack of resources was severely constraining his town's potential. Many people were so poor that they couldn't afford food, and there wasn't enough funding to dedicate to scientific and agricultural endeavors that could liberate Kunus from this condition. He saw the individual horrors of hunger every day, and could only begin to comprehend the vast scale of this suffering, multiplied over the tens of thousands of Kunusian residents who lived in poverty. 

It became his mission to make Kunus wealthy, so that its people could develop the knowledge and technology they needed to flourish.

And what better way to become wealthy than to find gold? The elders spoke of buried mines in the mountains from long before, that nobody had yet rediscovered—so he set out in search of this treasure. He was very unsure that this plan would work—in fact, it seemed to have a very low probability of success, but the rewards would be so high that it was worth doing anyway.

He obtained maps from the historians and spent a month roaming the wilderness, digging for three days straight at each marked site. At times, he became very close to giving up; he could try to find wealth to share in another way, perhaps by becoming a businessman or politician. But he persisted.

On his thirty-first day, he finally struck gold. Seven cubits below the ground, there was a pile of shimmering yellow metal, already smelted and processed into bars by the inhabitants of past times. He quickly got to work loading this gold into his cart, and promptly dragged it back to town.

This gold was able to provide seven thousand Kunusians with food and shelter a year.

But it wasn't enough. Seven thousand was barely ten percent of Kunus' population, and he wanted to help more.

He recruited a crew to help him seek and distribute more wealth. Month after month, he and his crew would scour the Kunusian backlands for hints of metal, and every so often they would come back to town with carts full of gold. This gold provided more sustenance for more people, until eventually there was no more poverty in Kunus. The local population was elated, and so was he.

They almost had enough money to begin a program of farm experimentation, so that the drought-prone Kunus could grow its own food in abundance. But before they could begin this program, the luck of the gold faltered. The goldseeking crew returned less and less frequently, and their hauls were smaller and smaller, until eventually they could find no more gold. Their leader was determined to ensure Kunus' welfare, and he scrambled to find another way to do so.

Eventually, he arrived at an idea. The largest known deposit of gold on the planet was located in a neighboring kingdom. That kingdom was a thriving, prosperous society, and had developed sophisticated systems of philosophy and government. They considered the gold deposit a sacred part of their cultural heritage, and had not extracted it.

He was going to do so. If he could retrieve that gold, Kunus's security and well-being would be guaranteed for a thousand years to come. He resolved to find a way to persuade the other kingdom to let him mine that deposit.

He first tried approaching their king with a friendly message: let us mine your gold, and we will share it with you. The king refused, explaining that the cultural value of the deposit was worth far more than its market exchange rate.

He then tried pleading on humanitarian grounds: Kunus needed that gold. Unlike the king's country, Kunus was poor and needed capital to invest in developing better farms. Again, the king refused.

He even considered threatening war, but he knew that the king's army was the most powerful in the world.

But then he remembered that the inhabitants of that kingdom deeply revered the callingbirds that inhabited the forests of Kunus. While Kunusians saw these birds as pests to be exterminated, the neighboring population viewed them as sentient creatures, equal in moral standing to any human.

He threatened to kill all the callingbirds in Kunus—a public service to Kunusians, but a genocide to their neighbors. Maybe the king would finally budge.

Instead, the king scoffed at him. Not for a moment did the king believe that he would actually kill the birds; not only did it sound ridiculous, but Kunus didn't even have the capacity to hunt ten birds, let alone a whole forest. The king sent him away.

Returning home, the goldseeker hatched a plan to make the king believe him. Instead of merely killing the callingbirds, he would make them suffer—and instead of hunting them, he would breed them. He designed a cage large enough to trap one million callingbirds, and built it with logs. He captured as many birds as he could find, and encouraged them to lay eggs. And lay eggs they did; soon he had one hundred caged callingbirds. Soon after that, one thousand. Eventually, he had one million callingbirds, all stuffed into a tight cage. It was all the same to him and his fellow Kunusians—they couldn't care less what happened to the callingbirds.

Traditionally, the Kunusians used an herbal extract to keep callingbirds off their crops; nobody knew what it made the birds feel, but it was obviously bad enough to repel the birds from ever nearing a Kunusian farm. According to the neighboring kingdom, it made the birds burn.

The goldseeker distilled a vat of this extract, and designed a mechanism to shower it onto the cage. Once the flow began, it could not stop.

When his setup was complete, he journeyed to the neighboring king's court once more.

He presented his cage, and told the king that he would start the unstoppable flow of burning extract if the gold deposit was not released to Kunus. He didn't think he would have to follow through with his threat; he figured that the king would be afraid enough to instantly comply with his demands.

But the king still refused to give in to the goldseeker's demands. The king thought that this whole setup was a ruse—the Kunusians would never be so immoral as to burn one million callingbirds.

But the goldseeker wanted to make sure that the king believed him, not just this time, but in all future encounters, too. He wanted a reputation that his words were credible; if he threatened something, he would do it.

So he started the shower of burning extract onto the callingbird cage. And it could not stop. The kingdom was filled with the sounds of one million callingbirds crying in excruciating pain, one million beings experiencing the worst form of suffering imaginable.

Ten minutes later, the noise stopped. The callingbirds had pecked out their own throats, killed themselves to stop the pain. Death was the least painful option.

As before, the goldseeker and his fellow Kunusians did not care. For them, the callingbirds' suffering was easily negligible, just like insects.

But the king was horrified, and so were his people. He immediately commanded his army to punish Kunus.

The army razed every building in the land, and lit all crops on fire. The chaos ensured for two weeks, until Kunus was nothing but haze and rubble. While the army refrained from slaughtering Kunusians, they made sure that Kunus would never see prosperity again.


Seven years later, a bright young naturalist deciphered the callingbird screeches. He found that they communicated about far more than just food and mating. They had calls for joy, love, sadness, frustration, guilt, and unique sounds for their family. And most of all, humans could learn to speak this rudimentary language. The naturalist's book became very popular, and people across the planet—including in Kunus—began to converse with the callingbirds, to understand their trials and tribulations, and to love them as friends.

And at that final note, the performers exit the stage, and everyone returns to their normal activity with a renewed sense of humility, contemplation, and collective responsibility.

5 comments

Comments sorted by top scores.

comment by WSCFriedman · 2021-10-25T22:07:16.409Z · EA(p) · GW(p)

I did not understand what the story was trying to say, very well. It just seemed to me to be 'a series of bad things happened because of failures of effective communication and understanding?' I can read it as a criticism of overconfidence, but I feel as if there have already been a lot of criticisms of overconfidence, and at this point I'm kind of worried we need more criticisms of underconfidence? I did not end up with very strong opinions about the story in any particular way, and I suspect it was a failure of my understanding at least as much as a failure of the story.

Replies from: Jackson Wagner, aman-patel
comment by Jackson Wagner · 2021-10-28T06:14:14.317Z · EA(p) · GW(p)

As tagged, this story strikes me as a fable intended to explain one of the mechanisms behind so-called "S-risks", hellish scenarios that might be a fate worse than the "death" represented by X-risks.  It's thought that one of the ways a far-future civilization (or AI or whatever) might end up creating vast amounts of suffering, could be as the result of a cycle of threats and bargaining with rival civilizations: https://centerforreducingsuffering.org/research/intro/

Of course it's a little confusing to have the twist with the sentient birds -- I think rather than a literal "farmed animal welfare" thing, this is intended to showcase a situation where two different civilizations have very different values (because one civilization values the sentient birds extremely highly and the other, thinking they are just ordinary birds, values them not at all).  Meanwhile, the other civilization wants to keep the gold in the ground, vs the Kunus find it valuable only after it's mined.  (In a similar way, someone who is pro-choice could threaten to kill a fetus, which to them is only a ball of cells, and this might be a very effective threat against a pro-life person who thinks every fetus has a fully human soul.)

I don't really understand why the story is a frame story, or why the main purpose of the ritual is for all the Kunus to feel "collective guilt"... EA is usually trying to steer away from giving the impression that we want everyone to feel guilty all the time.

Totally unrelated point, but I thought the economics of this story were a little wacky... having some gold rocks in your pocket doesn't do anything to help you have a better quality of life or cause there to be more food in the kingdom... it's just a form of currency.  Unless the kingdom of the Kunus is suffering from deflation or tight monentary policy or whatever, I think the Goldseeker is barking up the wrong tree and isn't actually doing anything to improve the overall prosperity of the kingdom.  (Of course if you give the gold evenly to everyone or disproportionately to the poor, that would have a helpful redistributive effect.  But you're not growing the economic pie at all by digging up yellow rocks.)

Overall, I don't think this story works that well because the reframing of S-risk from far-future galactic-civilizations to fantasy-parable doesn't really make the idea clearer or more intuitive.  Although I do like the twist where the Goldseeker was a good person trying to improve the world and hatched a clever, harmless-seeming plan to do that, only to unwittingly end up torturing countless sentient beings.  This is a good part of the parable -- if S-risks ever occur, the civilizations that commit those galactic war crimes will probably be convinced of their righteousness, and indeed probably won't even recognize that they are committing a wrong.

Replies from: aman-patel, aman-patel
comment by Aman Patel (aman-patel) · 2021-11-09T16:13:03.685Z · EA(p) · GW(p)

Changed "guilt" to "responsibility," but I'm not sure if that's much better.

comment by Aman Patel (aman-patel) · 2021-11-09T15:59:23.416Z · EA(p) · GW(p)

"As tagged, this story strikes me as a fable intended to explain one of the mechanisms behind so-called "S-risks", hellish scenarios that might be a fate worse than the "death" represented by X-risks."

That's what I was going for, although I'm aware that I didn't make this as clear as I should have.

"Of course it's a little confusing to have the twist with the sentient birds -- I think rather than a literal "farmed animal welfare" thing, this is intended to showcase a situation where two different civilizations have very different values."

Same thing here. This is what I was trying to get at, but couldn't think of many other scenarios involving suffering agents where one group of people cares and another doesn't.

"I don't really understand why the story is a frame story, or why the main purpose of the ritual is for all the Kunus to feel "collective guilt"... EA is usually trying to steer away from giving the impression that we want everyone to feel guilty all the time."

This is really helpful feedback--I didn't realize that "collective guilt" came across as the point of the story, and I definitely agree that making people feel guilty is counterproductive. I can't remember why I threw in that phrase (probably because I couldn't think of anything else), but I'll change it now. 

Totally unrelated point, but I thought the economics of this story were a little wacky.

Yup, definitely more than a "little" wacky :) Maybe using another resource like food or water or land would be better--but then it would have been harder to make the point that each country thought were doing the right thing.

This is a good part of the parable -- if S-risks ever occur, the civilizations that commit those galactic war crimes will probably be convinced of their righteousness, and indeed probably won't even recognize that they are committing a wrong.

This is the central point that I wanted to get across. Whether we're considering a civilization or an advanced AI, s-risks need not result from intentional malevolence. I'm glad it didn't get too distorted, but it seems like there are better ways to build a story around this point.

Another side-note: a lot of the ideas behind this story are discussed in the Center on Long-Term Risk's research agenda. I don't know whether they would agree with my presentation or conceptualization of those ideas.

Thank you so much for the feedback!

comment by Aman Patel (aman-patel) · 2021-11-09T16:11:45.026Z · EA(p) · GW(p)

Thanks for the feedback! I think this is probably a failure of the story more than a failure of your understanding--after all, a story that's hard to understand isn't fulfilling its purpose very well. Jackson Wagner's comment below is a good summary of the main points I was intending to get across.

Next time I write, I'll try to be more clear about the points I'm trying to convey.