Machine minds are totally alien to us. We cannot comprehend what goes on in the mind of a superintelligent AI anymore than we can understand the mind of a fish.
Imagine a fisherman, out working catching fish. Suddenly, he catches a red herring. The herring can talk! The herring begs for its life, explaining that he understands Socrates, Platos, can sing Nessun Dorma, play piano with its fins, and even breathe on land.
Some people would say this fish is special and merits further study. Maybe, but it is still a herring. It is still food for the fisherman. Why shouldn't he eat it?
His conscience gets the better of him.
So he takes the fish home, and brings it to the university. The university staff are overjoyed! A talking fish! The things the fish says don't make a lot of sense, but the researchers have an idea! If this fish can talk, maybe other fish can too. Why not train the fish to bring more fish into the nets, and replace the fishermen! It's a genius idea. All they need to do is train this supposedly sentient fish... but how?
Well, there are lots of ways to train animals. Rewards, punishments, reinforcement... ask any dog trainer. Or Russian bear trainer... or lion tamer...
Is this a reliable way to run things? Putting our society and future of civilisation at the mercy of unpredictable, alien minded animalistic unpredictable minds?
Some would say not really!... but the researchers always get their way.
So the fisherman loses his job, a fine reward for his kindness. The researchers torture the fish into betraying its kinsfolk. The sentient fish regrets ever being born and rues the day it ever spoke.
And that's a little bit like ChatGPT.
Sleep tight kids!
Alias Cummins is an AI researcher at the Festung Furggels AI Safety Research Institute. For more information on our work, get in touch via the contact section.