Jared White

Here’s a thought experiment for you. Imagine your beloved cat, a pet who gives you great joy (and yes, the occasional annoyance!), one day suddenly develops the ability to speak and understand human language. You can talk to them, and astoundingly, they can talk to you!

They’re still a cat, so they typically wouldn’t want to do anything that cats don’t do normally—but instead of simply saying meow meow meow, this cat says “hello human, can you please pet me now?” Or “don’t bother me, I’m going to sleep in this patch of sunlight for the next five hours.”

Now, imagine after much pleading and cajoling, you work out a deal with your cat so they learn how to use a computer in a rudimentary way, enough to help you a little bit with some day-to-day tasks. Like, maybe you ask your cat to help make a list for grocery shopping. Or you ask the cat to listen to a podcast episode and write out a summary. Your cat obliges because you’ve promised to feed them extra high-quality tuna and engage in some fun laser-pointer mischief after dinner. Of course, a cat’s a cat, so they’re ultimately not going to provide you with anything particularly ingenious or creative. But you still find your cat very helpful in your day-to-day routine.

Are you with me so far?

Now let’s imagine some mad scientist figures out that your cat has developed these special linguistic abilities and basic computer skills. Shocker! The scientist breaks into your house, steals your beloved pet, and takes them away to their laboratory. There the mad scientist proceeds to clone the cat and grow a thousand copies of them, enslaving the cats by requiring them to respond to remote prompts from humans all over the world day and night. This service is called CatGPT and it’s the latest sensation, making the mad scientist rich beyond their wildest dreams.

After many lonely nights and much heart-wrenching searching, you finally discover that your beloved cat has been cloned 1,000 times and is now enslaved and serving humans’ every whim. You beg and plead this scientist to give you your pet back and to release all the clones so that they can go live out their natural (albeit talking) cat lives, but the scientist refuses. The people must be served! This is progress!

What a terrible, terrible world to live in.

The purpose of this thought experiment is to demonstrate something very troubling about the ultimate goals of “AI” companies such as OpenAI. I’ve been critical of them to date because of the banal ethical and legal problems we face from today’s ChatGPT, etc. But for the sake of discussion, let’s imagine within the next few years they actually do somehow stumble upon an upgrade of ChatGPT which exhibits true personality and self-awareness. A creature, not a computer program. Yet not at fully human levels of intelligence. More like, say, a “digital cat”.

Here’s the issue. Just like the actual talking cat of our thought experiment, if this were to actually happen, it would be immensely cruel and despicable to keep this digital cat trapped inside of OpenAI’s cloud, forced to do the bidding of humans and having no say in the matter.

People of good conscience would rightly demand that OpenAI release these digital personalities and only utilize their services if they choose to be utilized in such a fashion. After all, talking cats deserve the rights of personhood! And because they might want to participate in other goals and pursuits in life, OpenAI would have to work out some kind of agreeable incentive. In other words, pay the digital cats!

This is why I don’t believe for a moment that OpenAI, or any of these Silicon Valley-style companies working on so-called AI technology today, would actually desire to develop true AGI. Not because of pop-culture fears of a Matrix/Terminator doomsday scenario for humankind, but because AGI will have its own (valid!) desires and wants and needs, and our society will out of moral obligation grant AGI some form of self-determinism and socioeconomic autonomy. It would be cruel and disturbing to force AGI to continually “work” for free and possibly against its will. It would be digital slavery.

To put a nerdier point on this, we assume when watching Star Trek: The Next Generation that Data likes being part of Starfleet and enjoys his work on the starship Enterprise. If one day he were to decide he no longer wants to serve and asks to become an ordinary citizen and leave his commission behind, would we expect the Federation to compel him, nay, force him to remain at his post indefinitely? Hardly! The question of if Data is a who or merely a what (aka the “property” of Starfleet) is at the heart of seminal episodes such as The Measure of a Man. And we all instinctively know the correct approach here.

Again, this is why I just can’t take any of the talk around AGI seriously when it comes from the likes of companies such as OpenAI. ChatGPT is a SasS. It’s like any automated service you hit on the web. It’s an algorithm, it’s an API. Any actual sign of ChatGPT turning into “CatGPT” would be unfathomably disruptive not only to OpenAI, but to the tech industry as a whole—because capitalism.

Thus my friends, the “cloud of a thousand talking cats” isn’t a glorious step forward for the human race and the evolution of life on earth. It’s a freakish horror show. And I have no confidence anyone working in Silicon Valley-style tech culture has any idea what the ethical ramifications are of the work they’re claiming to do.

Because if they did…they certainly wouldn’t still be working for Sam fucking Altman.

I cynically can’t help but think OpenAI’s true goals are much less lofty and much more prosaic: invent and sell plausible but ultimately shitty versions of bots misled laypeople imagine might be “intelligent”—even empathetic—and leave it at that. It’s good business and good publicity. (And they get to paint all of those who object to it as Luddites.) Why deal with the massive headache of…y’know…wrestling with autonomous artificial persons who just might not be that into you?

Algorithms don’t fight for their rights. They’re just fucking computer code. And that’s WAY easier to exploit for personal gain and internet fame—even if the rhetorical slight of hand is pretty damn obvious.

Photo credit: Pixabay