AI Creativity: Myth or Reality?
The ability of AI to transfer writing styles and connect concepts seems unlimited, impressing those who encounter it. However, we need to take a closer look at how LLM algorithms work to debunk the myth of AI creativity.
The Student Who Memorizes Facts
LLMs are large multilayered neural networks that relate tokens to each other, mimicking the structure of our brain. Their design is based on the idea that “neurons that fire together, wire together.” This is done through a system of weights that represent the relationship between tokens. Tokens that represent similar concepts will have a smaller distance than tokens that represent concepts that are unrelated.
Tokens and their relationships
One important breakthrough in the early design of LLM’s was increasing the amount of layers. This allows complex concepts to be stored and related to each other.
Intelligence is the ability to look at a novel problem and come up with a solution. When you ask a question to an LLM, it translates the input you gave it and searches for concepts related to the set of tokens that you provided. It then transforms this back into a textual answer. Because of the large amount of information that was used to train these models, it looks like they are omni-sapient.
“What is heavier: A kilogram of iron or a kilogram of feathers?” Ask this question, and all LLM’s will answer it correctly. But if you modify the question to be something like: “What is heavier: A kilogram of iron or two kilograms of feathers?” Some of the LLMs will be fooled and still answer that they weigh the same.
The Farmer, the Wolf, and the Cabbage
I decided to test this hypothesis by asking 4 LLMs a modified version of the famous riddle. The original riddle goes like this: A farmer wants to cross a river and take with him a wolf, a goat, and a cabbage. There is a boat that can fit him plus either the wolf, the goat, or the cabbage. If the wolf and the goat are alone on one shore, the wolf will eat the goat. If the goat and the cabbage are alone on the shore, the goat will eat the cabbage. How can the farmer bring the wolf, the goat, and the cabbage across the river?
I changed the riddle so that the boat could fit him plus two items (instead of one). Initially, all LLM’s confused the riddle with the original version. I had to point out that the conditions were different, and got quite amusing answers.
The $1M ARC-AGI Prize
Francois Chollet argues that current LLM algorithms are boxed and not really capable of original thinking. He has created a challenge, the $1M ARC-AGI prize. Its premise is to have a set of novel riddles that are not yet solved in public, and thus, absent from training data and to have systems compete with each other trying to solve them.
Francois Chollet and the ARC-AGI prize
In conclusion, AI seems to be able to transfer writing styles and connect concepts, but it is not truly creative. It can only answer from its large pool of data and is not capable of original thinking.
