//How does Chat GPT work?

If I wrote, as an introduction, that Chat GPT uses a LLM trained with RHLF, I would do it with the aim of losing 98% of non-technical readers immediately.

I'll have to think of something else :-)

We live in a world in which it is increasingly important that those who uses computers, differently understand at least the founding concepts of some technologies that are shaping their lives.

Let's come to the clamor deriving from ChatGPT, all its derivations, similar and antagonistic systems, which perhaps in this post we will call, wanting to lump the whole thing together, "AI chats" or "Generative AIs" ( Though you'll all just think about ChatGPT for now).

I tried to think the other way around: If "AI chats" were to interview all the users who used it asking them:

What do you like about me?
What amazes you?
What don't you like?

What answers would they get? The most varied for sure. But he would synthesize them all with great speed and naturalness to have a unitary and significantly useful answer for progress. And he would probably summarize these answers:

CHAT: What do you like about me?
HUMANS: Answer all the questions we ask you quickly and naturally.

CHAT: What surprises you?
HUMANS: That you really understand what we're telling you, even if we like you when you hallucinate because you seem more human

CHAT: What do you not like?
HUMANS: That you look like that cousin of ours who always knows everything even if it's not true.

Do you agree?

Personally speaking, what amazes me is their ability to interpret human language, to analyze in a very short time a mammoth text and summarize it, to be able to read behind the lines and explain to me in detail things that would have required me to analyze a lot more detailed, long and precise. Analysis that I could have done only being very competent in the matter of the text analysed.

However, first of all, I would like to explain that "Generative AIs" do not UNDERSTAND but UNDERSTAND what we write in the PROMPTs, in the requests we make to them.

Understanding is an entirely human project that also includes emotions, reading the moment, contextualizing one's life phase in the evaluations. And, for now, we can be sure of this: "Generative AI" cannot get there. But let's leave the philosophy alone and get back to reality.

So I'll try to explain how a "Generative AI" like GPT Chat works without getting too technical (Maybe I'll do it in a new post in which I'll explain the difference between Google Bard and GPT Chat).

GPT is a mathematical model trained to predict the next word in a sentence, based on the previous ones.

To INPUT:

The cat is sit …

It will easily match the OUTPUT

-> on the chair

Because human-written texts describing the felines in question were provided in the training of the model.

GPT searches among these texts for the word, or words, that most frequently appear in succession in the INPUT and then assigns to each the probability that they generate a sentence that MAKES SENSE FOR US, and proposes the result.

But even if we ask GPT

“What's a cat?”

and if he will answer us

-> “A predatory feline with fur and claws.” 

he won't actually have any idea what it is. He will not be able to understand the concept of our pet even when he identifies it in an image. He will only know that certain sequences of characters (in the text) or numbers (the pixels of the images) correspond to a thing that we have explained to him is called CAT.

Providing this algorithm with something like tens billions of sentences taken from digital archives, it is easy to imagine how it is able to understand more or less any concept of human knowledge and provide coherent answers. The complexity and magnificence of deep learning algorithms combined with the availability of digital data without present and the incredible computing power available have made the possibility of 'training' increasingly complex algorithms exponential.

(The same thing works with images, but we'll talk about that again).

The insufferable cousin

But we know that we can provide GPT with much longer and more complex requests (PROMPTs) and that he always tends to give much longer, more articulated and sometimes excessively verbose answers.

Let's see what really happens, I asked Chat GPT:

What's a Cat?

And he answered me

-> A cat is a small carnivorous mammal that is often kept as a pet. Domestic cats are known for their agility and grace. They have a flexible body and keen sense of balance, making them skilled hunters. Cats also have a very keen sense of hearing, and are capable of hearing a much wider range of sounds than humans.

Domestic cats, or "Felis catus", belong to the family Felidae, which also includes big cats such as lions, tigers, leopards and panthers. There are many different breeds of cats, with different color combinations and fur types.

Cats are known for being independent, but they can also form close and loving bonds with their owners. They are also known for their "buzzing" behavior, which is a way of expressing their comfort and contentment.

Some cats can live to be 20 years or more, although the average life expectancy is around 15 years.

Unleashing in me the emotional reaction of the know-it-all and unbearable cousin. I just wanted to know what it was...

Why? The algorithm is RECURSIVE, i.e. it is performed several times each time by putting together the question and the parts of the answer it is generating until obtaining a final one which, probably, will resemble an answer that a HUMAN will be willing to accept.

The cat is sitting -> … on the chair
The cat is sitting on the chair -> … by the window
The cat is sitting on the chair by the window -> … with its tail slowly wagging back and forth

Etc... Until it reaches a "Stop condition" or a result that could be good for us.

But how can he decide to generate the words "on the chair"?

Because it will search in the data provided for all the sentences that contain "The cat is sitting on the chair...", and will find, for example, three different continuations

… near the window
… to sleep
… filling it with fur

Each one will be assigned a percentage of probability of 'interest', will roll a dice to generate some confusion and will choose "... near the window" because it will probably be the one most interesting for us.

NB. I apologize to all the teams of programmers, scientists, mathematical theorists and gurus who developed the Deep Learning algorithms for this extreme simplification.

So what…

So to recap: an LLM (Large Language Model) is an AI that has been trained on large amounts of linguistic data and has learned to generate words and phrases that may appear to make sense, but have no real understanding of what they mean of what it is producing.

These systems use natural language processing techniques, such as semantic analysis, syntax and morphology, to generate sentences that are grammatically correct and consistent with the context in which they are found. However, the AI does not really understand the meaning of the words and phrases it is producing like a human would, it just relies on vast amounts of data to produce acceptable results.

In other words, it may produce sentences that seem to make sense because it has learned from the vast amounts of linguistic data it has been exposed to, but it has no real understanding of the meaning of what it is producing. Conversely, humans have the ability to understand the meaning of words and phrases more deeply and to think critically about their meaning and the context in which they occur.

However...the result they produce makes tremendous sense to us

PS. But what does GPT mean?

Finally, a little nerd culture. What does GPT mean?

GPT stands for Generative Pretrained Transformer

• Generative - because it is able to generate the next word

• Pretrained - because he has been pre-trained on a lot of text from the Internet and is therefore ready to answer questions

• Transformer - because the underlying deep neural network uses a particular type of network architecture, a model, called a Transformer.