LLMs are gigantic curves fitted to civilizational scale datasets. LLM predictions are based on this. A language model is a mathematical construct and can only be as intelligent as that Algebra book sitting on your shelf.
An algebra book is a collection of paper pages with ink on them. An LLM is... nothing like that at all. LLMs are complex machines that operate on data and produce data. Books are completely static. They don't do anything.
Do you have a better analogy? I'd like to hear more about how ML models can't be intelligent, if you don't mind.
I'm pretty skeptical of the idea that we know enough at this point to make that claim definitively.
>LLMs are gigantic curves fitted to civilizational scale datasets
>A language model is a mathematical construct
That is like telling someone from the Middle Ages that a gun is merely an assemblage of metal parts not too different from the horseshoes and cast-iron nails produced by your village blacksmith and consequently it is safe to give a child a loaded gun.
ADDED. Actually a better response (because it does not rely on an analogy) is to point out that none of the people who are upset over the possibility that most of the benefits of AI might accrue to a few tech titans and billionaires would be in the least bit re-assured by being told that an AI model is just a mathematical construct.
Pure LLM based approach will not lead to AGI, I'm 100% sure. A new research paper has shown [0] that no matter what LLM model is used, it exhibits diminishing returns, when you would be wanting at least a linear curve when looking for AGI.
Obviously feeding more data won't do anything besides increase the knowledge available.
Next steps would be in totally different fields, like implementing actual reasoning, global outline planning and the capacity to evolve after training is done.
I'm 100% certain that I need to do more than just predict the next token to be considered intelligent. Also call me when ChatGPT can manipulate matter.
A human brain certainly does do predictions, which is very useful to the bit that makes decisions. But how does a pure prediction engine make decisions? Make a judgement call? Analyze inconsistencies? Theorize? The best it can do is blindly follow the mob, a behavior we consider unintelligent even when done by human brains.
> But how does a pure prediction engine make decisions? Make a judgement call? Analyze inconsistencies? Theorize?
My intuition leads me to believe that these are arising properties/characteristics of complex and large prediction engines. A sufficiently good prediction/optimization engine can act in an agentic way, while never had that explicit goal.
I'm of the belief that the entire conscious experience is a side effect of the need for us to make rapid predictions when time is of the essence, such as when hunting or fleeing. Otherwise, our subconscious could probably handle most of the work just fine.
