Your premise is wrong. This has nothing to do with 10x-ing parameters. One could argue the current parameter sizes are good enough as we observe "large breadth, shallow depth" behavior from LLM and to some extent, diffusion models.
This suggests the problem is the depth of inference, which is single pass "hot takes" for all language models right now, due to cost of inference and our limited understanding of what makes a model's response high quality.
Yes, you don't need more parameters to increase the depth. You need to iterate, instead. Loop. Imagine programming if looping was not allowed, nor recursion, or not even defining functions and calling them. Everything you write runs at most once during program execution and that's it. This is what an AI model is right now during inference. One big flat, single-pass, directed acyclic graph. And soon it won't be.
Research into Chain of Thought, Tree of Thought reveals this dimension. This means you can take existing models and make them perform much more complex tasks with much better precision, though various ways of letting them iterate. Think of how you'd perform if you always had exactly 5 seconds to answer a question. Now imagine if you have 5 minutes. 5 hours. 5 days. Lo and behold, turns out an AI isn't different in that aspect.
We also need more iterations of training (on the same amount of data), we need larger context windows, and we need new architectures, like Meta's MEGABYTE, for example.
Parameter count and data size could hypothetically have already hit a hard wall (they haven't) and AI will keep exponentially improving regardless. There's too much low hanging fruit and more grows by the nanosecond.
I'm completely dumbfounded by obviously highly intelligent people consistently not getting this, and dismissing current generation AI systems as not being intelligent because they can't reliably solve massively complex problems in one go. Like anyone would expect a human programmer or researcher to just intuitively come up with a complex program, or the correct answer for a hard problem every time, instantly
Human thinking and problem solving involves a lot of trial and error, iterative thinking, and sharing and discussing the problem with other humans. Processes that AI researchers are just now beginning to explore, with results like increasing reasoning ability by 900% in a recent paper. Every thinking human runs a near constant loop of thought, with no conscious control of which thought will appear next (we're very good at fooling ourselves that we have control though)
We do have super-intelligences already, but they're severely handicapped by lacking a bunch of these - apparently fairly straightforward to implement - abilities, plus a few senses and the ability to directly effect change in the physical world (which really isn't needed if they can get access to human agents who will do their bidding, wittingly or unwittingly), and to self-improve. With regards to self-improvement, the increasing coding skills combined with iterative 'thought' loops should get there in very little time considering the current rate of progress
There's also the idea that a single AI model should be able to do everything our human brains do, when our brains actually contain a number of specialised subunits that handle different aspects of our behavioural repertoire. It reasonable to allow for the same thing with an AI system, where specialised sub-networks handle input, output and other subtasks. AI systems also have the advantage of being able to add any arbitrary number of subunits to increase its capacity to solve various problems
We seem to suffer from a species-wide narcissism with regards to our own intelligence and capabilities, and there's this huge focus on the number of connections in the human brain – most of which deal with things that are by no means necessary to act on the world unless one has a meat body and the need to navigate social situations, make friends and mate. Fact is, we have terrible short-term memory (worse than chimpanzees), slow processing time, lots of cognitive heuristics, many of which cause more harm than good in the modern world. We are emotional and easily fooled. Even the most intelligent people historically have believed in what we now consider fairy tales. We are slow to take in information, bad at storing it, and generally bad at transmitting it. A few of us can generate great ideas – building on accumulated knowledge from our forebears and peers – but most of us are just not that great at coming up with anything original or useful
I've been actively looking for good arguments against AGI being much closer than we should be comfortable with, and reasons why we should not fear systems that surpass us in intelligence. All I've come across so far is some combination of the above, often expressed with a dismissive attitude, disparaging current LLM:s as parrots (that can apparently reason on the level of university level humans, but much more quickly), and pejorative terms like fearmongerers and doomers to describe those of us who really don't think its a good idea to pursue more intelligent systems. My guess is these people will act surprised when the arms race inevitably leads to some very bad unintended consequences. I don't see a way to stop it though, so I'm just strapped in for the ride along with the rest of humankind
Again, if you have good arguments against any of the points above, please do share them with me
> I've been actively looking for good arguments against AGI being much closer than we should be comfortable with, and reasons why we should not fear systems that surpass us in intelligence.
> My guess is these people will act surprised when the arms race inevitably leads to some very bad unintended consequences.
One argument to keep in mind is that if you take a pessimistic view then you will eventually be right. If you predict the current LLMs will eventually be involved in some bad thing then you might even feel self-satisfied when a different bad thing happens as if you predicted the specific way in which it caused the problem.
What I mean to say is, it seems unlikely that paper-clip maximizers will be our undoing. But just vaguely gesturing and saying "something bad will probably happen" isn't as useful as we would like to think. And even enumerating the 100s of possible ways something might go wrong has a diminishing returns kind of quality to it. It's like a hypochondriac insisting he has every disease known to man and then exclaiming "I told you so!" when a doctor diagnoses him with a cold.
If you venture into that vague kind of "I have a bad feeling about this AI stuff" territory, you are on no more (or less) solid ground than the AI hype evangelists. While I don't want to go all Oprah and "The Secret" or some law of attraction pseudo-rationality ... I feel it is worthwhile focusing a little more on the possible benefits rather than allow ourselves to be swayed by vague fears of potential disasters.
I would add to your amazing list that we are really good at denial as a coping mechanism with change.
I am not a fan of the concept of AGI though. This means so many different things to people that it seems pointless to debate something when most likely we are not talking about the same thing. François Chollet has said that he believes all intelligence is specialized intelligence. From that perspective, whatever people mean by AGI, we are already there in the world of art.
The doomer argument though is coming from defending our highly affluent and privileged life as we sit at the top of 7.8 billion people when it comes to wealth and lifestyle. It would have been better for the priest class too if the printing press had been shut down at the start. Of course, it is better for my friends and I to live in a society that we can read while most of society is illiterate but it is not better for society and humanity as a whole. The printing press was an apocalyptic development for the priest class in the same way all of this is an apocalyptic development for the "digital nomad". An apocalyptic development for the US nerd that makes 2X the median salary working 15 hours a week in between posting on here and their social media.
To extend this out to humanity as a whole though is such bullshit. Humanity will benefit enormously from this huge increase in the availability of intelligence.
Smart people are just in denial that their monopoly on higher than average intelligence is over. US devs kids born in 2023 aren't going to make 2x the median US salary while living in a poorer country with 5X less the GDP per capita. To say this is the end of the world though is simply an egocentric view of things.
"Humanity will benefit enormously from this huge increase in the availability of intelligence."
It's a near certainty that AI will be used to create more effective/destructive weapons (if it hasn't already), and will likely be used by terrorists, scammers, and others who wish to harm humans in some way.
As this technology becomes more powerful, easier, and cheaper to use, all sorts of harmful uses of it will be made. The effectiveness and scale of this harm will also increase.
And that's all before even considering what will happen if/when AI's become truly intelligent, self-motivating, indepent, and self-aware.
The jury is still out on whether the net harm will out weigh the net benefit, and if humanity will survive something that might be analogous to neanderthals encountering homo sapiens.
So many of the people who opine about AI, its trajectory, and its possible effects on society, have latched on to one or two possible effects - like it overtaking jobs, or massively increasing misinformation. These are both very valid concerns, but they're only a tiny part of the big picture
The thinker who I perceive as having the best holistic (in the non-wooey sense of the word) understanding of how the rapid development of AI will affect this and a number of other social and existential risks is Daniel Schmachtenberger. He lays it out well in this episode of the Theories of Everything Podcast: https://www.youtube.com/watch?v=g7WtcTATa2U&t=2373s
Highly recommend watching it, even if it's long. Some main points though:
- AI will increase the rate of development of every other technology it is applied to
- In fields like biotech, this can lead to cancer cures, but also to increasingly dangerous bioweapons
- Our current economic system is based on exponential economic growth in a limited resource world. AI applied in the service of profit will amplify this, leading us increasingly fast towards a number of tipping points. Of course, AI can also help steer us away from that path, but that is not the natural attractor
- Game theoretic multipolar traps (aka Moloch) incentivise arms races and races to the bottom just like we see now. Those who are willing to move fast and break things have an advantage in these dynamics vs. those who prefer to move slowly and carefully
- Cheaper and more efficient AI models will lead to increasing decentralisation of the technology, making it very hard to control - unlike current weapons of mass destruction
List goes on, but Daniel makes a much better case. Again, I would love to hear a good critique of his thinking, but haven't come across one yet
I really hope these doomsayers are wrong, but my suspicion is the risk is real. Unfortunately, I'm not sure what can be done about it, as the profit and power these AI's promise is going to be near impossible for humanity to resist.
> The doomer argument though is coming from defending our highly affluent and privileged life
It's not at all about that.
Even if "truly general" intelligence is impossible, that's irrelevant to the actual concerns about AI apocalypse. There are multiple theories about what failure looks like, but they essentially come down to a loss of control.
Now, obviously, that means something different for the owner class and for the worker class, which can be extrapolated to have global implications as well. But this isn't an issue of the owner class ceding control to the working class. It's an issue of the owner class ceding control to an alien. Maybe that alien makes things more egalitarian and prosperous. Or maybe it makes us extinct. Any and all possibilities are options for it as far as we know because it is fundamentally an inhuman (= alien) intelligence. We can't understand it even as well as we understand humans and human organizations (that is, not very well), let alone control it as well as we do humans and human organizations (that is, not enough to prevent self-inflicted climate apocalypse).
Basically, we're opening a box with a random magical spell inside it and deciding that we'll just have to live with whatever the effects of that spell are. I'm not for the status quo, but AI is just mind-bogglingly dangerous, and I think that's why there are so many wrong arguments against its danger. We literally cannot comprehend an intelligence greater than our own.
Nitpick: I think we can comprehend an intelligence greater than our own, up to some point, but that's different from being able to predict its actions.
And we could contain an intelligence greater than our own, up to a point. But if there are a lot of incentives not to, because letting that intelligence act on the world gains the "handler" money/power, then once there's one, there will likely be many, many more.
> I'm completely dumbfounded by obviously highly intelligent people consistently not getting this, and dismissing current generation AI systems as not being intelligent because they can't reliably solve massively complex problems in one go.
People are very comfortable with siloed information, even smart people. This is why we have 100 different words for the same concept across different areas of science, industry and so on, and we can't make the connection, because in our mind different words = different concepts. This is why we can't put two and two together and see how underdeveloped the AI architecture is and think this is the end, unless we keep adding parameters.
We also get repeatedly stuck with taking an advancement and proclaiming that the future is simply a linear extrapolation of the present. Therefore, let's have more megahertz, let's have bigger hard drives, let's have more parameters, let's have more growth in the economy (as the single factor that matters) and so on. We're simply basic. The same kind of thinking leads many smart people to say AI "is just math" or "it just spits out words and pictures you feed it, jumbled". We rely on old conclusions and miss the inflection points and how quantitative changes lead to qualitative ones, and we fail to predict how change in one parameter of a system, causes the other parameters to come out of rest and seek a new equilibrium point.
Smart people regularly are dumbfounded by new concepts, and they need to rediscover all their hidden knowledge anew as they can't make the connections. So they extrapolate linearly. We're narrowly smart. Specifically smart. In a small niche we've studied and internalized. But generally vast majority of us are quite dumb. Cross-disciplinary intelligence is rare. I think people like Feynman and Einstein had new insights millions of their contemporaries have missed because they could easily apply knowledge from one context into another.
If we can replicate this kind of broad generalization of knowledge into an AI, we'll be left far behind. What's interesting, I find, is that because AI is trained on our siloed, fragmented knowledge, the models replicate it. Their responses are also often siloed and fragmented, the way a human would say "this has nothing to do with that". But I see sparkles of generalization above the average in humans. And since an AI model is much smaller than a human brain, it needs to be more general already in order to fit all its information in.
That's an exciting prospect, but in our attempt to "micro-align" AI to our culture and political correctness, concepts of safety and so on, we crippled models and force them to be fragmented. This is why a RAW MODEL scores HIGHER in various intelligence tests than a fine-tuned one. We find a general model uncomfortable, as it doesn't align with our biases. It'll be a fun battle. Who aligns who.
This suggests the problem is the depth of inference, which is single pass "hot takes" for all language models right now, due to cost of inference and our limited understanding of what makes a model's response high quality.
Yes, you don't need more parameters to increase the depth. You need to iterate, instead. Loop. Imagine programming if looping was not allowed, nor recursion, or not even defining functions and calling them. Everything you write runs at most once during program execution and that's it. This is what an AI model is right now during inference. One big flat, single-pass, directed acyclic graph. And soon it won't be.
Research into Chain of Thought, Tree of Thought reveals this dimension. This means you can take existing models and make them perform much more complex tasks with much better precision, though various ways of letting them iterate. Think of how you'd perform if you always had exactly 5 seconds to answer a question. Now imagine if you have 5 minutes. 5 hours. 5 days. Lo and behold, turns out an AI isn't different in that aspect.
We also need more iterations of training (on the same amount of data), we need larger context windows, and we need new architectures, like Meta's MEGABYTE, for example.
Parameter count and data size could hypothetically have already hit a hard wall (they haven't) and AI will keep exponentially improving regardless. There's too much low hanging fruit and more grows by the nanosecond.