> I've been actively looking for good arguments against AGI being much closer than we should be comfortable with, and reasons why we should not fear systems that surpass us in intelligence.
> My guess is these people will act surprised when the arms race inevitably leads to some very bad unintended consequences.
One argument to keep in mind is that if you take a pessimistic view then you will eventually be right. If you predict the current LLMs will eventually be involved in some bad thing then you might even feel self-satisfied when a different bad thing happens as if you predicted the specific way in which it caused the problem.
What I mean to say is, it seems unlikely that paper-clip maximizers will be our undoing. But just vaguely gesturing and saying "something bad will probably happen" isn't as useful as we would like to think. And even enumerating the 100s of possible ways something might go wrong has a diminishing returns kind of quality to it. It's like a hypochondriac insisting he has every disease known to man and then exclaiming "I told you so!" when a doctor diagnoses him with a cold.
If you venture into that vague kind of "I have a bad feeling about this AI stuff" territory, you are on no more (or less) solid ground than the AI hype evangelists. While I don't want to go all Oprah and "The Secret" or some law of attraction pseudo-rationality ... I feel it is worthwhile focusing a little more on the possible benefits rather than allow ourselves to be swayed by vague fears of potential disasters.
> My guess is these people will act surprised when the arms race inevitably leads to some very bad unintended consequences.
One argument to keep in mind is that if you take a pessimistic view then you will eventually be right. If you predict the current LLMs will eventually be involved in some bad thing then you might even feel self-satisfied when a different bad thing happens as if you predicted the specific way in which it caused the problem.
What I mean to say is, it seems unlikely that paper-clip maximizers will be our undoing. But just vaguely gesturing and saying "something bad will probably happen" isn't as useful as we would like to think. And even enumerating the 100s of possible ways something might go wrong has a diminishing returns kind of quality to it. It's like a hypochondriac insisting he has every disease known to man and then exclaiming "I told you so!" when a doctor diagnoses him with a cold.
If you venture into that vague kind of "I have a bad feeling about this AI stuff" territory, you are on no more (or less) solid ground than the AI hype evangelists. While I don't want to go all Oprah and "The Secret" or some law of attraction pseudo-rationality ... I feel it is worthwhile focusing a little more on the possible benefits rather than allow ourselves to be swayed by vague fears of potential disasters.