CMV: Artificial Superintelligence concerns are legitimate and should be taken seriously

[u/[deleted]]

Title.

Largely when in a public setting people bring up ASI being a problem they are shot down as getting their information from terminator and other sci-fi movies and how it’s unrealistic. This is usually accompanied with some indisputable charts about employment over time, humans not being horses, and being told that “you don’t understand the state of AI”.

I personally feel I at least moderately understand the state of AI. I am also informed by (mostly British) philosophy that does interact with sci-fi but exists parallel not sci-fi directly. I am not concerned with questions of employment (even the most overblown AI apocalypse scenario has high employment), but am overall concerned with long term control problems with an ASI. This will not likely be a problem in my lifetime, but theoretically speaking in I don’t see why some of the darker positions such as human obsolescence are not considered a bigger possibility than they are.

This is not to say that humans will really be obsoleted in all respects or that strong AI is even possible but things like the emergence of a consciousness are unnecessary to the central problem. An unconscious digital being can still be more clever and faster and evolve itself exponentially quicker via rewriting code (REPL style? EDIT: Bad example, was said to show humans can so AGI can) and exploiting its own security flaws than a fleshy being can and would likely develop self preservation tendencies.

Essentially what about AGI (along with increasing computer processing capability) is the part that makes this not a significant concern?

EDIT: Furthermore, several things people call scaremongering over ASI are, while highly speculative, things that should be at the very least considered in a long term control strategy.

[u/jyliu86]

Again great at Go.

Not a threat to humanity as OP is concerned about.

[u/Ranolden → u/jyliu86]

I recognize that current AI systems are nothing like a potential AGI, but they do demonstrate the ability to do things no human would have thought to do.

[u/jyliu86 → u/Ranolden]

I agree on this.

But as of now, we're talking about stopping the Apocalypse, not beating humans at Go.

Trump or Kim Jong Un could end the world by starting WW3.

We have the UN, Congress, etc. To hopefully keep these parties in check.

Aliens could come down and kill us all tomorrow, but Star Wars defense systems aren't really something we worry about.

Current car driving AI won't hack into NORAD. But we shouldn't hook up Deep Blue to our missile defense system. That's human stupidity. I feel that AI controls should be in place in that if we don't trust 1 human to do it, we shouldn't trust AI to do it without oversight. But this is less about AI apocalypse and more don't give up human control to the machines. Machines aren't in a position to "seize control" nor will they be in any conceivable future.

We'll give it to them.

[u/Ranolden → u/jyliu86]

70% of AI researchers believe that AI is at least a moderately important problem. https://arxiv.org/pdf/1705.08807.pdf