If anyone builds it, everyone dies

[Friendly Engineer]

This book was genuinely quite sobering for me.
I had at least hoped there was a chance artificial superintelligence would benefit humankind.

After reading the book it's quite clear that a so-called "good (aligned) artificial superintelligence" less likely than winning the lottery and would ultimately lead to our extinction.
Nuclear weapons couldn't permanently kill every human, even in the worst-case scenarios you'd have many survivors rebuild. This most likely would.

If Anyone Builds It, Everyone Dies (ideally buy the book here, but if you can't afford it there's a link to a free PDF below)

https://www.hedweb.com/private/if-anyone-builds-it-everyone-dies.pdf

 

[Friendly Engineer]

I would recommend everyone here reads it. It's 4.15AM here as I couldn't stop when I opened it, very much a page turner.

I was surprised by just how little even our most advanced AI engineers actually know what they're doing. It's mostly tweaking numbers, seeing what happens, and hoping for the best.

 

[Friendly Engineer]

I would recommend everyone here reads it. It's 4.15AM here as I couldn't stop when I opened it, very much a page turner.

I was surprised by just how little even our most advanced AI engineers actually know what they're doing. It's mostly tweaking numbers, seeing what happens, and hoping for the best.

It's almost as if our species has come across a technology from 1000 years in the future, something we're absolutely not ready for.