kamenlady@lemmy.world
on 15 May 2024 20:02
nextcollapse
Misaligned AI systems can malfunction and cause harm. AI systems may find loopholes that allow them to accomplish their proxy goals efficiently but in unintended, sometimes harmful, ways (reward hacking).
They may also develop unwanted instrumental strategies, such as seeking power or survival because such strategies help them achieve their final given goals. Furthermore, they may develop undesirable emergent goals that may be hard to detect before the system is deployed and encounters new situations and data distributions.
Today, these problems affect existing commercial systems such as language models, robots, autonomous vehicles, and social media recommendation engines.
The last paragraph drives home the urgency of maybe devoting more than just 20% of their capacity for solving this.
schmorpel@slrpnk.net
on 15 May 2024 20:28
collapse
They already had all these problems with humans. Look, I didnât need a robot to do my art, writing and research. Especially not when the only jobs available now are in making stupid robot artists, writers and researchers behave less stupidly.
dgerard@awful.systems
on 15 May 2024 20:24
nextcollapse
you can tell at a glance which subculture wrote this, and filled the references with preprints and conference proceedings
BaroqueInMind@lemmy.one
on 16 May 2024 13:18
collapse
I cannot, please elaborate.
dgerard@awful.systems
on 16 May 2024 13:18
collapse
the lesswrong rationalists
Zagorath@aussie.zone
on 16 May 2024 01:13
collapse
I genuinely think the alignment problem is a really interesting philosophical question worthy of study.
Itâs just not a very practically useful one when real-world AI is so very, very far from any meaningful AGI.
Soyweiser@awful.systems
on 16 May 2024 12:42
collapse
One of the problems with the âalignment problemâ is that one group doesnât care about a large part of the possible alignment problems and only cares about theoretical extinction level events and not about already occurring bias, and other issues. This also causes massive amounts of critihype.
sinedpick@awful.systems
on 15 May 2024 20:21
nextcollapse
This is like trying to install airbags on a car that can barely break 5 km/h.
Cool analogy, but I think youâre overestimating OpenAI. Isnât it more like installing airbags on a little tikes bubble car and then getting a couple of guys to smash it into a wall real fast to âcheck it out broâ
Soyweiser@awful.systems
on 16 May 2024 12:39
collapse
Cstross was right! You tell everybody. Listen to me. Youâve gotta tell them! AGI is corporations! Weâve gotta stop them somehow!
Soyweiser@awful.systems
on 16 May 2024 21:03
nextcollapse
(Before I get arrested for anti-corporate terrorism, this was a joke about soylent green).
smiletolerantly@awful.systems
on 25 May 2024 12:01
collapse
This is the first mention of Accelerando Iâve seen in the wild. (Assuming it is. Iâm not sure.)
Soyweiser@awful.systems
on 25 May 2024 13:25
collapse
It was more Soylent green, but it also is partially based on (friend of the club) the writings of C Stross yes. I think he also has a written lecture on corps being slow paper clipping AGI somewhere.
smiletolerantly@awful.systems
on 25 May 2024 16:33
collapse
That is a beautiful comparison. Terrifying, but beautifully fitting.
I read Stross right after Banks. I think if I hadnât, Iâd be an AI-hype-bro. Banks it the potential that could be, Stross is what weâll inevitably turn AI into.
gerikson@awful.systems
on 25 May 2024 17:26
nextcollapse
Banks neatly sidesteps the âAI will inevitably kill usâ scenario by making the Minds keep humans around for amusement/freshness. Part of the reasons for the Culture-Idiran war in Consider Phlebas and Look to Windward was that the Idirans did not want Minds in charge of their society.
smiletolerantly@awful.systems
on 25 May 2024 18:33
collapse
Noone was trying to force that on them though, the actual reason IIRC correctly that Idirans had a religious imperative for expansion, and the Culture had a moral imperative to prevent other sentientsâ suffering at the hands of the Idirans.
IMO he mostly sidestepped the issue by clarifying that this is NOT a future version of âusâ
gerikson@awful.systems
on 25 May 2024 20:29
nextcollapse
OK I misremembered that part. It makes sense that after suffering trillions of losses the Culture would take steps to prevent the Idirans from doing it again.
And by âusâ I meant fleshy meatbags, as opposed to Minds. Although in Excession he does raise the issue that there might be âpsychoticâ Minds. Gray Areaâs heart(?) is in the right place but itâs easy to imagine them becoming a vigilante and pre-emptively nuking an especially annoying civilization.
hirvox@mastodon.online
on 26 May 2024 03:31
collapse
@smiletolerantly@gerikson AFAIR there was a short story where the Culture takes a look at Earth around the 70ies and decides to leave it alone for now.
smiletolerantly@awful.systems
on 26 May 2024 04:55
collapse
Yep. They leave us alone so weâll function as a control group. This way contact can later point at us and go âlook! Thatâs what happens if we donât intervene!â
mawhrin@awful.systems
on 25 May 2024 20:10
collapse
strossâ artificial intelligences are very unlike corporations though, and different between the books. the eschaton ai in singularity sky is quite benevolent, if a bit harsh; the ai civilization in saturnâs children is on the other hand very humanlike (and the primary reason there are no meatsacks in saturnâs children et al. is that humans enslaved and abused the intelligences they created).
threaded - newest
<img alt="" src="https://awful.systems/pictrs/image/b5ddbc68-c31f-4f16-84c1-bfc0ef9799bc.png">
OpenAI: âOur AI is so powerful itâs an existential threat to humanity if we donât solve the alignment issue!â
Also OpenAI: âWe can devote maybe 20% of our resources to solving this, tops. We need the rest for parlor tricks and cluttering search results.â
Show me you can solve the alignment issue with a 2.5 year old first.
en.wikipedia.org/wiki/AI_alignment
The last paragraph drives home the urgency of maybe devoting more than just 20% of their capacity for solving this.
They already had all these problems with humans. Look, I didnât need a robot to do my art, writing and research. Especially not when the only jobs available now are in making stupid robot artists, writers and researchers behave less stupidly.
you can tell at a glance which subculture wrote this, and filled the references with preprints and conference proceedings
I cannot, please elaborate.
the lesswrong rationalists
I genuinely think the alignment problem is a really interesting philosophical question worthy of study.
Itâs just not a very practically useful one when real-world AI is so very, very far from any meaningful AGI.
One of the problems with the âalignment problemâ is that one group doesnât care about a large part of the possible alignment problems and only cares about theoretical extinction level events and not about already occurring bias, and other issues. This also causes massive amounts of critihype.
This is like trying to install airbags on a car that can barely break 5 km/h.
Cool analogy, but I think youâre overestimating OpenAI. Isnât it more like installing airbags on a little tikes bubble car and then getting a couple of guys to smash it into a wall real fast to âcheck it out broâ
Cstross was right! You tell everybody. Listen to me. Youâve gotta tell them! AGI is corporations! Weâve gotta stop them somehow!
(Before I get arrested for anti-corporate terrorism, this was a joke about soylent green).
This is the first mention of Accelerando Iâve seen in the wild. (Assuming it is. Iâm not sure.)
It was more Soylent green, but it also is partially based on (friend of the club) the writings of C Stross yes. I think he also has a written lecture on corps being slow paper clipping AGI somewhere.
That is a beautiful comparison. Terrifying, but beautifully fitting.
I read Stross right after Banks. I think if I hadnât, Iâd be an AI-hype-bro. Banks it the potential that could be, Stross is what weâll inevitably turn AI into.
Banks neatly sidesteps the âAI will inevitably kill usâ scenario by making the Minds keep humans around for amusement/freshness. Part of the reasons for the Culture-Idiran war in Consider Phlebas and Look to Windward was that the Idirans did not want Minds in charge of their society.
Noone was trying to force that on them though, the actual reason IIRC correctly that Idirans had a religious imperative for expansion, and the Culture had a moral imperative to prevent other sentientsâ suffering at the hands of the Idirans.
IMO he mostly sidestepped the issue by clarifying that this is NOT a future version of âusâ
OK I misremembered that part. It makes sense that after suffering trillions of losses the Culture would take steps to prevent the Idirans from doing it again.
And by âusâ I meant fleshy meatbags, as opposed to Minds. Although in Excession he does raise the issue that there might be âpsychoticâ Minds. Gray Areaâs heart(?) is in the right place but itâs easy to imagine them becoming a vigilante and pre-emptively nuking an especially annoying civilization.
@smiletolerantly @gerikson AFAIR there was a short story where the Culture takes a look at Earth around the 70ies and decides to leave it alone for now.
Yep. They leave us alone so weâll function as a control group. This way contact can later point at us and go âlook! Thatâs what happens if we donât intervene!â
strossâ artificial intelligences are very unlike corporations though, and different between the books. the eschaton ai in singularity sky is quite benevolent, if a bit harsh; the ai civilization in saturnâs children is on the other hand very humanlike (and the primary reason there are no meatsacks in saturnâs children et al. is that humans enslaved and abused the intelligences they created).