marginalium noun

the notes one makes in the margins.

A short note on

on the alien characteristics of llms

May 21, 2023

source: www.lesswrong.com

by Dorian Minors

I did brain science at Cambridge. Now I'm a behaviour science professor. Read my shit.

On the alien characteristics of LLMs: the Waluigi effect.

Short version:

After you train an LLM to satisfy a desirable property P, then it’s easier to elicit the chatbot into satisfying the exact opposite of property P

Why?

When you spend many bits-of-optimisation locating a character, it only takes a few extra bits to specify their antipode.

If you don’t do the thinking, the thinking will be done for you.

Join the Armchair Collective. Once a month, a brain scientist’s musings on patterns of human behaviour, so you can choose the ones you keep.
More where this came from. Subscribe.