← Notes

Robert Chapman
Maybe I'm getting the arguments around alignment on AI wrong, but I know one of the things that I find challenging when I hear, especially from people who are more doomer and pessimistic about strong AI, is that their characterization of AI always sounds like an evil villain from a campy movie. It's always Doctor Evil-level types of activity.

To me, it both undersells and oversells the capabilities of AI at the same time. In some ways, I think it overarticulates how AI will function in ways that are rather silly. At the same time, it probably underestimates the areas in which AI is actually quite clever and could potentially do something harmful without better controls around it.

I think this fundamentally goes to what I think is the biggest challenge with AI right now, which is that we don't have a good mental model of how to think of it. We either think of it purely as code that runs just like any other computer program, and to some degree that's accurate, or we think about it as people. Certainly, there are ways, with all the Turing tests that we continue to move the goalposts on, where that's fair as well.

When both of those things seemingly are true at the same time, that tells me that our characterization of this is probably wrong. Our mental model is wrong. If your mental model is wrong when you're doing alignment, then that effectively means that you're aligning more or less poorly.

I still think, as a cybersecurity professional, we have to fundamentally think about this not in terms of, can we get alignment right, but just like any system, assume it could behave in a way that I don't want it to. What are the controls that are going to keep it from doing that, and how do we build defensive depth around it?

Anything short of that is simply chasing the things that are uninteresting about it. It's like trying to build a calculator program that isn't abused in some example hacking scenario. You could do that, or you could stop the things that allow people to get to Calculator and to pop it. These are the kinds of things where I think we sometimes focus on the wrong area.

A note by Robert Chapman. More notes