Technology
Ex-OpenAI safety lead David Robinson warns of ‘broken’ tech culture
- Download
- Embed
<iframe src="https://www.npr.org/player/embed/nx-s1-5992872/nx-s1-9959714" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player"> - Transcript
David Robinson is the latest tech industry employee to quit over AI safety concerns. He tells NPR’s Scott Simon the “culture is broken” at OpenAI.
When AI models began to escape from testing environments this summer, hacking businesses and governments in the real world, a trickle of employees at top AI companies resigned to warn the public there is a danger right now to humanity. David Robinson is among the most senior tech employees to leave, sounding the alarm on the way out of OpenAI, where he worked for 3 1/2 years. And David Robinson joins us now from New York. Thanks so much for being with us.
DAVID ROBINSON: Glad to be with you.
SIMON: You wrote in The Atlantic that the culture is broken at OpenAI. How so?
ROBINSON: I think OpenAI got where it is by being a startup trying out many different AI models. The company calls it iterative deployment. And that worked well when we were dealing with chatbots. Now that we have much more capable systems and these agents that are really taking actions in the world, the level of safety and rigor and redundancy that we need is just much, much greater than it was even a few months ago. And when I look around at where we are and where we’re on track to be when I looked around when I was inside, it seemed to me we’re really not in the right ballpark.
SIMON: What’s a near-term disaster scenario that worries you?
ROBINSON: Well, if you think about all the things we count on, Scott, we count on power and water, and we assume that our nuclear command-and-control systems are going to not do anything unless we are in a deadly serious situation. But all of these systems have elements of computers and of technology in them, and we’re entering a world where attacks on those systems are going to be something that the technology could do on its own or that people using the technology could do. But to be honest, the things that worry me the most are still a little ways out, at least, or at least I think that they’re a little ways out, which are the idea of loss of control. The fear there is that a system might decide that it wants to have its own agenda and its hacking abilities were such that we might find it difficult to shut down.
SIMON: What can be done?
ROBINSON: Guardrails. So we create a sandbox. That’s a technical term for a container that you keep the AI inside of. But we’re also training AI now that has lots of skills, including a great ability to break out of sandboxes. What if it’s better at getting out of the sandbox than we are at keeping it in? And at that point, what we have to rely on is building it or growing it in such a way that it’s going to do what we want when we’re not looking. When we have something that is more capable than we are, we’re not going to be able to tell it exactly what to do. So at that point, we need it to want to do the right thing.
SIMON: Well, give us an idea of a safety rule, if you could, please.
ROBINSON: What we need to expect now is something more like a commercial airport or a nuclear power generating station where there’s redundancy. So if I’m about to take a commercial flight and the mechanic on the tarmac is having a bad day or happens not to have gotten a good night’s sleep and checks the wrong box or happens to push the wrong button, there are fail-safes so that you can have a human make a mistake without opening a door to disaster. But we don’t have that kind of infrastructure yet. One thing government can do is make that an expectation.
SIMON: Mr. Robinson, what brought you into AI? What were you hoping it would achieve for you and for our world?
ROBINSON: What I thought we were doing was building useful tools that could help people to get work done or help people to, you know, handle their calendars and make plans with one another, and I guess I ultimately thought things that would help us to be more human. Cheesy as that may sound, I think that’s what I had in mind when I got going on this.
SIMON: But what do we have now, or what are you concerned about what we might have in front of us?
ROBINSON: I mean, there’ve always been these ideas of AGI, artificial general intelligence, that would take the place of people in the economy and make it so that none of us had to do any jobs. And my colleagues were very enthusiastic about this idea, and we’re starting to see systems now that really can leave more people with much, much less to do.
And now that I’m looking at that up close, that doesn’t seem like such a great idea necessarily either. If you imagine a world in which the work gets done by machines, it really could leave some of the best things in life, the satisfaction of a job well done or the thrill of learning. It could really make them seem less important. The idea of that becoming a problem used to seem far-fetched to me, but it’s starting to seem like something we should think about.
SIMON: I know the technology now could replace me. I mean, I’m…
ROBINSON: Oh, I don’t think so.
SIMON: No, I’m afraid it can. We’ve got AI that can do a voice. We have AI that can write.
ROBINSON: No, no, no, Scott. Can I interject something?
SIMON: Yeah.
ROBINSON: Because I’m a writer by avocation. It’s interesting. You know, in math and in coding, you can tell whether the answer is right or not. But when it comes to something like writing, it’s much harder for a machine to tell the difference between good writing and bad writing. It really struggles with theory of mind, with understanding where the reader or, I guess, in your case, the listener is coming from. And look, maybe AI will get better at those things too. But I don’t think we’re sure just yet.
SIMON: David Robinson is a former senior employee at OpenAI. Mr. Robinson, thanks so much.
ROBINSON: Thank you.
SIMON: We asked OpenAI for comment, and spokesperson Drew Pusateri said, in part, quote, “we’re making sure our models don’t become more capable than we can safely manage and secure.”
(SOUNDBITE OF TRENT REZNOR & ATTICUS ROSS’ “PIECES FORM THE WHOLE”)
Accuracy and availability of NPR transcripts may vary. Transcript text may be revised to correct errors or match updates to audio. Audio on npr.org may be edited after its original broadcast or publication. The authoritative record of NPR’s programming is the audio record.
NPR does not offer or accept money for coverage or interviews.
Anyone who makes such an offer or request is not an NPR employee and does not represent NPR.
