Chapters
Shownotes Transcript
On September 28th, the Global Citizen Festival will gather thousands of people who took action to end extreme poverty. Join Post Malone, Doja Cat, Lisa, Jelly Roll, and Raul Alejandro as they take the stage with world leaders and activists to defeat poverty, defend the planet, and demand equity. Download the Global Citizen app today and earn your spot at the festival. Learn more at globalcitizen.org.com.
It's on!
Hi, everyone, from New York Magazine and the Vox Media Podcast Network. This is Nonprofit Open AI, which is now very much for-profit and 100% scarier. Just kidding. Actually, I'm not kidding. This is On with Kara Swisher, and I'm Kara Swisher. And I'm Naima Raza. It's amazing how an open-source nonprofit has moved to being a closed-source private company with the big deal with Microsoft. Are you shocked? No. No.
Not even slightly. It's a huge opportunity. I'm in San Francisco now, and it's really jumping with AI. Crypto didn't quite work out in any way. Those people moved to Miami. And so it's very AI-oriented right now. Everybody's thinking about a startup in
Are you more bullish on AI than Web3? Well, that's kind of a low bar. So, yeah, I've always been bullish on AI. I've talked about it a lot over the years. And, you know, this is just a version of it as it becomes more and more sophisticated and useful to people. So I've always thought it was important. And I think most of the key technologists in Silicon Valley have always thought it was important.
Agreed. I was talking to a VC yesterday, though, about how so many things that are not AI are being billed as AI tech companies now, and they're really not AI. They might have like a large learning model, but they're not quite AI. But last episode, we had Reid Hoffman on talking about what was possible with AI. And now we have one of Reid's many mentees, Sam Altman. Sam is the CEO of OpenAI, and he leads the team that has given us ChatGPT and GPT-4 and
He actually burst onto the scene as a young Stanford dropout, I think in 2005, with the startup Looped, right? Is that when you met him? Yes. When he had Looped. I visited him in his small—he was a little startup, and it didn't do very well. It was a location-based kind of thing. I don't even remember. Social network, right? Like, geosocial network. I don't even—you know, it was not Facebook, let's just say. So—
He was one of these many, many startup people that sort of were all over the valley. Very smart, but the company didn't quite work out. Yeah. It kind of went bust, I think, not many years later. But he became super important in the valley, especially in my generation. He's about my age because of Y Combinator. He led the startup accelerator that has incubated and launched Stripe, Airbnb, Coinbase. Yeah. He got there later. It was working before he got there, but...
He really led it to new heights, I think, in a lot of ways. He came in in 2014? Yes, I don't remember. I remember when he took over. But he really invigorated it and was very involved in the startup scene. It was a great role for him. He was a great cheerleader. And, you know, he's good at eyeing good startups.
Do you see him as like kind of one of the Elon Musk, Peter Thiel, Reid Hoffman's of his generation? Kind of, yeah. There's a lot of really smart people, but yeah, he's definitely special. And he really did, you know, he had a bigger mentality, more like Reid than the others, although they had it initially, not Peter Thiel. But he was thinking of big things with the startups at AI. And I really like him. I've gotten to know him pretty well over the years. And so I've always enjoyed talking to him. He's very thoughtful. He's got a lot of interesting takes on things. And
this is a really big deal now that he's sort of landed on taking open AI to these heights. Yeah, he has. Like you, he once entertained the notion of running for office in California. He thought about running for governor, something I think you've talked to him about. Yeah, we've talked about it. But he went on to revolutionize AI. So you think that's better or worse for humanity? I don't know. We'll see. You know, California is probably easier to fix than what we're going to do about AI once it gets fully deployed. Although, you know, the whole issue is there's lots of great things and there's lots of
And so we want to focus on both because, like I say, it's like when the Internet started. We didn't know what it was going to be. I think a lot of people are being very creative around what this could be and what problems it could solve and at the same time the problems it could create. Do you think that the fear is overblown like this pandemic?
Our jobs are at risk. AI is going to, you know... On those stories, yes. Yes. It's like saying, what has, you know, the car done for us or lights or something like that. You know, things will change as they always do. And so I've always thought most of the fears were overblown. But as I...
say, in the book I'm working on right now, which is why I'm in San Francisco, is everything that can be digitized will be digitized. That's just inevitable. And that's where it's going. So this will soon be two bots talking to each other? No, no. But search is so antiquated when you think about it. Typing words into a box, it's really Neanderthal in many ways. And this is an upright...
homo sapien. Well, it's been interesting because critics have kind of swarmed about ChatGPT earlier on and Sam was coming back on Twitter saying just wait for the next iteration, which we now have in GPT-4. We couldn't book the interview with him until GPT-4 was out. But
The model still has many issues, and he himself has noted this. He tweeted that it's still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it. This was about GPT-4. Yeah, I would agree. But that's a very interesting thing because the fact that it's more impressive on first blush than it is after you use it is part of the problem because I've been using my
my GPT plus and it pulls up all kinds of interesting, like write me a research paper and then it will look really good and it will have a bunch of false information on it. So this can compound the misinformation problem when something looks slick. Yeah.
Well... But isn't informed, right? Well, data in, data out. Crappy data, crap in, crap out. I mean, it's just the same. That's a very simplistic way of saying it. But I think, you know, it's like the early Internet really sucked, too. And now it kind of doesn't and sort of does. And there's great things about it. But if you looked at early Yahoo or Google or Google was much later, but early Yahoo and others, it was...
A lot of bubble gum and bailing wire. All right. Well, let's see what Sam Altman has to say and if he feels confident in the choice of having done open AI versus running for governor of California. We'll take a quick break and we'll be back with the interview. This episode is brought to you by Shopify.
That's shopify.com slash tech.
Sam, it's great to be in San Francisco, rainy San Francisco, to talk to you in person. We need the rain. It's good. I know. This atmospheric river is not kidding. A lot of them. I got soaked on the way here. I miss San Francisco. I'm here for a couple weeks. Come back. I'm going to. I'm going to. I'm trying to convince my wife. We're having a moment here. I agree. It's time to come back. I love San Francisco. I've never really left in my heart. So you started Looped. That's where I met you. Explain what it was. It was a location-based social app for mobile phones. Right. So what happened?
the market wasn't there, I'd say is the number one thing. Yeah, because? Well, I think like
You can't force a market. Like, you can have an idea about what something or what people are going to like. As a startup, part of your job is to be ahead of it. And sometimes you're right about that and sometimes you're not. You know, sometimes you make loops, sometimes you make open AI. Yeah, right, right, exactly. But you started in 2015 after being at Y Combinator. And late last year, you launched ChatGPT. Talk about that transition. You had been, you reinvigorated Y Combinator in a lot of ways.
I was handed such an easy task with Y Combinator. I mean, like, I don't know if I reinvigorated it. It was sort of a super great thing by the time I took over. No, what I mean is I think it was, it got more prominence. You changed things around. I don't mean to say it was failing. Yeah, yeah, not at all. Not at all. I...
I think I scaled it more and we sort of took on longer term, more ambitious projects. OpenAI actually sort of got, that was like something I helped start while at YC. And we did, we funded other companies, some of which I'm very closely involved with, like Helion, the nuclear fusion company. They were going to take a long time. So I definitely like had a thing that I was passionate about and we did more of it. But
I kind of just tried to keep PG and Jessica's vision going there. This is Paul Graham and Jessica. You had shifted, though, to open AI. Why was that? When you're in this position, which is a high-profile position in Silicon Valley, sort of king of startups, essentially, why go off? Is it you wanted to be an entrepreneur again? No, I didn't. You had started as a nonprofit. I am not a natural fit for a CEO. Like an investor really, I think, suits me very well. I got convinced that
AGI was going to happen and be the most important thing I could ever work on. I think it is going to like transform our society in many ways. And, you know, I won't pretend that as soon as we started opening, I was sure it was going to work. But it became clear over the intervening years and certainly by 2018, 2019, that we had a real chance here. What was it that made you think that? A number of things. Hard to point to just a single one. But by the time we made GPT-2 available,
Which was still weak in a lot of ways, but you could look at the scaling laws and see what was going to happen. I was like, hmm, this can go very, very far. And I got super excited about it. I've never stopped being super excited about it. Was there something you saw that it just scaled or what was the...
Yeah, it was the like looking at the data of how predictably better we could make the system with more compute, with more data. And there'd already been a lot of stuff going on at Google with DeepMind. They had bought that earlier, right? Around that. Yeah, there had been a bunch of stuff, but somehow like...
It wasn't quite the trajectory that has turned out to be the one that really works. But in 2015, you wrote that superhuman machine intelligence is probably the greatest threat to the continued existence of humanity. Explain. I still think so. Okay. All right. We're going to get into that. Why did you write that then? And yet you also called it the greatest technology ever.
I still believe both of those things. I think at this point, more of the world would agree on that. At the time, it was considered a very extremely like crazy position. So explain, roll it out that you wrote was probably the greatest threat to continued existing humanity and also one of the greatest technologies that could improve humanity. Roll those two things out.
Well, I think we're seeing finally little previews of this with chat GPT, especially when I've put GPT-4 out. And people can see this vision where, just to pick one example out of the thousands we could talk about, everyone in the world can have an amazing AI tutor on their phone with them all the time for anything they want to learn. That's really, we need that. I mean, that's wonderful. That'll make the world much better. The creative enhancement that people are able to get from using these tools to do whatever their creative work is,
That's fantastic. The economic empowerment, all of these things. And again, we're seeing this only in the most limited primitive larval way.
But at some point, it's like, well, now we can use these things to cure disease. So what is the threat? Because when I try to explain it to regular people who don't quite understand... I'm not a regular person? No, you're not. You're not a regular person. I'm so offended. I'm not a regular person. But when the internet started, nobody knew what it was going to do. When you thought superhuman machine intelligence was probably the greatest threat, what did you mean by that? I think there's levels of threats. So today, we can look at these systems and say, all right, no imagination required. We can see how this can contribute to...
computer security exploits or disinformation or other things that can destabilize society. Certainly there's going to be economic transition. And those are not in the future. Those are things we can look at now. In the medium term, I think we can imagine these systems get much, much more powerful. Now what happens if a really bad actor gets to use them and tries to figure out how much havoc they can wreck on the world or harm they can inflict? And then we can go further to...
all of the sort of traditional sci-fi, what happens with the kind of runaway AGI scenarios or anything like that. Now, the reason we're doing this work is because
We want to minimize those downsides while still letting society get the big upsides. And we think it's very possible to do that, but it requires, in our belief, this continual deployment in the world where you let people gradually get used to this technology, where you give institutions, regulators, policymakers time to react to it, where you let people feel it, find the exploits, find...
the creative energy of the world will come up with use cases we and all the red teamers we could hire would never imagine. And so we want to see all of the good and the bad and figure out how to continually minimize the bad and improve the benefits. And you can't do that in the lab. And this idea that we have that
We have an obligation and society will be better off for us to build in public, even if it means making some mistakes along the way. Right. I think that's really important. When people critique chat GPT, you essentially said wait for GPT-4. Now that it's out, has it met expectations?
A lot of people seem really happy with it. There's plenty of things that's still bad. And that you are expectations. Yeah, I'm proud of it. Again, very long way to go, but as a step forward, I'm proud of it. So you tweeted that at first glance that GPT-4 seems, quote, more impressive than it actually is. Why is that? Well, I think that's been an issue with every version of these systems, not particularly GPT-4. You find these, like, flashes of brilliance before you find the problems. And so...
A thing that someone used to say about GPT-3 that has really stuck with me is it is the world's greatest demo creator because you can tolerate a lot of mistakes there.
But if you need a lot of reliability for a production system, it wasn't as good at that. Now, GPT-4 makes less mistakes. It's more reliable, more robust, but still a long way to go. One of the issues is hallucinations. They're called hallucinations, which is kind of a creepy word, I have to say. What do you think we should call it instead? Mistakes. Mistakes. Or something like hallucinations feels like it's sentient. It's interesting. Hallucinations, that word doesn't trigger for me as sentient. But...
I really try to make sure we're picking words that are in the tools camp, not the creatures camp, because I think it's tempting to anthropomorphize this in a very bad way. That's correct. And as you know, there were a series of reporters wanting to date GPT-3. But anyway, sometimes a bot just makes things up out of thin air, and that's hallucinations happen. Now, it'll cite research papers or news articles that don't exist. You said GPT-4 does this less than GPT-3. We should all give them actual names, but it still happens. No, that would be anthropomorphizing.
I think it's good that it's letters plus a number. Not like Barbara? Anyway, but it still happens. Why is that? So the...
These systems are trained to do something which is predict the next word in a sequence. And so it's trying to just complete a pattern and given its training set, this is the most likely completion. That said, the decrease from 3 to 3.5 to 4 I think is very promising. We track this internally and every week we're able to get the number lower and lower and lower.
I think it'll require combinations of model scale, new ideas, a lot of user feedback. More data. Model scale is more data. Not necessarily more data, but more compute thrown at the problem. Human feedback, people like flagging the errors for us, developing new techniques so the model can tell when it's about to kind of go off the rails. Real people just saying this is a mistake. Yeah. One of the issues is that it obviously compounds a very serious misinformation problem. Yeah, so we don't... We pay...
experts to flag to go through and label the data for us. Not just bounties, but we employ people, we have contractors, we work with external firms. We say we need experts in this area to help us go through and improve things. You don't just want to rely totally on, you know, random users doing whatever, trying to troll you or anything like that. So humans, more compute, what else? To reduce the... Yeah. I think that there is going to be a big new algorithmic idea. Yeah.
that, you know, a different way that we train or use or tweak these models, different architecture perhaps. So I think we'll find that at some point. Meaning what? For the non-techie, the different architecture? Oh, well, it could be a lot of things, but you could say like a different algorithm, but just some different idea of the way that we create or use these models that encourages during training or inference time when you're using it, that encourages the
The models to really ground themselves in truth, be able to cite sources. Microsoft has done some good work there. We're working on some things. So talk about the next steps. How does this move forward? I think we're sort of on this very long-term exponential. And I don't mean that just for AI, although AI too, I mean that is like cumulative human technological progress.
And it's very hard to calibrate on that. And we keep adjusting our expectations. I think if we told you five years ago we'd have GPT-4 today, you'd maybe be impressed. But if we told you four months ago after you use ChatGPT, we'd have GPT-4 today, probably not that impressed. And yet it's the same continued exponential. So maybe where we get to a year from now, you're like, eh, you know, it's better. But sort of the new iPhone is always a little better too. Right. But...
If you look at where we'll be in 10 years, then I think you'd be pretty impressed. Right, right. Actually, the old iPhones were not as impressive as the new ones. For sure, but it's been such a gradual process. That's correct. That unless you hold that original one and this one back to back. Right, right. I just found mine the other day, actually. Interestingly enough, that's a very good comparison. You're getting criticism for being secretive, and you said competition and safety require that you do that. Critics say that's a cop-out. It's just about competition. What's your response? I mean, it's clearly not. We...
We make no secret of like, we would like to be a successful effort. And I think that's fine and good. And we try to be clear, but also we have made many decisions over the years in the name of safety that have been widely ridiculed at the time that are later, um,
you know, people come to appreciate when we, when we, even in the early versions of GPT, when we talked about not releasing model weights or releasing them gradually because we wanted people to have time to adapt, we got ridiculed for that. And I totally stand by that decision.
Would you like us to like push a button and open source GPT-4 and drop those weights into the world? Probably not. Probably not. One of the excuses that tech always uses is you don't understand it. We need to keep it in the back books. It's often about competition. Well, for us, it's the opposite. I mean, we've said all along, and this is different than what most other AGI efforts have thought, is everybody needs to know about this. Like, AGI should not go be built.
Mm-hmm.
familiarity with what is in the process of happening, the ability to weigh in, to shape this together. We want that. We need that input and people deserve it. So I think we're not the secretive company. We're quite the opposite. We put the most advanced AI in the world in an API that anybody can use.
I don't think that if we hadn't started doing that a few years ago, Google or anybody else would be doing it now. They would just be using it secretly to make Google search better.
He said, OpenAI was created as an open source, which is why I named it OpenAI, nonprofit company to serve as a counterweight to Google, but now has become closed source, maximum profit company, effectively controlled by Microsoft, not what I intended at all. We're talking about open source versus closed, but what about his critique that you're too close to the big guys?
I mean, most of that is not true. Okay, let's go through it. And I think Elon knows that. We're not controlled by Microsoft. Microsoft doesn't even have a board seat on us. We are an independent company. We have an unusual structure where we can make...
very different decisions than what most companies do. I think a fair part of that is we don't open source everything anymore. We've been clear about why we think we were wrong there originally. We still do open source a lot of stuff. You know, open sourcing Clip was something that kicked off this whole generative image world. We recently open sourced Whisper. We open sourced Tools. We'll open source more stuff in the future. But I
I don't think it would be good right now for us to open source GPT-4, for example. I think that would cause some degree of havoc in the world, or at least there's a chance of that. We can't be certain that it wouldn't. And by putting it out behind an API, we are able to get many, not all, many of the benefits we want of broad access to this society, being able to understand it, update and think about it. But when we find...
some of the scarier downsides were able to then fix them. How do you respond to when he's saying you're a closed source maximum profit company? I'll leave out the control by Microsoft, but in strong partnership with Microsoft. Which was against what he said. I remember years ago when he talked about this. This was something he talked about a lot. Was what part? Oh, we don't want these big companies to run it. If they run it, we're doomed. You know, he was much more dramatic than most people. So we're a capped profit company. Yeah. We invented this new thing. Mm-hmm.
Where we started as a nonprofit. Explain that. Explain what a cap profit is. Our shareholders can make us, which is our employees and our investors, can make a certain return. Like their shares have a certain price that they can get to. But if OpenAI goes and becomes a multi-trillion dollar company, whatever, almost all of that flows to the nonprofit that controls us. Not like people hit a cap and then they don't get any more. What is the cap?
It continues to vary as we have to raise more money, but it's like much, much, much, and will remain much smaller than like any tech company. What?
In terms of like a number, I truly don't know. But it's not a significant, the nonprofit gets the significant chunk of the revenue. Well, it gets, no, it gets everything over a certain amount. So if we're not very successful, the nonprofit might not, or it gets a little bit along the way, but it won't get any appreciable amount. The goal of the capped profit is in the world where we do succeed at making AGI and we have a significant lead over everybody else. And, you know, that could become much more valuable, I think, than
maybe any company out there today. That's when you want almost all of it to flow to a nonprofit, I think. Right. I want to get back to what Elon was talking about. He was very adamant at the time, and again, overly dramatic, that Google and Microsoft and Amazon were going to kill us. I think he had those kind of words. There needed to be an alternative. What changed in your estimation of
To do that, to change from that idea. Oh, it was very simple. Like when we realized the level of capital we were going to need to do this, scaling turned out to be far more important than we thought. And we even thought it was going to be important later.
Then, we tried for a while to find a path to that level of capital as a nonprofit. There was no one that was willing to do it. So we didn't want to become a fully for-profit company. We wanted to find something that would let us get the access to and the power of capitalism to finance what we needed to do, but still be able to
fulfill and be governed by the nonprofit mission. So having this nonprofit that governs this capped profit LLC
given the playing field that we saw at the time, and I still think that we see now, was the way to get to the best of all worlds we could see. In a really well-functioning society, I think this would have been a government project. That's correct. I was just going to make that point. The government would have been your funder. We talked to them. It wouldn't have not just been that they would have been our funder, but they would have...
started the project. We've done things like this before in this country. Right, sure. But the answer is not to just say, oh, well, the government
doesn't do stuff like this anymore. So we're just going to sit around and, you know, let other countries run by us and get an AGI and do whatever they want to us. It's, we're going to like look at what's possible on this playing field. Right. So Elon used to be the co-chair and you have a lot of respect for him. I do. I'm sure you thought deeply about his critiques. Have you spoken to him directly? Was there a break or what? You two were very close as I recall. We've spoken directly recently. Yeah. And what do you make of the critiques when you hear them from him?
I mean, it can be quite in your face about this. He's got his style. Yeah. To say the positive thing about Elon, I think he really does care about a good future with AGI. That is correct. And he's, I mean, he's a jerk, whatever else you want to say about him. He has a style that is not a style that I'd want to have for myself. He's changed. But I think he does really care and he is feeling very stressed about AGI.
what the future is going to look like for humanity. Yeah, he did apply that both to when we did an interview about Tesla. He's like, if this doesn't work, we're all doomed, which was sort of centered on his car. But nonetheless, who's correct? And the same thing with this. This was something he talked about almost incessantly, the idea of either AI taking over and killing us or maybe it doesn't really care. Then he decided it was like anthills. Do you remember that example? I don't remember the anthills part. He said we're like...
You know how we think when we're riddling a highway, antels are there and we just go over them without thinking about it. So they don't, it doesn't really care. And then he said, we're like a cat and maybe they'll feed us and bell us, but they don't really care about us. It went on and on. It changed and iterated over time. But I think the most critique that I would agree with him is that these big companies would control this and there couldn't be innovation in the space. Well, I would say we're evidence against that.
Except Microsoft, and that's why I think... They're like a big investor, but again, not even a board member. So when you think... Like true full independence from them. So you think you are a startup in comparison with a giant partner? Yeah, I think we're a startup with a giant... I mean, we're a big startup at this point. And there was no way to be a nonprofit that would work?
I mean, if someone wants to give us tens of billions of dollars of nonprofit capital, we can go make that work. Or the government, which they're not. We tried. Now, he and others are working on different things. He has an anti-woke AI play. Greg Bachman also said you guys made a mistake by creating AI with a left-leaning political bias. What do you think of the substance of those critiques? Well, I think that...
This was your co-founding. Yeah, yeah. I think that the reinforcement learning from human feedback on the first version of ChatGPT was...
pretty left biased, but that is now no longer true. It's just become an internet meme. There are people, some people who are intellectually honest about this. If you go look at like GPT-4 and test it on us, it's relatively neutral. Not to say we don't have more work to do. The main thing though, is I don't think you ever get to two people agreeing that any one system is unbiased on every topic. And so giving users more control and also teaching people about like
how these systems work, that there is some randomness in a response, that the worst screenshot you see on Twitter is not representative of what these things do, I think is important. So when you said it had a left-leaning bias, what did that mean to you? And of course, they will run with that. They'll run with that quite far. People would give it these tests that score you on, you know, the political spectrum in America or whatever. And like one would be all the way on the right, 10 would be all the way on the left. You know, it would get like a 10 on all of those tests, the first version. Why?
Because of, it was a number of reasons, but largely because of the reinforcement learning from human feedback step. We'll be back in a minute.
What do you think the most viable threat to open AI is? I hear you're watching Claude very carefully. This is the bot from Anthropik, a company that's founded by former open AI folks and backed by Alphabet. Is that it? We're recording this on Tuesday. BARD launched today. I'm sure you've been discussing it internally. Talk about those two to start. Honestly, I mean, I try to pay some attention to what's happening with all these other things. It's going to be an unbelievably competitive space. I think this is the first...
new technological platform in a long period of time. The thing I worry about the most is not any of those, because I think there's room for a lot of people, and also I think we'll just continue to offer the best product. The thing I worry about the most is that we're somehow missing a better approach, and that this idea, like everyone's chasing us right now on large language models, kind of trained in the same way.
I don't worry about them. I worry about the person that has some very different idea about how to make a more useful system. Like a Facebook 2. Probably not Facebook, to be honest. Like a Facebook 2. Oh, oh, oh. No, not like Facebook. Not Facebook. No, Facebook's not going to come up with anything unless Snapchat does and then they'll copy it. I'm teasing, sort of. But you don't feel like these other efforts, that they're sort of in your same lane. You're all competing. It's the one that is not in your lane. That's what I would worry about more, yeah. Like the people that are trying to
do exactly what we're doing, but, you know, scrambling a couple... Muscling it. But is there one that you're watching more carefully? Not especially. Really? I kind of don't believe you, but really? I mean...
No, the things I was going to say, the things that I pay the most attention to are not like language model startup number 217. It's when I hear about someone, it's like, these are like three smart people in a garage with some very different theory of how to build AGI. And that's when I pay attention. Is there one that you're paying attention to now? Yeah.
There is one I don't want to say. Okay. You really don't want to say? I really don't want to say. Okay. What's the plan for making money? So we're sort of like, we have a platform, which is this API that anybody can use to the model. And then we have like a consumer product on top of it. Right. And the consumer product, 20 bucks a month for the sort of premium version.
And the API you just pay us per token, like basically like a meter. Businesses would do that depending on what they're using it for. If they decide to deploy it in a hotel or wherever. The more you use it, the more you pay. The more the more you use it, you pay. One of the things that someone said to me that I thought was very smart is if the original internet started on a more pay subscriber basis rather than an advertising basis, it wouldn't be quite so evil. I am excited to see...
if we can really do a mass scale subscription funded, not ad funded business here. - Do you see ads funding this? That to me is the original sin of the internet. - We've made the bet not to do that.
I'm not opposed to it, maybe. What would it look like? I don't know. We haven't thought. Like, it's going great with our current model. We're happy about it. You've been also competing against Microsoft for clients. They're trying to sell your software through their Azure cloud business as an add-on. Actually, that I don't. Like, that's fine. I don't care about that. That's fine. But you're also trying to sell directly sometimes the same clients. You don't care about that. I don't care about that. You don't care. How does it work? Does it affect your bottom line that way? Yeah.
Again, we're like an unusual company here. We don't need to squeeze out every dollar. Former Google Tristan Harris, who's become a critic of how tech is sloppily presented to a group in D.C. of regulators, I was there. Among the points he made is that you've essentially kicked off an AI arms race. I think that's what struck me the most. Meta, Microsoft, Google, Baidu are rushing to ship generative AI bots when the tech industry is shedding jobs. Microsoft, Google.
Recently laid off ethics and society team within its AI org. That's not your issue. But are you worried about a profit-driven arms race? I do think we need regulation and we need industry norms about this. I am disappointed to see people like we spent many, many months and actually really the years that it's taken us to get good at making these models, getting them ready before we put them out.
You know, people, it obviously became somewhat of an open secret in Silicon Valley that we had GPT-4 done for a long time. And there were a lot of people who were like, you got to release this now. You're holding this back from society. You know, this is your closed AI, whatever. But like, we just wanted to take the time to get it right. And there's a lot to learn here. And it's hard. And in fact, we try to release things to help people get it right, even competitors. Yeah.
I am nervous about the shortcuts that other companies now seem like they want to take. Such as? Oh, just rushing out these models without all the safety features built. Without safety. So this is an art that they want to get in here and get ahead of you because you've had the front seat. Maybe they do. Maybe they don't. They're certainly making some noise like, you know, they're going to. So when you say worried, what can you do about it? Nothing. You've.
Well, we can and we do try to talk to them and explain, hey, here's some pitfalls and here's some things we think you need to get right. Yeah. We can continue to push for regulation. We can try to set industry norms. We can release things that we think help other people get towards safer systems faster. Can you prevent that? Let me read you this passage from a story about Stanford doing it. They did one of their own models. $600, I think it cost them to put up. They trained a model for $600? Yeah, yeah, yeah, they did. It's called Stanford Alpaca, just so you know. That's a cute name.
It is. It's a cute name. I'll send you the story. But so what's to stop basically anyone from creating their own pet AI now for $100 or so and training it however they choose? Well, OpenAI's terms of service say you may not use output from services developed models that compete with OpenAI. And Meta says it's only letting academic researchers use Lama under a non-commercial license at this stage, although that's a moot point since the entire Lama model was leaked onto 4chan. Within hours or something. Yeah. And this is a $600 version of yours.
One of the other reasons that we want to talk to the world about these things now is this is coming. This is totally unstoppable. And there are going to be a lot of very good open source versions of this in coming years. And it's going to come with, you know, wonderful benefits and some problems by getting people used to this now, by getting regulators to begin to take this seriously and think about it now.
I think that's our best path forward. All right. Two things I want to talk about, societal impact and regulation. You've said, as I told you, this will be the greatest technology humanity has ever developed. In almost every interview you do, you're asked about the dangers of releasing AI products. And you say it's better to test AI gradually, in an open quote, while the stakes are relatively low. Can you expand on that? Why are the stakes low now? Why aren't they high right now? Relatively is the key word there. Okay. What happens to the stakes if it's not controlled now? Well,
These systems are now much more powerful than they were a few years ago. And we are much more cautious than we were a few years ago in terms of how we deploy them.
We've tried to learn what we can learn. We've made some improvements. We've found ways that people want to use this. In this interview, and I totally get why in many of these topics, I think we're mostly talking about all of the downsides. No, I'm going to ask you about the upsides. Okay. But we've also found ways to improve the upsides by learning too. So mitigate downsides, maximize upsides. That sounds good.
And it's not that the stakes are that low anymore. In fact, I think we're in a different world than we were a few years ago. I still think they are relatively low to where we'll be a few years from now. These systems are still, they have classes of problems, but there's things that are totally out of reach that we know they'll be capable of. And
The learnings we have now, the feedback we get now, seeing the ways people hack, jailbreak, whatever, that's super valuable. I'm curious how you think we're doing. I know you're... I think you're saying the right things. You're absolutely saying... Not from saying, like how you think we're doing as you look at the trajectory of our releases. I think the reason people are so worried, and I think it's a legitimate worry, is because the way the early internet rolled out, it was gee whiz almost the whole time. Yeah. Almost. Up and to the right, gee whiz, look at these rich guys. Yeah.
Isn't this great? Doesn't this help you? And they missed every single consequence. Never thought of them. I remember seeing Facebook Live and I mentioned, I said, what about, you know, people who kill each other on it? What about, you know, murderers? What about suicides? And they called me a bummer.
A bummer?
They called me and they said, that's really hurtful or really nice. I said, I'm not worried about you. I'm worried about the next guy. Like, I don't know who runs your company in 20 years with all that information on everybody. And so I think, you know, I am a bummer. And so if you don't know what it's going to be, while you can think of all the amazing things it's going to do, and it probably be a net positive for society, net positive isn't so great either sometimes, right? It's a net positive. The internet's a net positive, like electricity's a net positive, right?
But every time, it's a famous quote, every time when you invent electricity, you invent the electric chair, when you invent this and that. And so that's what would be the thing here that would be the greatest thing. Does it outweigh some of the dangers? I think that's going to be the fundamental tension that we face, that we have to wrestle with, that the field as a whole has to wrestle with, society has to wrestle with. Especially in this world we live in now, which I think we can all agree has not gotten better.
gone forward it's spinning backwards a little bit in terms of authoritarians using this you know i am super nervous about yeah what is the greatest thing you can think now you're not you and i are not creative have to think of all the things we are not going to cut not even what from your perspective and uh you know don't do term papers don't do dad jokes what do you think that's fine is that what you thought i would say for the greatest thing at all but i'm getting tired of that i don't care that it can write a press release i don't care fine sounds fantastic
The one I am personally most excited about is helping us greatly expand our scientific knowledge. I am a believer that a lot of our forward progress comes from increasing scientific discovery over a long period of time. In any area? All the areas. I think that's just what's driven humanity forward. And if these systems can help us in many different ways...
greatly increase the rate of scientific understanding. You know, curing disease is an obvious example. There's so many other things we can do with better knowledge and better understanding of science. AI has already moved in that, or folding proteins and things like that. So that's the one that I'm personally most excited about. Is science. Yeah, but there will be many other wonderful things too. You just, you asked me what my one was. Is there one unusual thing that you think will be great that you've seen already that you're like, that's pretty cool? Yeah.
Using some of these new AI tutor-like applications is like, I wish I had this when I was growing up. I could have learned so much and so much better and faster. And when I think about what
kids today will be like by the time they're finished with their formal education and how much smarter and more capable and better educated they can be than us today. I'm excited for that. Using these tools. Using these tools. I would say health information to people who can't afford it is probably the one I think is most promising. That's going to be transformative. We've seen, even for people who can't afford it, this in some ways will just be better. It's broken for most people. Yeah, exactly. It's 100% broken. And the work we're seeing there from a
I think it's remarkable. So the last thing is regulation, because one of the things that's happened is the Internet was never regulated by anybody, really, except maybe in Europe. But in this country, absolutely not. There's not a privacy bill. There's not an antitrust bill, etc. It goes on and on. They did nothing. But the EU is considering leaving chat GBT high risk. If it happens, it will lead to significant restrictions on its use, and Microsoft and Google are lobbying against it. What do you think should happen? With AI regulation in general? This one, the high risk one.
I have followed the development of the EU's AI Act, but it is changed. It's, you know, obviously still in development. I don't know enough about the current version of it to say if I think this way, like this definition of what high risk is and this way of classifying it and this is what you have to do. I don't know if I would say that's like good or bad. I think like totally banning this stuff is not the right answer. And I think that not regulating this stuff at all. I mean, you're not TikTok, but go ahead. Yeah.
And I think not regulating this stuff at all is not the right answer either. And so the question is like, is that going to end in the right balance? Yeah.
Like, I think the EU is saying, you know, no one in Europe gets to use ChatGPT. Probably not what I would do. But the EU is saying, here's the restrictions on ChatGPT and any service like it. There's plenty of versions that I could imagine being, all right, super sensible. All right. So after the Silicon Valley non-bailout bailout, you tweeted, we need more regulation on banks. But what sort of regulation? I know. And then someone tweeted at you. Now he's going to say, we need them on AI. And you said, we need them on AI. But...
If the regulators aren't catching that, what are they doing? They did catch it, actually. They were giving warnings. They were giving warnings, but like there's often in an audit, you know, this thing is not quite like that's different than saying like... No, they were giving them pretty significant, you need to do something. They just didn't do anything. Well, they could have, I mean, the regulators could have taken over. Mm-hmm.
like six months ago. So this is what happens a lot of the time, even in well-regulated areas, which banks are compared to the internet. What sort of regulations does AI need in America? Lay them out. I know you've been meeting with regulators and lawmakers. I haven't done that many. Well, they call me when you do. They want to say they've seen you, I guess. What do they say? Well, you're like the guy now. So they like to say I was with Sam Aldman. Oh, I did one. He seems nice. I go, he's nice. I don't know what to tell you.
I did like a three-day trip to D.C. earlier this year.
It's just much more insight into what companies like ours are doing. You know, companies that are training above a certain level of capability at a minimum, like a thing that I think could happen now is the government should just have insight into the capabilities of our latest stuff released or not.
What our internal audit procedures and external audits we use look like, how we collect our data, how we're red teaming these systems, what we expect to happen, which we may be totally wrong about. We could hit a wall anytime. But like our internal roadmap documents, when we start a big training run, I think there could be government insight into that. And then if that can start now.
I do think good regulation takes a long time to develop. It's a real process. They can figure out how they want to have oversight. Reid Hoffman has suggested a blue ribbon panel so they'd learn up on this stuff. I mean, panels are fine. We could do that too. But what I mean is like government auditors sitting in our buildings. Congressman Ted Lieu said there needs to be an agency dedicated specifically to regulating AI. Is that a good idea?
I think there's two things you want to do. This is way out of my area of expertise, but you're asking, so I'll try. I think people like us that are creating content
These very powerful systems that could become something properly called AGI at some point. Explain what that is. Artificial General Intelligence. But what people mean is just like above some threshold where it's really good. Those efforts probably do need a new regulatory effort. And I think it needs to be global body. New regulatory body. And then...
people that are using AI, like we talked about the medical advisor, I think FDA can give probably very great medical regulation, but they'll have to update it for the inclusion of AI. But I would say like creation of the systems and having something like an IAEA that regulates that is one thing. And then having existing industry regulations
regulators still do their regulation. So people do react badly to that because the information bureaus, that's always been a real problem in Washington. Yeah, not everyone. Who should head that agency in the U.S.? I don't know. Okay. All right. So one of the things that's going to happen, though, is the less intelligent ones, of which there are many, are going to seize on things like they've done with
TikTok, possibly deservedly, but other things, like Snap released a chatbot powered by GPT that reportedly told a 15-year-old how to mask the smell of weed and alcohol in a 13-year-old, how to set the mood for sex with an adult. They're going to seize on this stuff. And the question is, who's liable, if this is true, when a teen uses those instructions? And Section 230 doesn't seem to cover generative AI. Is that a problem? I think we will need a new law for use of this stuff. And I think the liability will...
need to have a few different frameworks. If someone's tweaking the models themselves, I think it's going to have to be the last person that touches it has the liability. But there be liability. It's not full immunity that the platforms give. I don't think we should have full immunity. Now, that said, I understand why you want limits on it, why you do want companies to be able to experiment with this. You want users to be able to get the experience they want.
But the idea of like no one having any limits for generative AI, for AI in general, that feels super wrong. Last thing, trying to quantify the impact you personally will have on society as one of the leading developers of this technology. Do you think about that? Do you think about your impact? Like me opening up? Yeah, you, Sam. I mean, hopefully I'll have a positive impact. Do you think about the impact on humanity and the level of power that also comes with it?
Yeah, I don't. I think about like what OpenAI is going to do a lot and the impact OpenAI will have. Do you think it's out of your hands? No. No. But it is very much like the responsibility is with me at some level, but it's very much a team effort. And so when you think about the impact, what is your greatest hope and what's your greatest worry?
My greatest hope is that we create this thing. We are one of many people that is going to contribute to this movement. We'll create an AI. Other people will create an AI.
And that this, we will be a participant in this technological revolution that I believe will be far greater in terms of impact and benefit than any before. My view of the world is it's this like one big, long technological revolution, not a bunch of smaller ones, but we'll play our part. We will be one of several in this moment. And yeah.
that this is going to be really wonderful. This is going to elevate humanity in ways we still can't fully envision. And our children, our children's children are going to be far better off than the best of anyone from this time. And we're just going to be in a radically improved world. We will live healthier, more interesting, more fulfilling lives. We'll have material abundance for people. And we will be
a contributor and you know we'll put in our your part our part of that you do sound alarmingly like the people i met 25 years ago i have to say
You were not. I don't know how old you are, but you were young. You were probably very young. 37. Yeah, so you were 12. And they did talk like this. Many of them did, and some of them continued to be that way. A lot of them didn't, unfortunately. And then the greed seeped in, the money seeped in, the power seeped in, and it got a little more complex, I would say. Not totally. And again, because net, it's better.
I want to focus on you on my last question. There seem to be two caricatures of you. One that I've seen in the press is a boyish genius who will help defeat Google and usher in utopia. The other is that you're an irresponsible, woke tech overlord Icarus that will lead us to our demise. I have to pick one? Is it? No, I don't think. How old do I have to be before I can like drop the boyish qualifier? Oh, you can be boyish. Tim Hanks is still boyish.
Yeah. And what was the second one? You know, Icarus, Overlord, Tech Overlord, Woke. Woke something? Yeah. Yeah. Woke, whatever. The Icarus part is that I like. Some is still boyish, don't you think? I think we feel like adults now. You may be adults, but boyish always gets put on you. I don't ever call you boys. I think you're adults. Icarus meaning like I'm messing, we are messing around with something that we don't fully understand. Yes. Well, we are messing around with something we don't fully understand. Yeah. And we are trying to do our part in contributing to the responsible path through it.
All right. But I don't think either of those characters. So describe yourself then. Describe what you are. Technology brother. Oh, wow. You're going to go for techno. I'm kidding. I just think that's such a funny meme. I don't know how to describe myself. I think that's what you would call me. No, I wouldn't. No? 100%. All right. Because it's an insult now. It's become an insult.
I'd call you a technology sister. I'll take that. We leave it on that note? Let's leave it on that note. I do have one more quick question. Last time we talked, you were thinking of running for governor. I was thinking of running for mayor. I'm not going to be running for mayor. Are you going to still run for governor? No. No. I think I am doing the most amazing thing I can imagine. I really don't want to do anything else. You don't want to do anything else. It's tiring, but I love it. Yeah. Okay. Sam Altman, thank you so much. Thank you. It's on!
You said he sounded a lot like a lot of founders a generation before him. Yes. What are the lessons you would impart to Sam as someone who has so much impact on humanity? You know, I think what I said is that they were hopeful and they had great ideas. And one of the things that I think
people get wrong is to be a tech critic means you love tech. Like, you know, you really love it. You do. Yeah, of course. And you don't want it to fail. You want it to create betterment for humanity. And if that's your goal, when you see it being warped and misused, it's really sad and disappointing. And I think one of the things early Internet people had all these amazing ideas, the world talking to each other, we'll get along with Russia, we'll be able to communicate over vast distances. And again, just like I talked about with Rehabilitation,
Reid Hoffman, it's a Star Trek vision of the universe. And that's what it was. And boy, the money and the power and the bad people that came in were really significantly shifted it. Not completely by any means. I love my Netflix. You know, I just do. But the unintended or intended consequences changed.
ultimately are very hard to bear, even if it's a net positive. So it's just the money and the power that's corrupting is what you're saying. It's inevitable? No, not inevitable. But often, often. Often, yeah. Well, not him. Not a lot of people. But...
Let's see this standing the test of time, right? You're saying about Reid Hoffman and Max Levchin versus, say, Peter Thiel and Elon Musk. Well, I think Peter was always like that, you know? I don't think he's changed one bit. And so, or not in my estimation, he's been very consistent in how he looks at the world, which is not a particularly positive light.
I think that a lot of them do stay the same and they do stay true to what they're like. And I don't know why that is over certain people and others get sucked into it in a way that's really – I'm thinking about this a lot because that's what my book's about. Yeah, of course.
how people change and why and whether that's a good thing or a bad thing because, you know, one of the things about tech is constantly changing. One of the poems I'm using in the book is a poem by Maggie Smith called Good Bones. And I'll just read you the last part. Life is short and the world is at least half terrible for in every kind, it's
stranger, there is one who would break you, though I keep this from my children. I'm trying to sell them the world. Any decent realtor walking through a real shithole chirps on about good bones. This place could be beautiful, right? You could make this place beautiful. And that's how I feel about this. They could make this place beautiful. And I think Sam thinks that too. Yeah. It's not just a lie you tell your children, right? Well, no. But it is. You can't tell them terrible things all the time. They would be like just lying on the ground and
Yeah, but sometimes it's so idealistic. Like when he said global regulatory body to regulate AI, I'm like, oh man, we're fucked. That's never going to happen. Like when was the last good global regulatory body? I know, but it could work. It could work. There has to be, this has to be global. This has to be global. But there's no infrastructure to set up a system.
Yes, there is. In medicine, there is. What, you think the World Health Organization has been effective in medicine? No, I think there's stuff around cloning, around all kinds of stuff. It's never going to be perfect, but boy, there's a lot of people that hew to those ethics. I mean, I think it depends how bought in state governments are, including China, but...
The regulation thing is particularly tricky because it can also become a moat, right, for incumbents. Like Facebook's like, regulate them. It's like, well, you can afford the regulation in a way that new competitors maybe can't. I think the governments can play a lot of roles here. They do it in nuclear nonproliferation. It's never perfect, but we still haven't set one off, have we? I think that's largely the deterrent power and not because of any effective regulation. I am a great believer in nuclear nonproliferation. Yeah, I do too. I think there's lots of examples of it.
work. And I think the most significant thing that he said here was about the government's role, the U.S. government's role. It shouldn't give this all over to the private sector. It should have been the one to give them money and to fund them. And that is 100%. We've talked to Mariana Mazzucato about that. Yeah. And many people. That, to me, is the big shame is the government abrogating its role in really important issues.
things that are important globally and important for the U.S. But even when the government has played that kind of like, let's call it kindling role for industry, whether it be Elon Musk's loan for Tesla, whether it be what DARPA was doing that became, you know, parts of Siri and Echo and whatnot, the government is
here is bad at retaining like a windfall from that that would be reinvested into taxpayers. But it didn't used to. It used to just do it because it was the right thing to do, that we would research and investment by the government. You know, highway system seems to have worked out pretty good. The telephone system seems good. You know, I mean, we always tend to like talk about what they do wrong, but there's so much stuff that the government can't
contributed to that matters today. It used to be a culture also if people would want to go into government and civil service. My father was in that generation. And I think that it's interesting to hear Sam say, no, he won't run for governor. And in fact, you think sometimes, well, it would be so great if some of these bright minds went into... Except he's more effective where he is. Why would he do that when he's more effective where he is?
arguably the right regulator for this as a person who could have built it. Yeah. Or conceived building it. Maybe. Did you find his answers to the moderation questions and this idea of hallucination and overly impressive at first glance, did you find those satisfying? Yeah. I thought, he doesn't have answers. I think one of the things I like about Sam is if he doesn't have an answer, I don't think he's hiding it. I don't think he knows. And I think one of the strengths of certain entrepreneurs is they're
I don't really know. And I think a lot around AI right now, anyone that's going to give you a certainty is lying to you. Well, they had experimented with using these, you know, these low wage workers in Africa through SAMA and outsource. Well, I think it was that it was exposed. They were paying them less than $2 an hour and training them to build up like...
But what was reported a content moderation AI layer, which is ironic when you think about it. So there were workers in Africa being paid less than $2 an hour to train machines to replace them for that job. Well, have you been to an Amazon warehouse lately? There's a lot of machines doing everything. That's the way it's going. It doesn't, that's like...
you're telling me something that happens in every other industry. Yeah, I know. And yet we're going to grow smarter. Do you think that's true? AI tutors, everyone's going to be smarter? I do. I think we do a lot of rote, idiotic work that we shouldn't be doing. And we have to be more creative of what the greatest use of our time is. My great hope for AI is actually that it takes out the rote bits and all of a sudden creative industry flourishes because those are the parts that can't be replicated. And though I think a sad reality of technology in the last generation has been that kids
maybe don't read as well or as much or as fast or as early as used to, but they make video. Right. What if they're spoken to smarter? Like the idea of education on these things or information or health care in an easy way is really, these phones are just getting started and they will not just be phones. They will be wrapped around us in more good information you get and the more communication you get. That's a good thing.
They might just be getting started, but we are ending. Do you want to read us our credits today? Yes. Remember, you can make this place beautiful or ugly. Got good bones. Got good bones. Got good bones.
Today's show was produced by Neha Miraza, Blake Nishik, Kristen Castro-Rossell, and Rafaela Seward. Special thanks to Hayley Milliken. Our engineers are Fernando Arrudo and Rick Kwan. Our theme music is by Trackademics. If you're already following the show, you get the red pill. If not, Rick Deckard is coming after you. Go wherever you listen to podcasts, search for On with Kara Swisher and hit follow. Thanks for listening to On with Kara Swisher from New York Magazine, the Vox Media Podcast Network, and us.
We'll be back on Friday, that's tomorrow, with a special bonus episode.
