Eliezer YudkowskyEvery podcast appearance, updated as new ones drop

Malcolm recounts a heated debate with AI theorist Eliezer Yudkowsky on AI safety. He explains his belief that subsystems in an advanced AI would converge on the same utility function, while Yudkowsky insists no AI would subdivide that way. Simone notes Yudkowsky's surprising lack of knowledge in physics and neuroscience given his confidence. They express concern his ideas ruin youth's outlooks and discuss hypothetical clapbacks. Overall they conclude that while well-intended, Yudkowsky's certainty without humility on AI risks is dangerous. Simone: [00:00:00] What's Malcolm: really interesting is that he actually conceded that if this was the way that an AI structured itself, that yes, you would have terminal convergence, but that AIs above a certain level of intelligence would never structure themselves this way. Malcolm: So this was very interesting to me because it wasn't the argument I thought he would take. And that would be true. I, I will agree that if the AI maintained itself as a single hierarchy, it would be much less likely for its utility function to change. But the problem is... Is essentially no government structure ever created and has functioned that way. Malcolm: Essentially no program ever created by humans has run that way. Nothing ever encoded by evolution has run that way. i. e. the human brain, any brain, any neural structure we know of. There are none that are coded that way. Malcolm: So it is very surprising. So I said, okay, gauntlet thrown. Are you willing to be disproven? Malcolm: , because we will get some more understanding into AI interpretability, into how AIs. Think in the near future. If it turns out [00:01:00] that the AI's that exist right now are actually structuring themselves that way, will you concede that you are wrong about the way that you tackle AI apocalypticism? Malcolm: And then he said, and this is really interesting to me. He's like, no, I won't Malcolm: I was also like, yeah, also, we could run experiments where we do a bunch of basically unbounded A. I. S. and see if they start to show terminal convergence. Malcolm: Do they start to converge on similar utility functions? You know what they're trying to optimize for again? He was like, well, even if we saw that, that wouldn't change my views on anything, right? Like his views are religious in nature, which was very disappointing to me. Like, I thought that maybe he had more of like a logical or rational perspective on things. Malcolm: That. And it was, it was really sad. Malcolm: You know, we don't talk negatively about people on this channel very frequently, but I do think that he destroys a lot of people's lives. And I do think that he makes the risk of AI killing all humans dramatically higher than it would be in a world where he didn't exist. Would you like to know more?[00:02:00] Simone: Hello, Malcolm. Malcolm: Hello. So we just got back from this wonderful conference thing we were at called Manifest. So we had gone out to SF to host a few pronatalist focused dinner parties and randomly we got looped in to something called Manifest, which was a conference for people who are interested in prediction markets. Malcolm: But interestingly, we ended up meeting a bunch of people who we had known through like online stuff. Some were absolutely fantastic, like, Scott Alexander, absolutely. I never met him before in person. We'd communicated on a few issues, really cool guy. Would you say so Simone? Simone: Yeah. Like super awesome. Malcolm: Richard Hedania, a really nice guy as well. Robin Hanson, who we, we, we'd actually met him before. But. And o

Artificial Intelligence (AI) researcher Eliezer Yudkowsky makes the case for why we should view AI as an existential threat to humanity. Rep. Crenshaw gets into the basics of AI and how the new AI program, GPT-4, is a revolutionary leap forward in the tech. Eliezer hypothesizes the most likely scenarios if AI becomes self-aware and unconstrained – from rogue programs that blackmail targets to self-replicating nano robots. They discuss building global coalitions to rein in AI development and how China views AI. And they explore first steps Congress could take to limit AI's capabilities for harm while still enabling its promising advances in research and development. Eliezer Yudkowsky is a co-founder and research fellow at the Machine Intelligence Research Institute, a private research nonprofit based in Berkeley, California. Follow him on Twitter @ESYudkowsky

Eliezer Yudkowsky is a researcher, writer, and advocate for artificial intelligence safety. He is best known for his writings on rationality, cognitive biases, and the development of superintelligence. Yudkowsky has written extensively on the topic of AI safety and has advocated for the development of AI systems that are aligned with human values and interests. Yudkowsky is the co-founder of the Machine Intelligence Research Institute (MIRI), a non-profit organization dedicated to researching the development of safe and beneficial artificial intelligence. He is also a co-founder of the Center for Applied Rationality (CFAR), a non-profit organization focused on teaching rational thinking skills. He is also a frequent author at LessWrong.com as well as Rationality: From AI to Zombies. In this episode, we discuss Eliezer’s concerns with artificial intelligence and his recent conclusion that it will inevitably lead to our demise. He’s a brilliant mind, an interesting person, and genuinely believes all of the stuff he says. So I wanted to have a conversation with him to hear where he is coming from, how he got there, understand AI better, and hopefully help us bridge the divide between the people who think we’re headed off a cliff and the people who think it’s not a big deal. (0:00) Intro (1:18) Welcome Eliezer (6:27) How would you define artificial intelligence? (15:50) What is the purpose of a firm alarm? (19:29) Eliezer’s background (29:28) The Singularity Institute for Artificial Intelligence (33:38) Maybe AI doesn’t end up automatically doing the right thing (45:42) AI Safety Conference (51:15) Disaster Monkeys (1:02:15) Fast takeoff (1:10:29) Loss function (1:15:48) Protein folding (1:24:55) The deadly stuff (1:46:41) Why is it inevitable? (1:54:27) Can’t we let tech develop AI and then fix the problems? (2:02:56) What were the big jumps between GPT3 and GPT4? (2:07:15) “The trajectory of AI is inevitable” (2:28:05) Elon Musk and OpenAI (2:37:41) Sam Altman Interview (2:50:38) The most optimistic path to us surviving (3:04:46) Why would anything super intelligent pursue ending humanity? (3:14:08) What role do VCs play in this? Show Notes: https://twitter.com/liron/status/1647443778524037121?s=20 https://futureoflife.org/event/ai-safety-conference-in-puerto-rico/ https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy https://www.youtube.com/watch?v=q9Figerh89g https://www.vox.com/the-highlight/23447596/artificial-intelligence-agi-openai-gpt3-existential-risk-human-extinction Eliezer Yudkowsky – AI Alignment: Why It's Hard, and Where to Start Mixed and edited: Justin Hrabovsky Produced: Rashad Assir Executive Producer: Josh Machiz Music: Griff Lawson 🎙 Listen to the show Apple Podcasts: https://podcasts.apple.com/us/podcast/three-cartoon-avatars/id1606770839 Spotify: https://open.spotify.com/show/5WqBqDb4br3LlyVrdqOYYb?si=3076e6c1b5c94d63&nd=1 Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zaW1wbGVjYXN0LmNvbS9zb0hJZkhWbg 🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1 Follow on Socials 📸 Instagram - https://www.instagram.com/theloganbartlettshow 🐦 Twitter - https://twitter.com/loganbartshow 🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow About the Show Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-

Eliezer Yudkowsky is a researcher, writer, and philosopher on the topic of superintelligent AI. Please support this podcast by checking out our sponsors: – Linode : https://linode.com/lex to get $100 free credit – House of Macadamias : https://houseofmacadamias.com/lex and use code LEX to get 20% off your first order – InsideTracker : https://insidetracker.com/lex to get 20% off EPISODE LINKS: Eliezer’s Twitter: https://twitter.com/ESYudkowsky LessWrong Blog: https://lesswrong.com Eliezer’s Blog page: https://www.lesswrong.com/users/eliezer_yudkowsky Books and resources mentioned: 1. AGI Ruin (blog post): https://lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities 2. Adaptation and Natural Selection: https://amzn.to/40F5gfa PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: – Check out the sponsors above, it’s the best way to support this podcast – Support on Patreon: https://www.patreon.com/lexfridman – Twitter: https://twitter.com/lexfridman – Instagram: https://www.instagram.com/lexfridman – LinkedIn: https://www.linkedin.com/in/lexfridman – Facebook: https://www.facebook.com/lexfridman – Medium: https://medium.com/@lexfridman OUTLINE: Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time. (00:00) – Introduction (05:19) – GPT-4 (28:00) – Open sourcing GPT-4 (44:18) – Defining AGI (52:14) – AGI alignment (1:35:06) – How AGI may kill us (2:27:27) – Superintelligence (2:34:39) – Evolution (2:41:09) – Consciousness<

Eliezer Yudkowsky, expert reconnu des risques liés à l'IA: "We’re All Gonna Die" ( entretien de près de 2 heures sur Youtube , transcript here ) Eliezer Yudkowsky est sans doute la figure la plus connue et respectée depuis 20 ans dans le milieu de la recherche sur les façons d'aligner l'IA sur nos valeurs humaines Wikipedia : Eliezer Yudkowsky is an American decision theory and artificial intelligence (AI) researcher and writer. He is a co-founder and research fellow at the Machine Intelligence Research Institute (MIRI), a private research nonprofit based in Berkeley, California. His work on the prospect of a runaway intelligence explosion was an influence on Nick Bostrom's Superintelligence: Paths, Dangers, Strategies. Yudkowsky's views on the safety challenges posed by future generations of AI systems are discussed in the undergraduate textbook in AI, Stuart Russell and Peter Norvig's Artificial Intelligence: A Modern Approach. Eliezer Yudkowsky a livré il y a quelques semaines un entretien de près de 2 heures durant lesquelles il a partagé et expliqué sa conviction profonde: "nous allons tous mourir des mains d'une super intelligence artificielle", se montrant plus résigné que jamais, mais disant malgré tout vouloir "fight until the end with dignity" Eliezer Yudkowsky s'est dit d'abord surpris par le rythme des progrès en IA ces dernières années, à ses yeux, il est très probable qu'on parvienne à développer une super IA, plus capable que tous les êtres humains réunis à un moment ou un autre ce siècle-ci ("3 ans, 15 ans, plus ? Difficile de savoir...") Ses travaux depuis 20 ans l'on conduit à un constat sans appel : nous ne savons pas comment programmer une super IA pour être certain qu'elle ne nous nuise pas, et ne sommes pas près de le faire, c'est une tâche éminemment compliquée et sans doute impossible, qui demanderait qu'on y consacre des ressources extraordinaires, et il est trop tard pour cela C'est tout l'inverse qui se passe selon lui, les meilleurs labos en IA foncent tête baissée, leurs précautions sont bien insuffisantes et principalement de façade L'actualité semble lui donner raison : "Microsoft got rid of its entire company division devoted to AI "ethics and society" during its January layoffs" "Most of their 30-person staff was reassigned way back in October, leaving just seven employees to manage the department." "Months later, though, they were all dismissed, along with the division — right as the company announced its mammoth $10 billion investment in OpenAI." ( source ) à noter toute fois que Sam Altman, CEO d'OpenAI, écrivait récemment : "Some people in the AI field think the risks of AGI (and successor systems) are fictitious; we would be delighted if they turn out to be right, but we are going to operate as if these risks are existential.", linking to t hat article AI Could Defeat All Of Us Combined Eliezer Yudkowsky a eu un espoir en 2015 quand il a participé à la grande conférence sur les risques de l'IA organisée par Elon Musk, rassemblant des experts du sujet comme Stuart Russel, Demis Hassabis (co-fondateur de DeepMind en 2014), Ilya Sutskever et bien d'autres. Mais il a très vite déchanté, la conférence a accouché du pire des résultats à ses yeux : la création d'OpenAI peu après (par Ilya Sutskever, Sam Altman, Elon Musk, Peter Thiel et d'autres) Au lieu de freiner le développemen

Eliezer Yudkowsky is an author, founder, and leading thinker in the AI space. ------ ✨ DEBRIEF | Unpacking the episode: https://shows.banklesshq.com/p/debrief-eliezer ------ ✨ COLLECTIBLES | Collect this episode: https://collectibles.bankless.com/mint ------ We wanted to do an episode on AI… and we went deep down the rabbit hole. As we went down, we discussed ChatGPT and the new generation of AI, digital superintelligence, the end of humanity, and if there’s anything we can do to survive. This conversation with Eliezer Yudkowsky sent us into an existential crisis, with the primary claim that we are on the cusp of developing AI that will destroy humanity. Be warned before diving into this episode, dear listener. Once you dive in, there’s no going back. ------ 📣 MetaMask Learn | Learn Web3 with the Leading Web3 Wallet https://bankless.cc/ ------ 🚀 JOIN BANKLESS PREMIUM: https://newsletter.banklesshq.com/subscribe ------ BANKLESS SPONSOR TOOLS: 🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE https://bankless.cc/kraken 🦄UNISWAP | ON-CHAIN MARKETPLACE https://bankless.cc/uniswap ⚖️ ARBITRUM | SCALING ETHEREUM https://bankless.cc/Arbitrum 👻 PHANTOM | #1 SOLANA WALLET https://bankless.cc/phantom-waitlist ------ Topics Covered 0:00 Intro 10:00 ChatGPT 16:30 AGI 21:00 More Efficient than You 24:45 Modeling Intelligence 32:50 AI Alignment 36:55 Benevolent AI 46:00 AI Goals 49:10 Consensus 55:45 God Mode and Aliens 1:03:15 Good Outcomes 1:08:00 Ryan’s Childhood Questions 1:18:00 Orders of Magnitude 1:23:15 Trying to Resist 1:30:45 Miri and Education 1:34:00 How Long Do We Have? 1:38:15 Bearish Hope 1:43:50 The End Goal ------ Resources: Eliezer Yudkowsky https://twitter.com/ESYudkowsky MIRI https://intelligence.org/ Reply to Francois Chollet https://intelligence.org/2017/12/06/chollet/ Grabby Aliens https://grabbyaliens.com/ ----- Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research. Disclo

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Discussion with Eliezer Yudkowsky on AGI interventions, published by Rob Bensinger, Eliezer Yudkowsky on the AI Alignment Forum. The following is a partially redacted and lightly edited transcript of a chat conversation about AGI between Eliezer Yudkowsky and a set of invitees in early September 2021. By default, all other participants are anonymized as "Anonymous". I think this Nate Soares quote (excerpted from Nate's response to a report by Joe Carlsmith) is a useful context-setting preface regarding timelines, which weren't discussed as much in the transcript: [...] My odds [of AGI by the year 2070] are around 85%[...] I can list a handful of things that drive my probability of AGI-in-the-next-49-years above 80%: 1. 50 years ago was 1970. The gap between AI systems then and AI systems now seems pretty plausibly greater than the remaining gap, even before accounting the recent dramatic increase in the rate of progress, and potential future increases in rate-of-progress as it starts to feel within-grasp. 2. I observe that, 15 years ago, everyone was saying AGI is far off because of what it couldn't do -- basic image recognition, go, starcraft, winograd schemas, programmer assistance. But basically all that has fallen. The gap between us and AGI is made mostly of intangibles. (Computer Programming That Is Actually Good? Theorem proving? Sure, but on my model, "good" versions of those are a hair's breadth away from full AGI already. And the fact that I need to clarify that "bad" versions don't count, speaks to my point that the only barriers people can name right now are intangibles.) That's a very uncomfortable place to be! 3. When I look at the history of invention, and the various anecdotes about the Wright brothers and Enrico Fermi, I get an impression that, when a technology is pretty close, the world looks a lot like how our world looks. Of course, the trick is that when a technology is a little far, the world might also look pretty similar! Though when a technology is very far, the world does look different -- it looks like experts pointing to specific technical hurdles. We exited that regime a few years ago. 4. Summarizing the above two points, I suspect that I'm in more-or-less the "penultimate epistemic state" on AGI timelines: I don't know of a project that seems like they're right on the brink; that would put me in the "final epistemic state" of thinking AGI is imminent. But I'm in the second-to-last epistemic state, where I wouldn't feel all that shocked to learn that some group has reached the brink. Maybe I won't get that call for 10 years! Or 20! But it could also be 2, and I wouldn't get to be indignant with reality. I wouldn't get to say "but all the following things should have happened first, before I made that observation". I have made those observations. 5. It seems to me that the Cotra-style compute-based model provides pretty conservative estimates. For one thing, I don't expect to need human-level compute to get human-level intelligence, and for another I think there's a decent chance that insight and innovation have a big role to play, especially on 50 year timescales. 6. There has been a lot of AI progress recently. When I tried to adjust my beliefs so that I was positively surprised by AI progress just about as often as I was negatively surprised by AI progress, I ended up expecting a bunch of rapid progress. [...] Further preface by Eliezer: In some sections here, I sound gloomy about the probability that coordination between AGI groups succeeds in saving the world. Andrew Critch reminds me to point out that gloominess like this can be a self-fulfilling prophecy - if people think successful coordination is impossible, they won’t try to coordinate. I therefore remark in retrospective

For our final and ultimate episode, we are joined by Eneasz Brodski, Matt Freeman, Eliezer Yudkowsky! We had a great time producing this show for the last year and the best way to cap it off was an awesome conversation with all of the people who made it possible. Thank you to all of our guests and to all of you for listening. Original chapters, written by Eliezer Yudkowsky, can be read here and the audiobook chapters, recorded by Eneasz Brodski, can be found earlier in this podcast feed and on the website . Album art courtesy of Lorec. Thank you! Coy manages an RSS feed that compiles the relevant audiobook chapters with the WW MoR counterparts. Just copy and paste that link into your favorite podcast app in the “add by url” option. Thanks, Coy!

We welcomed the one , the only , Eliezer Yudkowsky to the Bayesian Conspiracy for a quick chat. His latest project is Arbital , an ambitious effort to solve online discussion . It is focusing on solving online explanation. They want to do for difficult explanations – and someday, complicated arguments in general – what Wikipedia did for centralizing humanity’s recounting of agreed-on facts. The initial demo page is an explanation of Bayes’s Rule About Eliezer in his (heavily truncated by me) own words: One Eliezer Yudkowsky writes about the fine art of human rationality . That Eliezer Yudkowsky’s work can be found in the Rationality section. The other Eliezer Yudkowsky concerns himself with Artificial Intelligence . Very shortly – on a historical scale, that is – we can expect technology to break the upper bound on intelligence that has held for the last few tens of thousands of years. For more on this see the Singularity tab. My short fiction, miscellaneous essays, and various other things can be found under Other . I am a full-time Research Fellow at the Machine Intelligence Research Institute , a small 501(c)(3) public charity supported primarily by individual donations. A majority of all my written material is currently stored at the community blog Less Wrong . We asked him about questions ranging from his opinions on AI to rationality to his popular Harry Potter Fan Fiction . Eliezer brought up a few concepts, and you will definitely want these links : He spoke about potential types of AI, including Sovereign and Genie . Read the full Fun Theory Sequence , which is summarized in the 31 Laws of Fun . Eliezer wants you to see this spifftastic Guide to Bayes Rule . We will be reposting that last link, I’m sure. As always, you can email us at BayesianConspiracyPodcast@gmail.com with your valued feedback, or comment at the subreddit . Other Relevant Links Surely You’re Joking, Mr Feynman! A Step Farther Out, by Jerry Pournelle Death Note The first Miles Vorkosigan book

This is the third episode of a new podcast I’m involved in – The Bayesian Conspiracy . www.thebayesianconspiracy.com

Forgive me Father – it’s been at least two weeks since my last podcast. I figured you guys needed some time to digest my last run of shows. Ready for more yet? Another show on the coming of the technological singularity today. My guest is Eliezer Yudkowsky , co-founder and research fellow at the Singularity Institute for Artificial Intelligence in California. I have been aware of Eli for ten years or more. He was featured fairly prominently in Damien Broderick’s book The Spike and was a contributor to Natasha Vita-More’s Extropian mailing list in the mid to late Nineties. An autodidact prodigy, Eliezer wrote the first version of his Singularity “call to arms”, Staring Into The Singularity , at the age of 15 and has been re-working it continually since then. On this episode I’ve tried to capture Eliezer’s vision for the different forms that the Singularity might take, the timelines for it, and his motivations for trying to make it happen as soon as possible. I hope you enjoy it. If you want to hear more interviews about The Singularity or AI, try these previous episodes of G’Day World: Dr Aubrey de Grey Ray Kurzweil Roger Williams Dr Ben Goertzel The post G’DAY WORLD #238 – Eliezer Yudkowsky appeared first on cameronreilly.com .