![[21-Geoffrey_Litt.jpg]] *Dialectic Episode 21: Geoffrey Litt - Software You Can Shape - is available on [Spotify](https://open.spotify.com/episode/5B8B6At0i7RWpiToYwFFas?si=cKtu_-6LR_ysBfFfj4yXWQ), [Apple Podcasts](https://podcasts.apple.com/us/podcast/21-geoffrey-litt-software-you-can-shape/id1780282402?i=1000713518703), and [YouTube](https://youtu.be/RromJIXfYyI?si=eA7RVa3wBNdGi5Es), and all podcast platforms.* <iframe style="border-radius:12px" src="https://open.spotify.com/embed/episode/5B8B6At0i7RWpiToYwFFas?utm_source=generator" width="100%" height="152" frameBorder="0" allowfullscreen="" allow="autoplay; clipboard-write; encrypted-media; fullscreen; picture-in-picture" loading="lazy"></iframe> <iframe allow="autoplay *; encrypted-media *; fullscreen *; clipboard-write" frameborder="0" height="175" style="width:100%;max-width:660px;overflow:hidden;border-radius:10px;" sandbox="allow-forms allow-popups allow-same-origin allow-scripts allow-storage-access-by-user-activation allow-top-navigation-by-user-activation" src="https://embed.podcasts.apple.com/us/podcast/21-geoffrey-litt-software-you-can-shape/id1780282402?i=1000713518703"></iframe> <iframe width="560" height="315" src="https://www.youtube.com/embed/RromJIXfYyI?si=3nLPadAw4lAPEmtX" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> # Description Geoffrey Litt ([Website](https://geoffreylitt.com/), [X](https://x.com/geoffreylitt)) is a designer, engineer, writer, and researcher at [Ink & Switch](https://www.inkandswitch.com/), where he champions *malleable software*: the idea that ordinary people should be able to mold the digital tools they rely on every day. Ink & Switch is an independent research lab focused on how computers can help us think and work. While researching and writing, Geoffrey and team also build products and prototypes to explore how their ideas can exist in practice. Geoffrey got his PhD at MIT CSAIL, where he built on his inspiration around computational media like spreadsheets, hoping to push more software toward the ethos of end-user programming, but without the technical complexity. In a sense, why should using software and changing it be any different? Previously, he built software for teachers at [Panorama Education](https://www.panoramaed.com/), which he joined out of school as one of the first employees. [Geoffrey and collaborators recently published a definitive piece on malleable software and we discussed it in detail](https://www.inkandswitch.com/essay/malleable-software/). We dig into why most modern apps feel like sealed boxes rather than flexible tools and environments, and what changes when your app, document, or workspace, feels more like Lego than machinery. Geoffrey makes his case that we want software tooling to[ feel like a chef knife, not an avocado slicer](https://x.com/geoffreylitt/status/1932516047665471895), and we talk about how the best designed tools help users up a smooth slope of learning and ability. He argues in favor of deeper understanding, illustrated by one of my favorite ideas: [The Nightmare Bicycle](https://www.geoffreylitt.com/2025/03/03/the-nightmare-bicycle). We talk about how LLMs are enabling malleable software and how local tinkerers might be able to build systems for themselves and their team or communities that understand their needs more deeply than any professional designer could. Finally, Geoffrey lays out a call to arms for founders: build products that treat users as co-authors who understand their own needs, not just consumers. On one level, this is a conversation about software and design. But it is really about agency. I hope it inspires you to pop open the hood on various aspects of your life, look at what's inside, and trust yourself to tinker. As Steve Jobs said many years ago, "the minute you can understand that you can poke life, and if you push in, then something will pop out the other side; that you can change it, you can mold it—that's maybe the most important thing." --- **This episode is brought to you by [Hampton](https://joinhampton.com/community)**, a private, highly vetted membership for founders. Hampton surveyed over 100 members with net worths of $1M-100M to create its **2024 Wealth Report.** They asked about financial goals, spending habits, how much founders themselves, investment portfolio breakdowns, risk tolerance, estate planning and philanthropy, and more. Visit https://joinhampton.com/community to access the report. # Timestamps - 2:12: Agency in a Digital World and Geoffrey's Creative Medium: Software - 12:17: Intro to Malleable Software - 20:42: "Popping Open the Hood" & The Nightmare Bicycle: A Case for Understanding How Systems Work - 27:47: Computational Media, Spreadsheets, and Digital Informality - 34:01: Legos and Home Cooking as Metaphors for Software - 42:30: Two Types of Malleable Software: Modular-by-Design and Hacking - 48:35: Hampton - 50:13: Designing for a Smooth Slope - 58:20: Unbundling Apps into Environments and Tools - 1:17:58: Why Do the Work at All When AI Can Do It? When Should We be in the Details? - 1:29:22: Empathy & Design: Enabling "Local Developers" Who Know Their and Their Community's Needs - 1:38:23: A Case for Optimism About Human Agency - 1:51:09: AI's Impact on Malleable Software - 1:59:03: Commercial Incentives and Ecosystem Change - 2:04:17: Research and Ink & Switch - 2:11:46: ChatGPT as a Muse - 2:15:34: Working at MUBI and Solving the "Too Many Things to Watch" Problem - 2:18:27: Japan's Culture of Care - 2:22:15: Mastery and Variety - 2:24:34: Joy and Clarity as a Parent - 2:25:30: Expressing Care Through What we Make # Links & References - [Malleable software: Restoring user agency in a world of locked-down apps - Geoffrey Litt, Josh Horowitz, Peter van Hardenberg, Todd Matthews](https://www.inkandswitch.com/essay/malleable-software/) - [How Buildings Learn - Stewart Brand](https://www.goodreads.com/book/show/38310.How_Buildings_Learn) - [Stewart Brand, working in a shipping container - Austin Kleon](https://tumblr.austinkleon.com/post/587667749) - [Douglas Engelbart](https://en.wikipedia.org/wiki/Douglas_Engelbart) - [Christopher Alexander](https://en.wikipedia.org/wiki/Christopher_Alexander) - [Patchwork: Version control for everything (Ink & Switch)](https://www.inkandswitch.com/patchwork/notebook/) - [12. Che-Wei Wang & Taylor Levy (CW&T) - Iterating Together with Time - Dialectic](https://jacksondahl.com/dialectic/cwandt) - [Apple Notes bell curve meme](https://i.redd.it/84un9k2ifdaa1.jpg) - [Obsidian](https://obsidian.md/) - [Don't delegate understanding - Steph Ango](https://stephango.com/understand) - [Opening the Hood of a Word Processor - Alan Kay](https://worrydream.com/refs/Kay_1984_-_Opening_the_Hood_of_a_Word_Processor.pdf) - [Avoid the nightmare bicycle - Geoffrey Litt](https://www.geoffreylitt.com/2025/03/03/the-nightmare-bicycle) - [Changing Minds - Andrea diSessa](https://www.goodreads.com/book/show/1752380.Changing_Minds) - [Seymour Papert](https://en.wikipedia.org/wiki/Seymour_Papert) - [Webstrates](https://webstrates.net/) - [Geoffrey's PHD Thesis: Building Personal Software with Reactive Databases - MIT](https://groups.csail.mit.edu/sdg/pubs/2023/litt_phd_thesis.pdf) - [Embark: Dynamic documents for making plans (Ink & Switch)](https://www.inkandswitch.com/embark/) - [Foam: Software as Curation - Geoffrey Litt](https://www.geoffreylitt.com/2020/07/19/tools-over-apps-for-personal-notetaking) - [An app can be a home-cooked meal - Robin Sloan](https://www.robinsloan.com/notes/home-cooked-app/) - [Your pie doesn't need to be original (unless you claim it so) - Geoffrey Litt](https://www.geoffreylitt.com/2024/08/25/your-pie-doesnt-need-be-original) - [Browser extensions are underrated: the promise of hackable software - Geoffrey Litt](https://www.geoffreylitt.com/2019/07/29/browser-extensions) - [8. Steph Ango - Tools for Amplifying Our Light - Dialecticl](https://jacksondahl.com/dialectic/steph-ango) - [File over app - Steph Ango](https://stephango.com/file-over-app) - [C. Thi Nguyen](https://objectionable.net/) - [Malleable software in the age of LLMs - Geoffrey Litt](https://www.geoffreylitt.com/2023/03/25/llm-end-user-programming.html) - [Gabriel Leydon](https://x.com/gabrielleydon) - [Apps are avocado slicers](https://x.com/geoffreylitt/status/1932516047665471895) - [Project Cambria: Translate your data with lenses (Ink & Switch)](https://www.inkandswitch.com/cambria/) - [tldraw](https://www.tldraw.com/) - [Dynamicland](https://dynamicland.org/) - [Bret Victor](https://worrydream.com/) - [Josh Horowitz](https://joshuahhh.com/) - [Bring Your Own Client - Geoffrey Litt](https://www.geoffreylitt.com/2021/03/05/bring-your-own-client) - [Triggers and Barriers to Customizing Software - Wendy Mackay](https://www.lri.fr/~mackay/pdffiles/CHI91.Triggers.pdf) - [Reality has a surprising amount of detail - John Salvatier](http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail) - [Put-that-there - Chris Schmandt, Richard A. Bolt - MIT Media Lab](https://www.media.mit.edu/publications/put-that-there-voice-and-gesture-at-the-graphics-interface/) - [A Small Matter of Programming - Bonnie A. Nardi](https://www.goodreads.com/book/show/581700.A_Small_Matter_of_Programming) - [Amusing Ourselves to Death - Neil Postman](https://www.goodreads.com/book/show/74034.Amusing_Ourselves_to_Death) - [The Recurse Center](https://www.recurse.com/) - [AI-generated tools can make programming more fun - Geoffrey Litt](https://www.geoffreylitt.com/2024/12/22/making-programming-more-fun-with-an-ai-generated-debugger) - [Stevens: a hackable AI assistant using a single SQLite table and a handful of cron jobs - Geoffrey Litt](https://www.geoffreylitt.com/2025/04/12/how-i-made-a-useful-ai-assistant-with-one-sqlite-table-and-a-handful-of-cron-jobs) - [I fired myself - Bryan Johnson](https://medium.com/future-literacy/i-fired-myself-d74e2228e873) - [ChatGPT as muse, not oracle - Geoffrey Litt](https://www.geoffreylitt.com/2023/02/26/llm-as-muse-not-oracle) - [On the "hallucination problem" - Andrej Karpathy](https://x.com/karpathy/status/1733299213503787018) - [Gordon Brander](https://gordonbrander.com/) - [Brian Eno-Oblique Strategies](https://enoshop.co.uk/products/oblique-strategies?variant=51221629501780) - [Is Mubi Worth $1 Billion? How the Streamer Is Taking on A24 and Neon - Variety](https://variety.com/2025/film/features/mubi-worth-billion-streamer-taking-on-a24-1236395959/) - [What a School Performance Shows Us About Japanese Education - The New York Times](https://www.nytimes.com/2024/11/18/opinion/japan-education-childhood.html) - [13. Nabeel S. Qureshi - The Will to Care - Dialectic](https://jacksondahl.com/dialectic/nabeel-qureshi) Dialectic with Jackson Dahl is available on all podcast platforms. [Join the ⁠telegram channel for Dialectic⁠](https://t.me/dialecticpod) [Follow ⁠Dialectic on Twitter⁠](https://x.com/dialecticpod) [Follow Dialectic on Instagram](https://www.instagram.com/dialecticpod/) [Subscribe to Dialectic on Spotify](https://open.spotify.com/show/2IEN4eE9HvNKJHnLv5EMG9?si=f1b82ca1795d4f92) [Subscribe to Dialectic on Apple Podcasts](https://podcasts.apple.com/us/podcast/dialectic/id1780282402) [Subscribe to Dialectic on YouTube](https://www.youtube.com/@Dialectic) # Transcript **Jackson** Good to be here with you. **Geoffrey:** Good to be here. Me too. **Jackson:** In what seems to be summer in DC, it's officially warm. **Geoffrey:** Summer is here for sure. **Jackson:** We have lots to chat about today. It's clear we could do this for a long time on a long range of topics. Based on our lunch before this, we're going to stay relatively narrow for a little while. We're ## [00:02:12] Agency in a Digital World and Geoffrey's Creative Medium: Software **Jackson:** going to talk about an idea that you've spent at least five years working on. It's at root about software and programming, but really about this idea you have of malleable software. I'll read from you. First, you say: "The best environments come from adaptation. When the people living or working in a space pay attention to their own needs and gradually evolve to meet those needs. The result is beauty and quality. Everyone deserves the right to evolve their digital environments. It's the only way to fulfill our creative potential and maintain a sense of agency in a world that increasingly resides in software." You've been chasing after this idea in this dream for a while. Starting zoomed out: Why is this idea so interesting to you? Why is code and software broadly the creative medium of choice that you've been so attracted to? **Geoffrey:** I find it almost painful when people can't change their surroundings, their environments, tools. Whenever I see someone fighting against the grain, unable to do the creative work they want or unable to focus the way they want, because some tool in their life is either pushing a weird incentive on them or just isn't configured the way they want, it's this visceral feeling: that person is lacking agency and has been robbed of something really important. One specific story, maybe, that was my waking up to this: I used to work at a startup building education technology software for schools. We would make data reports, thinking really hard about the best report for K-12 schools to see. On feedback calls, people would say a lot of stuff was missing or wrong. Sometimes it was tiny stuff, like, "Could you change this one word in the UI? Because at our school, that word is really contentious." Something I learned working at a startup is that usually, you're supposed to say no. If you're a good product manager, you're screwed if you say yes. A lot of your life is telling people, "We can't do that just for you." In Silicon Valley thinking, this is kind of valorized. You end up with a simpler thing; don't just give people a faster horse. But people know what they need. The only reason we don't give it to them is because we assume this kind of mass-produced product. It's true: if you're building it in the factory, you obviously can't build it for just you or just me. I started getting really intrigued by whether there could be another way to think about software that grants people a little more control and agency over their lives. The reason software matters to me is that it's the medium I chose to start building in when I was younger, because I could and I didn't have to ask permission. It's taking over our lives. How much time do we spend? I'm at my desk, but really I'm in my phone or in my computer. It's a portal. If we can't figure out how to give people agency in their digital worlds—that's so much of our lives now—we might even start losing the feeling that we own or control anything. If we spend all our time in platforms that dictate every detail for us, a way of thinking might be lost. That's really scary to me. On the flip side, it's really exciting to imagine a world where more people feel. There's a famous Steve Jobs quote: "Everything around you was invented by someone sort of just like you," I think is the paraphrase. That's a really exciting way to think about things. **Jackson:** A great way to open up the conversation. I also think if you surveyed young people today, particularly 20-year-olds, we are starting to see what you just described: people feel as if they have no control over their own life. That's a terrifying thing. **Geoffrey:** Everything in the technology world has become more complicated. It used to be that you could open up the car and understand a lot of what was going on. Now there are a hundred computers in there. You can't open up your iPhone and meaningfully repair it. In my own life, I currently rent an apartment, and there's a really big difference in mentality when you start thinking a little bit more like an owner and not a renter. It could be something as simple as a toilet: an object that I can understand and do something about, as opposed to the reaction, "Oh no, call for help immediately!" I try to cultivate in myself over time and grow that part of myself: Can I actually understand what my environment is built of and take more agency over shaping it? **Jackson:** I found that overwhelmingly. I don't know if you would explicitly think of yourself as a designer, but what you described—the way you started your answer—feels to me like the most design-oriented philosophy: when I encounter friction, I want to shape it and fix it or tweak it or make it a little smoother. I think it's a really powerful gift to imagine giving to other people. Before we go deeper—and you started to talk about this a little bit—why, at a super high level, do you think more people should program? Granted, what that means might change or be in the middle of changing, but why should they be interested in using code as a creative medium? **Geoffrey:** To be clear, I don't have a strong opinion on whether people should be programming more. As you said, what that even means is changing fast right now. Frankly, the fact that until now you've needed to learn to program to shape your digital environment is one of the biggest problems in our software world. And so, I'm not against people learning to program; it's just that my goal is not for more people to program. The reason I want more people to get involved in crafting their software is that, especially in creative work, there are certain types of work that can only emerge out of the correct creative environment. I think of it as almost shaped like a leather shoe around the person, rather than the person trying to contort themselves to fit into a tool. You see this with creative spaces. I love looking at photos of people's studios. There are amazing ones. Like Stewart Brand, when he wrote this great book, \*How Buildings Learn\*, had this—I think it was a shipping container. He put up a bajillion photos on the walls and was surrounded by this space he created, this container, literally to do his best work. For myself, the choice of tools matters. The tools I use to write with, the tools I use to draw and think with, meaningfully affect the outcome of my thought process. It's not just an auxiliary. **Jackson:** Environments matter. When you're in a room, what does a room feel like? **Geoffrey:** It matters a lot. This goes back to the original promise of computers: to let people create the ideal environment for themselves to do their best work. There's some great thinking from pioneers like Doug Engelbart, who is known in a shallow way for inventing the mouse, but in a deeper way, he invented the idea of computers as a way to augment human intellect. He gives a great example: Imagine you had to write an essay with a brick tied to your pencil. Think about how much that "de-augmentation" would affect your ability to think. Now imagine the inverse of that. What's a tool that would deepen your ability to do your best work? I find that potential so intoxicating. Computers are probably the only tool we've ever invented that are a universal simulator. It's a meta-medium that can do anything. The ceiling on how cool that could be for people, in terms of our ability to think and create, is so high, and we're so far from hitting it. That's the potential that excites me. **Jackson:** It's easy to forget that this is a very new medium too. If you think about it in the super abstract of the physical world and the digital world, we are only in the very early innings of getting accustomed to the dynamics of this new medium. It's also powerful to think about that Stewart Brand idea you mentioned—the environment. Those environments, the ones that feel right, are never top-down. They are never constructed and pre-designed. It's more like a garden. It's a very Jane Jacobs kind of idea, which is cool. **Geoffrey:** They have to evolve slowly as needs arise. They gradually form around a person and a way of working rather than being planned upfront. Christopher Alexander has been an influence on both of us. He has a lovely idea that you shouldn't focus too much on the blueprints for the house. When you build a house, you should roughly make a plan and adjust the windows as you build it because you'll see where the sun is coming in. Crucially, you shouldn't build the whole house. You should build part of it, start living in it, and wait and see. As things come up, you'll finish it off or evolve it over time. I find that way of thinking to be how a lot of the best things come to be in the world. **Jackson:** They're iterative. The irony is that's really hard to do with bricks or wood. It should be easier with the digital, plastic idea of code. ## [00:12:17] Intro to Malleable Software **Jackson:** Let's dive in a little bit more. I have a smattering of quotes of yours that encapsulate some of what we were just talking about and also set up how I want to talk about what malleable software is as you think about it. First you say, "In the physical world, the act of crafting our environments comes naturally because physical reality is a malleable place." You go on to say, "Many small tweaks—taping a Post-it note to the wall, rearranging some drawers, moving a piece of furniture—can be done instantly without asking anyone's permission." "We can also take on larger changes that require more effort and skill, like building a workshop or renovating a kitchen. We work and live in a physical space that we can control." This is both Stewart Brand and Christopher Alexander, the architecture. You then say, "Computerizing work led to a loss of agency. Previously, new ideas took minutes to try with a roll of tape; now they could take hours of complex configuration, if they were possible at all. Why even bother imagining new processes? The team developed a sense of learned helplessness." "Software ought to be the ultimate medium for free expression. We are not bound by the laws of physics." "In practice, the structures we have today for creating software too often get in the way for skilled developers. They introduce mountains of incidental making it far harder than it should be to build great user experiences." "And for end users without much programming expertise, using a computer usually boils down to prefabricated experiences created by developers without much hope for modification." Finally, "This is the dream of malleable software: editing software at the speed of thought." This touches on many themes we were just discussing. What you're implying, and clearly what you've spent so much of your work on, is that as amazing as that vision you opened the conversation with is, that's not really how software is. It's that great irony. My first question is, is it so critical that these mediums, formats, or tools be extremely reactive and editable in that speed-of-thought way? More personally, how has editing your tools and environments, whether digitally or physically, informed some of this? **Geoffrey:** First, I want to say you're quoting from an essay I wrote with collaborators Josh Horowitz and Peter Van Hardenberg at the research lab Ink & Switch, where I work on this research vision. Much of this is our collective thinking, and we have many collaborators at the lab who've contributed to this work. It's by no means a solo effort. **Jackson:** That should be a good blanket statement, as I'm going to quote you a lot. **Geoffrey:** That applies to the whole way I think. This is heavily influenced by many other people. To answer your question, I'll start with a story of modifying tools in the moment, which actually comes from writing the essay you're quoting from. I was writing this piece. We have an in-house, collaborative writing tool that we use, and it's designed to be malleable, ideally. The piece felt really long, and we got feedback: "This is way too long." **Jackson:** It's not a short piece, or at least it wasn't when I read it. **Geoffrey:** We want to go in depth and be rigorous, but also respect people's time and be concise. I was trying to figure out why an essay was too long, where the fat was. I couldn't tell from scrolling; it was just too long. I realized I wanted an outline with a word count on each section so I could feel it out. Maybe I could have gone to the App Store or found a website to do that, but it wasn't immediately obvious how to find it. It was a need I had. In our system called Patchwork, where I was writing this, I tabbed over to my AI-assisted IDE and said, "I want a view of the essay I'm writing with an outline and word counts." A couple of minutes later, it was done. I ran one command, and now, in our environment, that is a new tool I have in my arsenal, running on top of the existing essay we were already working on. I can share that tool with all my collaborators, and we can have conversations around it. As the essay updates, the tool shows the latest word counts. To get to the first part of your question, the reason it has to be in the moment is so often... **Jackson:** By the way, how long did that process take that you just described? **Geoffrey:** It literally took about five minutes, is my guess. Importantly, there are two halves to that. One is AI code generation, which has unlocked tremendous velocity that wasn't possible before. Two, our environment has something that a lot of AI coding tools out there don't have today: a story for composing smaller tools together into a larger workflow. I could go into cloud artifacts and make a tool that can do an outline word count of a markdown document. But then how does that access my existing document? Do I keep copy-pasting it in every day? What we really want is an environment where those new tools we create so quickly have a place to live—almost an operating system. You can think of it as a place where it's expected that you'll be constantly making new stuff and integrating it with what you already have. **Jackson:** Which is how the real world works. **Geoffrey:** That's totally how the real world works. **Jackson:** You don't need an API to use your hammer and your work. **Geoffrey:** You can buy a new hammer or a new knife, bring it into your kitchen or woodshop, and it instantly starts working with all the other tools you already have. You don't build a new kitchen every time. A lot of what we talk about in the essay is that the way software is built today assumes each company runs a kitchen where they make all the knives and spatulas. There's no ability to buy a different knife at the store and bring it in; there's no ability to compose tools. That's a goal we're reaching for with this environment, and the reason it matters. Not every tool is possible to build in the moment, and it's by no means a universal thing that has to be true. However, many good ideas wouldn't get built if they took a day. I don't have a day to build that thing; I'm focusing on writing this essay. So, if I can... **Jackson:** The difference between five minutes and a day involves focus, train of thought, friction, and nuance. You might be willing to take five minutes, but an hour would break that. **Geoffrey:** There's a critical threshold: if you can stay in flow and change something, then it becomes worth doing. And that, I think, is huge. **Jackson:** A person in a workshop might modify a tool while they're building something. **Geoffrey:** Absolutely. Woodworkers—I'm not a woodworker, but I've talked to them—call them jigs. You can build a custom thing to help you with one project. **Jackson:** I interviewed this couple, CW&T. They're industrial designers. Their workshop is where they build tools that they sell, but they mainly build tools for themselves. **Geoffrey:** That's so cool. Those custom personal tools, some of them might be easier or harder to build, but there's a ceiling on how much effort you're going to put into something that's not a product I'm shipping. It's just a scrappy tool. **Jackson:** Way you run the risk of becoming the optimized. Reading a lot of your stuff made me think of the classic Apple Notes bell curve meme. That's the downside. The downside of being a tweaker-type person is you get over-focused on your tool and stop focusing on the work. **Geoffrey:** Yes. That's absolutely a trap for anyone who enjoys thinking about tools for thinking. I'm now on a multi-year sidetrack of building not just tools for thought, but a platform for building tools for thought. **Jackson:** It's bigger than just you, which makes it a little more tolerant. **Geoffrey:** I like evaluating people's tooling based on their end output, not on how cool their tooling is. So I'll take positive lessons from anyone who's shipping good things into the world, not from someone who has the coolest Obsidian setup or whatever. **Jackson:** Yes, we love Obsidian. **Geoffrey:** We're both Obsidian fans. But there's a correct amount of investment. **Jackson:** I've spent a lot of time doing what you just described, thinking I was going to make a quick tweak, even with ChatGPT, and then getting stuck. We're going to come back to a bunch of the things you just said, but I want to stay high level for just a little bit longer. ## [00:20:42] "Popping Open the Hood" & The Nightmare Bicycle: A Case for Understanding How Systems Work **Jackson:** One metaphor that you've referenced a bunch of times, and that I continue to see coming up, is Alan K's notion of popping open the hood, which is a really beautiful metaphor. It's also a little ironic given that most people, if they drive, probably aren't popping open the hoods of their cars or they're driving a Tesla, and it's a front. But the essence of it is beautiful. There's another metaphor here, which is exposing the seams of something. Conversely, we live in a world that increasingly tells you that you don't need to understand how a system works. You don't even need to see the internals, understand the internal dynamics. Arguably, that's what's happening to software right now. Stefan Go, who I've interviewed and who I think you're loosely familiar with, has a great essay on "Don't Delegate Understanding." That's playing with this a little bit. What's your case, maybe in the specific software case or broadly: why should we actually want to pop open the hood sometimes and understand how systems work? **Geoffrey:** This question is plaguing me right now, so I'm glad you asked it. As you alluded to, AI is a tricky moment because it's making more things possible without developing deeper understanding. This question feels urgent. The reason that resonates with me goes back to your mention of Alan Kay popping the hood. He wrote a great essay in the 80s about the idea of what it would feel like to pop open the hood of software and what you would see. The reason he wanted people to do that was that he wants people to be able to edit their tools the way they edit their documents. It's very directly this malleable software vision that we talk about. He says, here's how you do it: You have people use the thing, and surreptitiously, as they use it, they're accidentally learning how it works. That's great because then, the moment they think of something they want to work differently, or if they need to repair it, they've already developed that knowledge. If you don't learn how it works as you use it, then the moment something is wrong, is broken, or doesn't work the way you want, you're starting from zero. That's a lovely way to think about the value of understanding. I wrote a blog post about this metaphor called the Nightmare Bicycle, which touches on this. I'm a huge fan. It's a lovely idea from the book \*Changing Minds\* by Andy diSessa, which is one of the best books ever written about computational thinking and the value of understanding. **Jackson:** You've mentioned it a few times. I haven't read it, but I did a little more digging. My initial impression was that it was a software book. It doesn't seem to be at all. It seems to be a book largely about education, even for kids, and first principles thinking, that kind of idea. **Geoffrey:** Absolutely. It's a book that falls into some of the tradition Alan Kay started, where programming isn't just about building apps; programming is about developing your own mind. Andy diSessa's work falls in that tradition. Seymour Papert and Logo are another reference. The point of using the computer isn't just to build things; it's about you being changed by it. Your thinking being changed by it is the point. To get back to the Nightmare Bicycle idea, he tells a story about a nightmare he had involving a hypothetical bicycle. On this bicycle, the gear shifters are not numbers; they are buttons with modes on them. It's quite funny: gravel mode, uphill mode, downhill mode, puddle mode, whatever. It comes with a manual that says if you're going uphill, hit 'uphill,' or whatever. **Jackson:** It's intuitive. It should be. **Geoffrey:** It's intuitive. He says you can imagine the product manager who came up with this because they would say something like, "People can't understand numbers. We need to show them how to use it." He points out that this might be fine if you're only sticking within the four buttons that are there. Then, the moment you hit downhill on gravel, what do you do? Those four buttons aren't teaching you anything about how it works under the hood. You're not developing a more general framework, a kind of physics, to think about what this thing can do and how to make it do what you want. The thing is, bike gears are not that hard to understand. Most people can figure it out just fine. You don't even need math understanding. You get some experience with it, you start to feel how it works, and you're good. **Jackson:** My theory is, having learned it a long time ago, it would be very hard to understand looking at a piece of paper, but incredibly quick while riding a bike. This ties it back to your point. **Geoffrey:** I think it's a damning critique of a dominant design philosophy in Silicon Valley products. Maybe it's actually better to expose more of this mechanism so that people can surreptitiously develop that understanding of the innards. That has all these secondary benefits. **Jackson:** You give the example of the microwave that has the popcorn button as another example of this. **Geoffrey:** Another nemesis of mine is the popcorn button. **Jackson:** A little less bad, but quite frustrating. Are there software examples that come to mind on either end—either bad ones, or ones that are intuitive in a really wonderful way, like bicycles, which we maybe take for granted because we're used to them? **Geoffrey:** Great question. I think there are some lovely positive examples of software doing it well. Spreadsheets is one that I always go back to as one of the greatest inventions in software history. There are a lot of different things that are great about spreadsheets. One of them is that they are made for accounting, and maybe that was their initial nexus. But we've all made spreadsheets for all sorts of stuff. You can plan anything in a spreadsheet just by throwing together some tables. You can do financial stuff like budgeting, or a roadmap. There are all sorts of things you can do. What's great about a spreadsheet is it doesn't try to force any particular use case. It just gives you a bag of ingredients. You learn how cells work, how formulas work, and how to color things in. With those few primitives, there's a lot of different use cases that you can serve once you start building up that knowledge. The other thing that's great about spreadsheets is you don't need to be a super expert to get started. You can ride a bike without knowing how to change gears; it might be difficult at times, but you'll power through it. With spreadsheets, you don't even need to know much about formulas to get started. I think that's an example of a pattern, and it's not just spreadsheets. There are other examples of software that have this combinable Legos quality. Notion is a more recent example I like, and Trello. There are various products out there that have this quality. ## [00:27:47] Computational Media, Spreadsheets, and Digital Informality **Jackson:** You have referenced this. The term originally comes from Web Straits, this idea of computational media, which is at least adjacent or similar to malleable software, if not very related. To me, the spreadsheet feels like the quintessential example. I wanted to read the definition from the Web Straits page about it, which I thought was pretty cool. They say computational media is software that blurs the distinction between documents and applications and between developer and user. principles that modern computational media should adhere to. It should be: one, malleable, so the user can change and shape software to their personal needs; two, shareable with others and enable collaboration; three, distributable across a variety of heterogeneous devices and operating systems; and four, computable by supporting custom computation from within a document. I love that you brought up spreadsheets. You clearly obsess with them. I found your PhD thesis, and you literally open it by saying all of this is based on the idea that why can't making apps be as easy as making or using a spreadsheet? Which is pretty amazing. The other thing that's cool in there is that it has that element of the bicycle: the learning curve is explicitly tied to using it, and it can be as usable by the most basic person as the most complex person. Are there other elements that you find yourself coming back to about spreadsheets that make it the template for computational media? And as an additional question, we think about malleable software and computational media as effectively pointing at the same thing, or are there slight differences you would point out? **Geoffrey:** That's a great question. I'll start with the first one. Oh man, there are so many things that make spreadsheets great. **Jackson:** I love spreadsheets. **Geoffrey:** Here's another one from the list that is really important and is also a quality of most good computational media: they allow for informality. This is something that we're obsessed with at Ink & Switch. We think a lot about how physical media like pen and paper are incredibly good at allowing informality. You can sketch anything, and even if you don't know quite where it fits, or even if you don't know how to write it down in words, you can express it. Then you can figure out later how to clean it up and categorize it. **Jackson:** Back to the other thing, by the way: no APIs. Yes. Which is ridiculous. **Geoffrey:** You own your information. Digital media tend to be really, really good at formality. A database of a million sales records is probably better to manage in software than in a filing cabinet. But they often are really bad at informality, at letting you express things before they're ready to be categorized or captured in a formal way. This is really true of a lot of software, which forces you into premature formalism of how you think about something. A concrete example is we've tried building a couple new computational mediums for different use cases. Recipe management and travel planning are two of the big ones. Let's take a travel planning situation. There are apps out there that will help you plan a trip. You can ingest all your hotels, your flights, and all that stuff, and it shows you this pretty picture of what your trip is going to be. It's a pretty picture, very nicely formatted, and the mockup looked great. Then, you want to jot down a few notes about what's going to happen this day. I might be doing this one of these two days. Here's a list of restaurants and ideas for things to do. Where do you put all that stuff? Often, I'm thinking of apps like Kayak Trips or TripIt. There are others. They have a database schema, and if your thoughts don't fit into that schema for what they think a trip is, you're screwed. So, with collaborators Paul Sonnentalag and Alexander Obenauer, we developed this computational medium called Embark. We started from something that looks more like an Apple Note or an Obsidian note, where you're just writing stuff down about what you want to do, and it's completely unstructured. Then, can you gradually start giving it a little bit of structure, just as much as you need? Maybe you'll organize it by day, separate out some things, or tell the computer, "Oh, this hotel, it's actually this Google Maps link." Then your computer can do stuff with that. It goes from a very sketchy thing to gradually having the computer help you out more. It's very smooth, and you never have to hop into a specific, rigid way of thinking. That's something spreadsheets are really, really good at. It also causes the biggest problems with spreadsheets, frankly. You messed up the formula, or the table; the range didn't get extended. There are problems with it. But people often completely forget about the good parts. They try to solve the problems. They say, "We can't run our business on spreadsheets. We need 10 SaaS products to do it for us." And they forget that the spreadsheet actually had these amazing benefits that you're losing. **Jackson:** I'll link to Embark, which isn't usable. I dug around a little bit. **Geoffrey:** We've not publicly released it. There are reasons for that, and it's not intended as a commercial product. This was a research exploration for us. The goal isn't that you can use it. The goal is that we and you see the ideas and get inspired to build a better computing fabric. **Jackson:** I'll link to it. There's a vibe in it of you comparing it to four different windows of the travel planning thing. I think of us sitting next to one of these boards where you can stick pins in. You can imagine layering on more and more contacts as you go in this very bottom-up way. It makes me think of the magical promise of computing as this magical substrate that you can imagine. If you had a magic bulletin board that, as you stuck stuff to it, updated, it would be really cool. There are little threads of that which I found pretty awesome. ## [00:34:01] Legos and Home Cooking as Metaphors for Software **Jackson:** We're going to talk a lot about the very tactical side of designing malleable software. One last high-level thing, or philosophical thing, is a couple of metaphors you use that I find really helpful. The first is around Legos. You say It is very valuable to have someone think through an entire unified product experience and make it all fit together in a coherent way, rather than having to put together pieces. This is a case for the toy out of the box. This is from an essay where you're talking about Foam, which was an early Obsidian-like editor. You say, "I think this is a neat way of creating value in software. The essential value of Foam isn't code; it's the opinionated curation of existing building blocks." If one end of the spectrum is the pre-built toy that comes out of the box, the other end of the spectrum is you have to build it entirely yourself. LEGO is a cool metaphor. You've talked about it in the context of: you can build exactly the LEGO set that they tell you to build with the instructions, or you can go wild. Mode, play mode. Is the construction element of LEGO still too much friction, maybe? **Geoffrey:** This is a really important point where I don't want to be misunderstood: the vision of malleable software is not that everyone should build everything from scratch. That would be an enormous waste of time. Other people are making great stuff often. I have lots of apps that I use that are 95% perfect for me. I love them. I agree with the taste of the creator. The point of the LEGO analogy is that when someone gives me a LEGO set, and I love 95% of it, but I hate that that turret is in that position, it's no problem. I just take the turret off and move it. It's built out of pieces I already know how to use. I can do that last-mile tweaking. Eventually, if I keep playing with this LEGO set for a year, it might look pretty different at the end. It's not like I started from zero. That's a philosophy that I find could be applied more to software. **Jackson:** It's worth noting, and we'll probably talk more about this later: LEGO is one company and one very explicit design language. And there's a whole bunch of us. **Geoffrey:** There's a big tension there: they have an API standard that they adhere to, which is critical. **Jackson:** If only it was the dictator who told us. **Geoffrey:** We think a lot about that problem ourselves: how do you create the equivalent of the LEGO standard in software? The LEGO standard is very prescriptive and very precise, but it's also open-ended enough to enable a lot of stuff. With Lego, you can mix and match. If there's a part of one set and a part of another set that are both good, and you want to make a spaceship that has those wings and that cockpit, you can probably figure it out. One of the problems with apps is this. There are certain app categories I really care about; let's take email, for example. I don't use email very much at work, but it is one of the big pains in my life, so I try to go to any length possible to manage it reasonably. A lot of people probably feel similarly. I have found so many email clients that are 80% of what I want, but they're each a different 80%. I keep thinking to myself, if I could just have the categorizing algorithm from one and the inbox view from another, that would be the perfect email client. I would even probably pay for both products just to have the ability to mishmash them like that. But the way we build software today, that is not a thing—not even close. We are so far from having software built in a way where that is possible, like Lego mix and match. What it means is that I need to go all in on one company doing everything right and just go with their strengths and their warts. I have no ability to mix and match. **Jackson:** It's telling that the quintessential strategy for many people in software today is literally, "Oh, I only use Apple," or "I only use Google." That's the extreme. **Geoffrey:** Again, back to the physical analogy. There are a lot of non-software places where we have more modularity than that. We talked about the kitchen. You can swap in a different knife, a different cutting board, and different pots and pans; it's a pretty modular thing. There are levels to it. For example, it would be a lot harder to change out the handle of the knife separately from the blade, so there are nesting layers of this stuff. But in software, we tend towards these big monolithic things instead of starting from more modular pieces. **Jackson:** The second metaphor of the two I mentioned, also from that foam essay, is where you say, "But foam is more like a recipe than a final dish. A recipe tells you a good combination of ingredients, and you get to make the food yourself, substituting ingredients to your liking along the way." In another quote, you say, "Yes, the restaurant food is truly better in many ways, and there's a role for restaurants in society. But I wouldn't want to live in a world where no one cooks and food is something we can only choose off a menu. Software is increasingly heading to that place." There are a few other related ideas. I love the little essay you linked to about how an app can be a home-cooked meal. In an essay yourself, you say your pie doesn't need to be original. You're talking about the notion that every time you bake a pie, it doesn't have to be compared to every pie that's ever been baked, which is really beautiful and empowering. It ties a little to the Lego idea. If I were to be critical, I would say both Legos and home cooking are much closer to, much more on, the play or leisure end of the spectrum than the utility end. They're not as serious, at least in most cases. I'd love for you to riff a little bit on why you think malleable software also applies on the more serious end of the spectrum for people doing real work, people coordinating, people working on teams. It's a little bit like the Apple Notes thing again, where serious people use Apple Notes instead of tinkering with their Obsidian or whatever. **Geoffrey:** I think you're getting at an interesting aspect of malleability: maybe there are two different reasons to do it. One is because you get better, more productive tools, or somehow you do better work as a means to an end of an output, which goes back to... **Jackson:** Importance of lightweight flow. Lightweight edits. **Geoffrey:** Yes, totally. There I was not trying to enjoy making tools; I was trying to make a good essay. And then there's a second thing, which is more about the inherent enjoyment or meaning we find from engaging with the world in a certain way. I take your point on cooking home-cooked meals. I might push back a little and say, actually, home cooking is pretty serious. It's the main way that most people feed themselves, and what you feed yourself matters. When you think about how are you going to eat in a healthy way, or how are you going to have your kids have memories of a certain culture, this is true, it's personal. And it's totally true that there's also an inherent enjoyment: I enjoy cooking, and I find meaning in the act. **Jackson:** Maybe you're even meal prepping at the beginning of the week on Sunday. **Geoffrey:** It's great pushback. It is true for me that I would not be able to eat the food that I want if I couldn't cook at home. Maybe if I hired a personal private chef or something I could, but no amount of restaurant and meal kit is going to suit my needs. I'm pretty particular about the food that I want to eat; I like food a lot. So I do think even in these smaller scale, more local and personal use cases, sometimes the ability to tailor it to me is one of the most important things. No amount of professional skill at a mass production factory far away is ever going to make up for that ability to make it. I cook for me and my wife, and I know what we both like. I can adjust things and make it so that it's something we're both going to love. We've tried meal kits, and it just fails to deliver on that. **Jackson:** Totally. It's a great pushback. There are a couple of threads there that I think we'll come back to: both the notion that you know better for yourself than others, and the notion that maybe we, in the software metaphor, might soon have a professional private chef that can cook for you, which is interesting. I want to go a little more grounded first. ## [00:42:30] Two Types of Malleable Software: Modular-by-Design and Hacking **Jackson:** Feel free to push back as much as you want on this. As I was reading through and thinking about this, I'm seeing two core buckets for malleable software today. One is designed malleable software. You could think about this as a settings menu, to a robust plugin architecture like Obsidian, to something like spreadsheets—native computational media. The other is hacked malleable software. You write a ton about browser extensions, but I think another great example would be games modding, which is really cool. It seems that you are particularly inclined to the hacking category of things. I think you give a lot of love to both; maybe I find the hacking thing particularly interesting. So my question is, one, would you edit that, push back on that framing at all? And two, perhaps almost philosophically, do you think there's an opportunity to reclaim hacking in the broad sense, particularly in a way that makes it more accessible? Right now, I think a very small subset of people can hack. For the rest of the people, it goes back to the root of the philosophy of all of this: should software actually be something that anyone can tinker with? **Geoffrey:** These are different philosophies, and I think they're both good, but they're really different from one another. We think a lot about the tradeoffs. The best thing about the hacking approach, the modding, the browser extensions, is that you don't need to wait for the person who made the thing to give permission or to set up the interface to hook into. You just do it, and you don't need their permission. Often, they can't even stop you. They might be able to make your life harder. **Jackson:** Life harder, which you make an amazing point about. AdBlock is a super simple version of. **Geoffrey:** Although, I'll point out, AdBlock gets a lot of attention among browser extensions, and it is the most popular extension. There, it's very adversarial. The websites don't want you to do it, and you're doing it. That is a thing. Often, it's not that the original creators of the app want to stop you; it's just that they're busy. One analogy I think about in physical space is imagine if every time you wanted to stick a Post-it on the wall, you had to call up the wall maker company and say, "Can you install a hook for me to add a Post-it note to the wall, please?" **Jackson:** You have another example of just painting your wall. Does the homeowners association approve of green? If so, we'll apply it to everyone. If not, no. **Geoffrey:** Exactly. It would be so ridiculous to block tiny tweaks on the original creator, but we do that all the time in software. You can't change a single word; you can't change a single color. What's great about hacks is that they are a way around that. Especially for tiny tweaks, you can often get away with just messing around from the outside without official support, as long as the platform is built in such a way that enables that. This is something we think a lot about. Browser extensions work because the Chrome team did a good job designing a certain interface. **Jackson:** It's actually both of those things at the same time. It's the upfront design decision that enables the hacking. **Geoffrey:** Exactly. But it's an upfront design decision at a lower level. It's a foundational platform that thought about this. They did that once, and then all the app developers don't have to think about it. This is a really consistent pattern in systems that I like for malleability. Assume app developers don't have time to think about this. They're not anti, they're not pro. Can you make it so that when they ship software, it naturally inherently is going to be moddable? The web platform is probably the best platform out there in terms of this right now, much better than iOS, for example. So I think that's really cool. There are problems, too. Browser extensions have a lot of issues. They break. Security and privacy are a complete nightmare, and that's a big challenge. **Jackson:** Yeah, it's telling there are very few browser extensions. Almost all the best ones are incredibly lightweight. You give this amazing example of so much of what you're talking about in the Bank of America checkbox extension. But it's stuff like that; it's not robust apps. I think that's kind of illustrating the point you're making. **Geoffrey:** There's a depth. There are limits to what you can reasonably do hacking around from the outside. Often, it's 95% good. What I need changed sometimes isn't that big; it's tweaking around the edges. For that, hackable software platforms are great. At Ink and Switch, we think a lot about how we can do better than browser extensions. What would it look like to create a platform that's even more malleable than the browser? A couple of aspects we consider are: one, AI exists now, so everyone can code to some approximation, and that's going to become increasingly true. How would you design an operating system from the ground up, assuming everyone will be constantly writing code? This isn't the assumption we started from in today's operating systems. Second, at Ink and Switch, we've done a lot of work around what we call local first, which is a philosophy around owning your data. It relates to 'file, not app,' which Steph was talking about on your podcast. That's another piece that fits in for us because one of the big problems with browser extensions is cloud software. You don't own the data or the software; it's running off on some server. Most things I want to change on websites can't be done in browser extensions because of that fundamental architectural choice. All of Google Docs runs on a server; I don't own it in any meaningful way. In our work, we think about whether you can give people not only ownership over their data but also deep ownership over the software. This way, the hacking approach could be applied to any part of your software, allowing you to change anything. **Jackson:** In many ways, a lot of this in practice is about making more stuff hackable and making hacking lower friction and more possible. ## [00:48:35]   Hampton ## [00:50:13] Designing for a Smooth Slope **Jackson:** I want to talk more about To me, you are clearly a design-oriented person, and so much of this is about design, whether at the high-level system type stuff we were just discussing or in the very micro. We already talked about the nightmare bicycle, which was one of my favorites. At root, so much of this thinking—the ideological and philosophical underpinning—is the fundamental belief that people understand their own needs well, are competent, can learn, and should be trusted. That's the simple explanation of the nightmare bicycle. In some ways, it's being paternalistic; it's saying, 'I don't trust people to be able to do stuff.' Another idea you explore that ties into this is the idea of a smooth slope or a gentle slope. In many ways, the funny thing with the bicycle analogy is that the great benefit of software is that it is adaptive. It isn't like you make one decision once on the handlebars, and that's how it is; the gears are the way they are. There are almost two slopes in this design space. One is a slope of agency or learning, like the bicycle or any high skill-floor problem. The other is a slope of malleability: how malleable should it be? Obviously, those two things run into each other. I'd be curious for you to think about: is there any world where, while wanting these thoughtful, gentle slopes, you might also overdesign the slope of malleability such that it develops some elements of the nightmare bicycle? **Geoffrey:** Let me start by telling people a bit about this idea of the gentle slope and what it means. We talk about this in our Sai Malleable software, but there's a lovely paper by Alan McLean and some co-authors where they introduce this idea of the gentle slope. The idea is you want to go deeper and deeper into customizing some systems. Often in software, at some point you hit a wall. Past this point, good luck; you're going to have to learn to program or something. They call that a cliff because you have to instantly learn a ton more stuff to make one more step forward. **Jackson:** You make this point with the browser extension. It's either you can use a browser extension or you can learn a crazy amount to be able to make one. **Geoffrey:** Exactly. Making browser extensions is not easy, even for programmers. Leave the browser, open some API docs, write a bunch of code, distribute it through an extension store. There are all these barriers. The idea of the gentle slope is if you want to do something that's a little more complex or more involved, you should only need to learn a little bit more stuff or go a little bit deeper. We want to minimize the amount of upfront cliff learning you have to do to get further. I think about this in the analogy of physical space. Obviously, there's a big difference between moving a Post-it note, moving a desk, and moving a wall. Those are not the same thing. Physical space often has a reasonably gentle slope. To move a desk, I don't need to hire a general contractor. I can ask my friend to pick it up with me and move it. As I get deeper into crafting this space, there are skills I have to learn or people I have to cooperate with. There's this linear relationship. In software, the way this analogy applies is as if you had to hire a construction crew to move a Post-it note or something. It's this very incommensurate amount of effort invested versus the thing you were trying to do. **Jackson:** we only have one way of moving things, and it's a moving team. **Geoffrey:** Exactly. You can't move it with your hand; it's a backhoe. In the browser extension example, we think a lot about what other things you could do to smooth that slope. If I can make a theme and change a color in the browser, then I don't need to learn how to make a browser extension if that's all I wanted to do. **Jackson:** The browser company explored some productizing. **Geoffrey:** Exactly. That's a perfect example of taking some part of what people want to do, making it easier, and establishing that linear curve. It doesn't mean things aren't ever hard. It doesn't mean you can magically do everything easily, or that you can do everything yourself and never have to work with anyone. All it means is that easy, simple stuff shouldn't be ridiculously complicated. **Jackson:** Critically, there's a broad idea I'm a huge believer in, which I think I originally sourced from this guy, C. Thi Nguyen, who writes about games and agency. What a good game designer does is make the ability and goal tension consistently in the right delta. It's never too out of whack. **Geoffrey:** I love thinking about game design as an inspiration for this stuff. I'm not a game designer, and I almost wish I had that skill set because those people are often doing some of the coolest work in terms of what we call onboarding. **Jackson:** It's all agency. **Geoffrey:** They think a lot about how to organically develop people's skills by staging the learning process into small bites appropriately, and disclosing it naturally in the course of using the thing. I might push back a bit on your idea of those two slopes of learning and malleability. I think they're very related, and they might be the same slope. While you use the thing, you should learn more about how to use it. Part of using it is changing it; that's a natural progression. **Jackson:** It's a little paradoxical, and it's not really how a lot of things are in the real world. It's a unique part of software, and an ideal frame of software that isn't always the case. **Geoffrey:** It's definitely not always the case. But I think it's more like a person. **Jackson:** A person. This is a classic trouble: we think about ourselves as always in stasis and other people as static. No, we're both reflexively changing all the time. And software, as you said earlier in the conversation, is a relationally dynamic medium, which is really important. **Geoffrey:** Earlier, you talked about computational media and blurring the line between document and application. I think that's relevant here because, in some sense, the problem is that we think of using and changing as two separate things. But with a document, are you using it or changing it? It's a material, and you're doing stuff to it. It's just one process. Great point. I think software ultimately would be better in a lot of cases if it felt more like that. You don't enter a separate mode to change the software; it's just a natural part of using it. When you think far future, imagine hypothetically an AI that can, in a snap of your fingers, tweak any part of your software instantaneously—literally at the speed of thought. At that point, what does it even mean to use a piece of software? You might ask if a feature exists, and if it doesn't, it will materialize instantly on the fly. We could see a lot more interesting blurrings happening as we figure out the cogen stuff and what better foundations look like for this. **Jackson:** What's funny is this is how TikTok works. Am I training the algorithm, or am I watching? **Geoffrey:** Totally. **Jackson:** You wouldn't think about it that way. **Geoffrey:** I don't know how I feel about that comparison. It makes me nervous. **Jackson:** But I think those guys are masters at that notion of incrementally moving you up a curve or taking you somewhere it's not even fully conscious. And again, maybe in a dark pattern. You could say the same thing about Match three games. **Geoffrey:** In some sense, I think games are a healthier example to learn from, hopefully. **Jackson:** What I'm pointing at is a notion from Gabe Layden, a game guy. He talks about this notion that the only game in the future might be a game where you open it, and depending on how you play Level one, you end up in Match three or League of Legends. **Geoffrey:** Yeah. **Jackson:** And I think that's inside of what you're starting to point out. **Geoffrey:** Totally. ## [00:58:20] Unbundling Apps into Environments and Tools **Jackson:** You referenced it briefly earlier, but one of the core parts of this most recent essay is the notion of tools over apps. You say decomposing applications into composable tools. There's a broad notion today that software is extremely container-first or container-oriented, which you might even describe as the tool and the environment bundled together. use a lot of these metaphors that think about allowing tools to be open and dynamic and free and move across different environments. The classic metaphor in the piece is this idea of a chef's knife that you can carry around the kitchen. There's one excerpt where you say a fluid workflow where all your tools and materials are always at hand is impossible in the world of big box applications. It's as though you have to carry ingredients from one end of the kitchen to another every time you want to use a tool. The other funny metaphor you use is the avocado slicer, which you say apps are avocado slicers. This ridiculous tool that I didn't even know existed has three parts that are all good at slicing an avocado. Your critique in the piece is, just learn to use a chef's knife. You will be so much better for it rather than buying this stupid piece of plastic. Some of this too, and you brought up Embark earlier, are a few of these ideas you built, which is like creating little spaces for a set of tools to exist inside, versus the nightmare of Google Maps and Kayak and Flight. I was doing this this weekend trying to book some travel from Europe. It's like, ChatGPT, does the flight exist? That allows you to edit the medium while editing content, which I think is really powerful. There's an excerpt on those two that I thought was cool. You say both Potluck and Embark provide a gentle slope. They start out as user-editable content and layer on interactive behavior optionally and gradually. They also support tools, not apps, using documents as a substrate for composing tools. You can gather all the relevant information for a project in one document, and then all the tools in that document can easily access the context. Then you can visually compose tools within the document. My interpretation of this is almost as though you're really talking about two ideas. You're certainly oriented around tools, but it's actually about creating substrates or environments and flexible tools. Those two things are both pretty needed. If the earlier thing is like the app is the integration of those two in a way that's bundled, is that the right way to think about it? Can they be thought about independently, or do they need to be thought about in that combined way? It's both the kitchen and the chef's knife. **Geoffrey:** That's a reasonable way to split it up. In some ways, designing the substrate or the environment might be the harder part. The reason we don't have this isn't just because someone was evil and didn't want it. It's because it's really hard to do well in software. It comes back a lot to that stuff about formality we were talking about. A lot of the difficulty is when you represent stuff in a really formal way that's really particular to your way of thinking about it, then it's no longer compatible with the way someone else is thinking about it. That's very different from physical reality. An apple is just a thing. **Jackson:** There is already a defined substrate. **Geoffrey:** Totally. **Jackson:** Somebody decided on. **Geoffrey:** It's a substrate. Someone who knows decided on. We all really understand it well because our brains are wired to know what's possible in a substrate. That's a really high bar that we might not quite be able to hit in software, but it's worth treating that as a normal. **Jackson:** But we take it for granted that we're at the extreme opposite end of the spectrum. **Geoffrey:** Totally. **Jackson:** There are no shared substrates at all. You're living on little islands. **Geoffrey:** I think there have been lots of really interesting ideas in computer science about how to solve these challenges, but I don't think we've fully figured out the answers yet. One is, how do you represent information in a way that different tools can work with it, even if they have different opinions about how to think about that information? **Jackson:** Yes. **Geoffrey:** We did a project at NCAN Switch called Cambria, where the idea was letting different views of information coexist at the same time so that different apps could see it in a different way lens, while having some shared underlying thing that they can both use. That's one problem. **Jackson:** This is effectively another way to do what Stefan and the Obsidian guys, or the local-first folks, are doing with files. **Geoffrey:** Absolutely. What's the file format? Obsidian is a great example where they've picked Markdown, mostly, as their bit. There are a lot of good things about that, especially the fact that it's already in use. But that also prevents them from doing certain things. **Jackson:** JSON has a bunch of benefits. **Geoffrey:** Totally. Yes. The big dimension that I think is the hardest to get right is how much structure and domain specificity you want to encode in the information. JSON is great. It's very general and can represent a lot of things, but it doesn't necessarily get you that much. Just saying it's JSON means, okay, what are the fields in it? A Microsoft Word document is another file format. It's now open. It's very, very, very complicated because it has accumulated decades of the Microsoft Word feature set. I think getting the level of representation right is really tricky. I'm optimistic that AI will help here because one of the things that AI is really good at is translating information across formats. **Jackson:** You add in MCPS, whatever. **Geoffrey:** Exactly. It's also really good at interpreting less structured information. It's possible the shared substrate will be less structured. It could be a folder of PDFs that all our apps use. Because that's how the world worked before apps: it was all pieces of paper that we... **Jackson:** Now we have OCR served. **Geoffrey:** OCR in our heads, and it's pixel—that's how it worked. Maybe that works for tool composition. Data representation is one challenge. How do you make user interfaces that feel good when they're made up of different parts? If someone good makes an app that's nicely designed, it's very cohesive. It all fits together. **Jackson:** Notion by default is way snappier than Obsidian, even at a very... **Geoffrey:** They're very opinionated about their blocks. They let you embed third-party stuff, but when you do, it feels really weird. If I embed a TLDraw whiteboard in Notion, it's really obvious that it came from a different planet. **Jackson:** Yes. **Geoffrey:** Some amount of that is okay and inevitable. A mishmash, scrappy aesthetic is fine and maybe something we should try to get more used to. It reminds me of a house where someone actually collected stuff over 20 years versus a house that some developer designed. You can tell. **Jackson:** I went to Florence for the first time last summer. Florence is Notion. There are basically two design periods. It has the exact same design language. It's really cool, but the city is trapped in the 1500s or at least the 1900s. New York has problems, but it's pretty cool too. **Geoffrey:** Another influence that reminds me of is old European villages. This connects to Christopher Alexander's work. There's a way that things can be beautiful that's not top-down. Alexander talks a lot about this idea of patterns that can compose so that there's coherence. Yes, it's not random, but also it's not all uniform and factory-produced. I think that's a really powerful idea, not just at the aesthetic layer. It also goes deeper than that. You can get a ton of coherence out of everyone. Build your own house, but here's the local vernacular. This is what a roof should be made out of, and these are the colors that we like to use because that's what we use. **Jackson:** And there can be a lot of creative freedom inside those constraints. **Geoffrey:** Exactly. It's not quite what people call a design system, which tend to be much more prescriptive and top-down. It's closer to an open-ended set of patterns that people intuit how to apply. A crude way of saying it is, can you make the right 50 built-in UI components so everything has the same look and feel? That's actually sort of what macOS and iOS do. They give you 50 components and like a 300-page book telling you what feels good and how things work. **Jackson:** In this world, in some sense, Apple—there's a lot to critique about Apple, I would argue, especially at the platform level—but this is one of the things they're best at: doing this in a way that both seems to be empowering to developers and somewhat cohesive. **Geoffrey:** Absolutely. I'm a huge fan of Apple products. I agree with you, there's lots to critique in the way that they are. I don't think that they value malleability very much. **Jackson:** I don't think they value developers very much. **Geoffrey:** That is also true. That's a whole separate conversation. But they definitely value a cohesive user experience. I don't like bad user experiences. I don't love Linux on the desktop because it doesn't feel good to use. I think there is an inherent tension between skilled professional designers crafting everything really well and people doing stuff on their own. The synthesis might be this kind of pattern language. **Jackson:** On the note of the environment and substrates, any thoughts on how this is going to evolve? I think you spent a little time at and written about Dynamic Land as one kind of extreme example, Brett Victor. How substrates of the future? Maybe an additional question to tack onto that would be that one thought I continue to have as you make all these wonderful metaphors is that software or bits broadly basically lack dimensionality, at least until we get to VR. So some of these metaphors around the ergonomics and the malleability and spatial feel and even the Post-it note, they're good, but they aren't totally exactly apt. It's a very open-ended question, and maybe just totally postulating, but any stuff like Dynamic Land that you've seen or other examples or just speculation on how maybe we actually might get more freedom in the environment substrate space as HCI changes? **Geoffrey:** Dynamic Land, for those who don't know, is basically a room-scale computing experience. The way it feels is that you sit at a table with people and you think about stuff, and you have magical materials on the table that come alive and do more than paper can do. **Jackson:** That's sort of my bulletin board from earlier. **Geoffrey:** Exactly. It's the magic bulletin board. Things start animating, you can run simulations. That's their vision. **Jackson:** I think it's projectors on the ceiling. **Geoffrey:** It is. Brett's a huge influence on my work. One of my co-authors on the Malleable Software essay, Josh Horowitz, who I've also collaborated with in other ways, worked at Dynamic Land and built a lot of their stuff. It is projector-based, but that's not the point. I think they would say. **Jackson:** I'm just trying to fill in. People can look up a video. What you just described... **Geoffrey:** I think what that team wants people to focus on is it's a prototype of how the thing should feel. It doesn't matter how they built it. What matters is you have things coming live on the table. Maybe things are even moving automatically. **Jackson:** What I'm trying to clarify is you're not wearing glasses. There's no VR. **Geoffrey:** No VR. Yes. **Jackson:** It's in the real world. **Geoffrey:** And it's really with pieces of paper. I visited a few times. I was not involved at all in it, but I got to feel what it was like to be there. The thing I found most compelling and awesome about it was you can sit around a table and make eye contact, I can see what you're looking at, and I can point at stuff. When we want to change the color of something, we just scribble on it with a pen. They have some cool examples of leaning into physics. If you want to make a spinner, you just make a spinner using a thumbtack and a piece of paper, and that's your spinner. It's this lovely starting from physicality. **Jackson:** It's combined to the tension we've been talking about this whole time. So much of your work is trying to make aspects of the digital world more like the physical world. And they're just like, "Let's literally combine them." **Geoffrey:** Exactly. I think it's a really bold and inspiring approach. It's not what I'm doing, partially because I work remotely, which doesn't work with their "around a literal table" vision. **Geoffrey:** But I do think a lot about... **Jackson:** I read somewhere that Brett said he hasn't programmed on a real computer in a long time. He only programs in Dynamic Land. **Geoffrey:** That's pretty sure that's possible. Another thing that I find really inspiring about their research approach is they really have tried to live in the thing that they were making. That's a legacy that they drew inspiration from computing pioneers like Doug Engelbart, whom they thought that the way you build a good thing is you try it out yourself for years and you really go deep on living in that space. And that's something we try to do at our lab too. **Jackson:** Very Apple-esque. That's where Brett came from. Makes sense. **Geoffrey:** It's prototype-based and experience-based. It's not theoretical. We try to do that at our lab. We call it living in the future. Sometimes it feels futuristic. Sometimes it feels like we took a step backwards by accident. But that's all part of it. Being accountable to make something that really works for you is, I think, a really important part of a design process. I find that inspiring. **Jackson:** We talk a lot. To extend the metaphor of the kitchen, on the note of the knife and tools: what is the difference in this context between a feature and a tool? In Embark, a lot of those things you're describing are tools that live in this spreadsheet. I would argue AI is only going to collapse feature and tool, but is it worth distinguishing between those two? Or more broadly, how do you think about what a tool is in this idea, assuming that you have different substrates and environments? **Geoffrey:** There isn't a hard boundary, but there's a spectrum from feature to tool. An avocado slicer is more on the feature end, and a knife is more on the tool end. The press release for the avocado slicer, written before they designed it, said: "We have this person in mind. They're a 32-year-old product manager at Google, hosting a dinner party, and they need to slice 10 avocados. We're going to make a perfect thing that does avocado slicing." What are the stages of avocado slicing? How can you make it so easy and so intuitive they don't need to learn anything to use it? It's very driven by a specific problem and scenario, with zero attempt to generalize further. A knife, on the other hand, is very different. It's a mechanism; it's an object for cutting stuff. **Jackson:** The avocado thing is more like an appliance. **Geoffrey:** Yes. Appliance is also a slippery word, but I agree that appliances are often closer to doing one thing. **Jackson:** It's "do the job for me." It's like a toaster. **Geoffrey:** Yes. **Jackson:** In this case, not literally; you still have to. But you could imagine an avocado slicer that is like a toaster: put it in the box, and it slices. **Geoffrey:** Exactly. One of the qualities of a good tool is that you can do different things with it, you can keep getting better at it, and learn to use it. There are good examples of this in software. We've talked about some of them. Parts of spreadsheets and parts of Notion have this quality where you learn this primitive and can apply it in a lot of different ways. **Jackson:** A programming language, for example. **Geoffrey:** Absolutely. Once you know a good programming language, you can do many things. Programming languages are an advanced Lego kit. **Jackson:** You mentioned a non-obvious benefit of substrate or environmental interoperability—or at least, the notion of being able to bring your own client to something. It's not the notion of trying every Notes app; it's that I've used the same thing forever. I've used the same code editor forever, which is very cool. Do you have any personal examples of that? And more broadly, what makes a great tool? What's a chef's knife of software, or even just one that you have gotten to go deep on and enjoy? **Geoffrey:** This is an aspect that I've come to appreciate more over time as I rack up years. I used to be really interested in what's the coolest new best tool that I could pick up right now and try them all. **Jackson:** The mid-curve. **Geoffrey:** Exactly. I've been through the curve. Perhaps I've made it to the other side. Something I really value now is thinking on a decades scale in terms of longevity: what are tools that I plan to use my whole life, and what's going to be worth investing in that's actually going to last that long? **Jackson:** We generally don't think about digital stuff in this way. Only now, people around our age have been using it for a little longer. When you started, it was all new; the Internet was new. **Geoffrey:** Now I'm thinking about where do my notes live so that I can use them when I'm 50? **Jackson:** Yes. **Geoffrey:** Where do my photos live? All the photos I've ever taken in my entire life are in this photo reel, and I think it's backed up somewhere. If Apple keeps being good, it should all be fine. But is Apple going to be good in 20 years? A lot of the local first, file-over-app philosophy is applying this to your data. I also think it applies to software. There's a paradox in malleable software. Wendy Mackay, an HCI researcher, did a study at MIT in the 90s where she asked people, "Why do you change your software?" One of the top reasons was, "Because they shipped an update and I didn't like it, and I just wanted to keep it the way that it was before." That's hilarious, and we've all been through this. If you come from a product mindset, you tend to think of those users as bad. They're the ones who want to stay in the past and don't want to learn the cool new stuff that we're shipping. "Why don't you want to use our cool new stuff?" But there's another way of looking at it: if I own my environment and I like it the way it is, you don't get to come in and change it. Part of owning a house is if you like the way the house is, you get to live there forever. **Jackson:** And there are affordances that come from having used this knife for five years or lived in this house for five years. **Geoffrey:** Exactly. Sometimes software feels like I've been using this knife to cut onions for five years. Then one day I show up to cook dinner, and they announce, "Hey, we've made a new knife. It's way better. Try it out now. By the way, we threw away the old knife; it's been deprecated." My thought is, "Oh, no, this is terrible timing. You could have been more polite. I didn't need the new one anyway." **Jackson:** We use these tools all day. You notice a gram of weight in a knife. **Geoffrey:** It is not worth investing and learning the thing if it's not going to stay usable and stay predictable. It's an ironic thing that part of ownership is keeping things the same as well. ## [01:17:58] Why Do the Work at All When AI Can Do It? When Should We be in the Details? **Jackson:** We texted about this beforehand. One idea that ties into this tool and environment framework is the notion of a container around code, which is what software or apps are today. It feels like the dominant frame around AI is that we now have a world where you can ask a really smart—choose whatever metaphor you want: butler, assistant, appliance—to do the thing for you, capture intent. I don't need to know how it happens. I don't need tools, apps, or software. You have this line in one of your essays: "Chat will never feel like driving a car, no matter how good the bot is." Which is great. We were texting, and you mentioned Waymo. We're almost moving from driving, then cruise control, to now just having Waymo. Where, when, and why will we actually still have tools? How do you think about where we're going as the containers around this stuff start to dissolve? One of the core things you're driving at in the "tools over apps" framework is pointing at this. In a world where you can edit code at the speed of thought, you don't need to write programming. You don't need to program yourself; you just ask the butler to do things for you. I have some of my own ideas. I think you're getting at some of them with the chat and driving a car idea. You have another example: What if ChatGPT produced a spreadsheet for you? I'd love your thoughts on where containers and tools should show up beyond just intent. **Geoffrey:** This is something I'm grappling with right now. I want to try to unpack it with you. There is a trap I'm trying not to fall into: I have a lot of creative friends who love direct contact with the work. I'm that way, too. A simple way of putting it is, I like writing words. It feels good to do that activity. **Jackson:** Independent of the output. **Geoffrey:** Independent of the output, exactly. That is a separate question from: Do I actually need to be involved in the details to get the best output in this context? Unbundling those two things is a vital first step to trying to figure out what's going on here. From the perspective of providing economic value to the world, no one cares what I feel about doing the work. It's about whether the output is good. **Jackson:** Yes. **Geoffrey:** To a first order, there might be something you can say: If the bot gets good enough, you can keep doing the thing as a hobby, but you don't need to do it anymore because the AI can do it for you. That's one possible stance. However, when you're trying to do certain kinds of creative work, being involved in the details is the only way to get to the unique output that would have come from you, and there's no shortcut. **Jackson:** I would certainly claim that with writing today, but again, I could still be underrating. **Geoffrey:** It's obvious with writing. Writing is the clearest case because writing is thinking. Coding is harder. To take myself as an example, a lot of the work I do is prototyping user interfaces. When prototyping a user interface, not only am I going to throw away the code, but the code probably doesn't really matter. What matters is how it feels to use. **Jackson:** Yes. **Geoffrey:** In that case, I'm finding a lot of use for AI in helping me do that task. I'm moving faster and I'm able to prototype and get to the interesting parts faster. However, I sometimes wonder if I'm accidentally losing opportunities to notice things. AI is making a lot of micro-decisions that I might have been more involved with before. Maybe if I had been involved in that decision, I would have had a spark of an idea that I'm not having now. I try to monitor that and think about the quality of the time I'm spending. One way I think about it is if I'm doing serious work, I don't think there's a way to reduce the amount of time I'm spending on it. That's not my goal. It's more, can I ramp up the quality of the time I'm spending so that every minute is focused on the important bits? Maybe that's a way to get better results. **Jackson:** There's an essay I read a while ago called "Reality Has a Surprising Amount of Detail." I don't know if you've... **Geoffrey:** I love that essay. **Jackson:** That's great. For example, with me editing this, I could have AI transcribe it. I do do that. I could have AI do everything. Yet I'm sitting with the same tension as you, which is, I don't know if it's just fear. I used to work with lots of Twitch streamers and YouTubers, and they would always have this feeling: "I can't let somebody else edit the videos; only I can do it." What they all eventually found is that if you take the time and train a person, you have more time. The question is, and maybe we'll come back to this, where do you want to... You don't get to spend a lot of your time on much, but where do you actually want to spend a a lot of your time. **Geoffrey:** Yes. One question I've been trying to overlay on this is: when is it worth it to get in the details, and when is it possible to delegate? Editing a YouTube video is a great example of a tricky case. The more variability there is in the output, and the more subjective and taste-driven it is, the more important it is to be involved in the details. At the far extreme of this is something like art. I'm not an artist, but I think humans being involved in the details of art is inherently part of the artistic process. On the other far extreme, let's go back to that Waymo example I was lobbing at you. Often, I just want to get somewhere, and there's not any variability in output. I know where I want to go. **Jackson:** Yes. **Geoffrey:** Driving the car, I'm not going to change my mind about where I'm going. So if I have this clear objective, it's verifiable and optimizable. However you can get me there cheaply and fast, I will take. And safely, I'll do. Literally, it doesn't matter to me; it's a black box. In those cases, it's appropriate to delegate because it's a closed-ended problem. I think this is sort of a roadmap for where I think AI coding is going to go: basically, any subtask that has a very clear criteria for what success looks like is going to be automated. They're going to train models. **Jackson:** Known knowns. **Geoffrey:** Exactly. And this surface of specifiability, I think, doesn't always match what we intuitively think of as difficult necessarily. So, I might have a goal: make this thing 100 times faster, make it 100 times cheaper to manufacture, and make it 100 times more robust. Or I could have a more insidious goal: make the conversion rate 10x. These are clear goals that can be measured. I think AIs will eventually figure out how to optimize any metric. **Jackson:** It's a math problem. **Geoffrey:** It's a math problem. It might be really difficult by human standards to do some of those tasks, but we'll figure it out. The tasks that are left for me to work on are the ones where we don't know what the answer is. **Jackson:** On one last note on the container idea, one thing you rightly got at is: where do I want to go through the work and the complexity and use tools for my own purpose? But on the other end, I'm curious what you think is broadly going to happen to software people use. Presumably, people will talk about this world where you're just going to have a master agent. Why do you even need user interfaces? Why are you still interested in designing user interfaces? Are you just designing for hobbyists who are doing this for fun? I don't think so. I'm curious, to what extent do you think about what types of use cases and applications? Maybe a simpler way to ask this is: where do ergonomics still matter? Where are the cases where chat isn't as good as something like a steering wheel? It's never going to feel the same. **Geoffrey:** I regret the steering wheel analogy as a choice a bit because in most cases, it doesn't matter that much where you steer. Maybe a better example is: chat will never feel like holding a paintbrush. We can map this distinction we're making about things where the output is known versus unknown to input-output modalities in an HCI sense: what modalities are going to be best for the kinds of interaction we want to have? If you have a fuzzy intent and you want to say it in natural language and have something go do something for you, natural language, voice, or chat is a good way of specifying that. Often, on the computer-to-human side, text is not sufficient, and we really want more visual stuff, like showing a command interface. **Jackson:** I was just travel planning on ChatGPT, and it's actually not that great; it's a wall... **Geoffrey:** ...of text is not a good UI. People know that. **Jackson:** We'll have Generative View. One other thought here, which you get at with the "what if ChatGPT produced a spreadsheet?", is that chat isn't really a computational medium, to but presumably, ChatGPT is going to start generating me UIs that I can play with on the fly. **Geoffrey:** There are layers there; maybe we'll be making new computational media within chats. Direct manipulation—by which I mean clicking on stuff and moving things, this metaphor of the screen having objects on it that you can think about—engages a different part of our brain. Precision is way more involved. You can point to stuff, and there are big advantages there. I'm really excited about hybrid modalities. An idea that I'm obsessed with is hybrid voice and pointing. I want to, Minority Report style, use my fingers and my voice simultaneously. There's a famous HCI paper from decades ago called "Put That There"; it's this really old demo where this guy is just pointing at stuff and saying, "put that there," and it moves it there. It's a perfect demonstration of the power of this hybrid modality. That's going to be really cool. There are all these different mashups, but you can basically break the problem down into the pros and cons of each input and output method, what tasks are appropriate for them, and go from there. I hope we get a new operating system at some point. **Jackson:** Using the eye tracking on the Vision Pro was pretty magical and felt a little bit like what you were just describing, or the Dynamic Land type of stuff. **Geoffrey:** Maybe it will be VR/AR; I'm not sure. I think AI might disrupt our way into a new operating system emerging. I really believe that the big deal isn't going to be a new IDE for developers. The big deal is going to be a new environment for users to live in. We'll see. **Jackson:** On that last note of precision, you have a line where you say, "Any creative endeavor where the person is reaching for greatness benefits from precise control." And as Thesis or Linus—someone we both know—says, "Great creative tools allow for virtuosity." Maybe that's in the creative sense, to your point, but there's still a lot of room for that. ## [01:29:22] Empathy & Design: Enabling "Local Developers" Who Know Their and Their Community's Needs **Jackson:** One other bit about design that I found incredibly interesting, surprising, and, the more time I spent on it, resonant, that you pointed out briefly, is about empathy. I suppose this is probably pretty different than most designers' views on design and most people's views on design. As you said earlier, the average Silicon Valley orientation is a set of quotes that I'll read in full. You say, "In some domains, the problem is obvious, and the solution is really hard. For example, the prompt might be, 'Make cheap, clean energy.' The rest is engineering. "But in my experience, most design is the opposite. The hard part is understanding the needs, and the solutions follow naturally, maybe with a bit of editing from a pro. "In these domains, I believe empowered end users will beat external experts every time. "I think it's possible to have great respect for the craft of design while also respecting that people know their own circumstances and can learn to design quite well themselves." And then finally, perhaps most strongly, "I'm pretty pessimistic that it's possible to design truly great software for someone other than yourself, especially if that person is operating in a complex environment." You talked about this at the top of the conversation regarding designing for teachers earlier in your career. The root of this, which I think comes up verbatim or close to it a few times, is perhaps we just need more people to design software. If you're right, if the hard part is actually understanding needs, why aren't more people capable designers? **Geoffrey:** To fill in that backdrop a little, the story that led me to this conclusion was designing for teachers. I was a designer-developer. We had a lot of former teachers on the team. We were doing the correct thing, talking to teachers and principals all the time. But I had never been in the classroom. I always felt I was operating off a hazy view of what a teacher's day is like. I hated how I would constantly be blindsided by new things I hadn't understood about being a teacher—things I think I would have understood if I had been one. **Jackson:** You're trying to understand the bicycle gears on a piece of paper—something like that. **Geoffrey:** I'm reading a book about it and I haven't experienced it enough. A lot of it is stuff that isn't part of the formal job specification. When do teachers do stuff? When are they on their computers? How much time do they have? I know they're busy, but what exactly does that feel like? There's all this design thinking stuff about maxing out your empathy. To credit design thinking, one of the strategies people use is not only deep ethnographic embedding, but getting as close as possible to actually doing it yourself. Maybe some people might say I should have gone and become a teacher for a year, and then I would have been in better shape. I'm sure that's true. But I also think, to push it further, maybe 100% of teachers are not going to be good designers of software. But I do believe, and I know from talking to certain ones, that some of them are. Some of them combine the lived experience and the deep understanding with a certain way of thinking and a design mentality about improving stuff. Maybe, let's say it's 10% of teachers. How many of those people are currently empowered to do anything in their software worlds? Not many. I sometimes see the mission as: 1% of people can build software today. We don't need to get it to 100; we just need to get it to 10%. That 10% is going to be distributed across every profession and every corner of the world. We just need those local experts to learn enough to do useful stuff. **Jackson:** I love this idea. It also starts to get us to how this will improve in practice, as we've been talking a lot about theory. How are we going to get more malleable software? We haven't talked extensively about the multiplayer side of this. You frame it as communal creation, this idea of the local developer. The best version of this I found was your example of the one guy in the office who is great at Excel. Most people aren't great at Excel, but you know a guy who can do it for you, which is great. There are two sets of quotes I wanted to read that I think nail this. The first is broadly on the single player versus creating what I think you call malleable infrastructures: "And there's an even more fundamental reason to think of malleability through a communal lens. We use computers together. A product team needs a single system for tracking projects. A department at a hospital needs a single system for patient intake forms. A community organization needs a single system for video conferencing. These communities are certainly not well served by one-size-fits-all applications. But the solution can't be every user for themselves. Instead, malleable infrastructures need to help communities build and maintain shared solutions to their problems." And then on the local side, "Studying spreadsheet use at companies, Bonnie Nardi found local developers, amateur enthusiasts who could write bits of code for their coworkers and guide their team up a gentle slope of spreadsheets. Malleable software can look to examples like this. Clay Shirky termed this pattern 'situated software,' describing how his students rapidly built software for their communities by taking advantage of social infrastructure or context-sensitive information. On an even more intimate level, Robin Sloan memorably described how an app built for his family could be a home-cooked meal." As we talked about, whether it's 1% or 10%, what do we need to do for these local people? You also talk about increasing software literacy in this context, which is really cool when compared to reading or literacy rates in the past. What are the characteristics of these people? Are they basically your market? What do these people need? Is it everything we've been talking about? Are there specific frames you think about in terms of the person? **Geoffrey:** I love this term local developer. Another one that I like from Alan McLean's work that I cited earlier is software handyman. I think that's a nice way to think about it. This local person knows more than you—maybe a little, maybe a lot more—but isn't a construction crew. They're a mid-tier person. Probably, you could learn to do most of what they do if you owned a home, cared, and learned it. You don't need to be a super professional. There's this middle zone, and I think that's where local developers fit in. To your question of what we need to do to empower them: Bonnie Nardi, an NHCI researcher who's done a lot of foundational work in this field, including this book called \*A Small Matter of Programming\*—she, I believe, coined the term 'local developer' in this work that was studying spreadsheet use. They noted a couple of interesting things about spreadsheets. We're going to add to the list of 10 reasons why spreadsheets are great here. They pinpointed what makes spreadsheets good for working with local developers. There were a couple of things. One is that the novice person can still do a lot of it themselves: they can enter all the numbers and only need to ask for help for the parts they can't do. When the expert comes in, they work in the same shared space where the other person already was working. They're just entering formulas that you could have typed yourself. This is not like there's the zone you do, and then the other person's operating behind some wall that you can't see. You share this space. And that lets you do is: education can happen. **Jackson:** It's not magic. **Geoffrey:** It's not magic; it's all there. You can pick up some of the formulas, you can read what they did, and maybe you can tweak it and then copy-paste it next time. There's a lot of tacit transfer that can happen from coexisting in that space. To put a point on it, in software, often what you have is the lobby where you work. Then, if you want anything changed, there's some hidden boiler room somewhere, which is the code that you never see. The developer might say, 'Sure, we'll do that feature for you.' Then they do it, and it magically appears, but you don't know it, you can't see it, you can't edit it. I think what we need to enable the software handyman, the local developer, is not necessarily just tools for those people. We need everyone to be working in these spaces where you can more easily bring in help. And I'll note, as you mentioned earlier, increasingly that help could be AI. I'm really enamored by this idea that the local developer could be your AI. That has all these nice consequences where, again, the goal isn't that you delegate to them in the end. The goal might be that you end up not using the AI anymore. If your local AI teaches you all the spreadsheet stuff you need to know, you don't need them anymore. That's actually a pretty cool endpoint, I think. ## [01:38:23] A Case for Optimism About Human Agency **Jackson:** It might be a little optimistic, but it's certainly possible. Speaking of AI, you said— **Geoffrey:** I want to say one thing about that. Often people respond to this vision by saying, "Man, you're being very optimistic about human nature. People don't want this." If you're talking short term, you totally might be right. If you're trying to make a startup that's going to make a billion dollars by next year, maybe this whole thing is a bad idea. However, I really believe that people respond to their culture and their environment in terms of how much agency we feel. The way we're brought up really shapes the way that we act. One of the reasons I care about this is that if every teenager lives in their iPhone and you can't change anything in your iPhone, then there'll be a self-fulfilling prophecy of this pessimistic view of people as passive consumers. There's a lot of forces in our society that are pushing us that way. Anything that can nudge us the other way, even if it doesn't apply to most people today, it could contribute to a more positive feedback loop. **Jackson:** Mediums shape us very deeply. I read Neil Postman's \*Amusing Ourselves to Death\* earlier this year. He makes this argument about culture being shaped by television, but his stronger argument is that we ourselves are being shaped in terms of how we think. You embody that in a line: "Customizing software in this way is an uphill battle because the tools and infrastructure that we use to develop and deploy software treat users as passive recipients rather than active co-creators." Since you brought it up, let's talk about it a little bit more. As I mentioned earlier, this is an agency story. At the end of the day, you have to believe in agency to be doing what you're doing. I am definitely sympathetic to the environment argument; mediums affect people. At the end of your piece, you're talking about culture and ask, how do we cultivate a movement towards personal agency where people want to modify their environments broadly, digitally or physically? It seems like the world is broadly moving to people not wanting home cooking; they want DoorDash, let alone software. Is there something at root, a seed of agency that you are pulling on? Or do you think that we're all plastic and you can push the pendulum either way? Maybe that's a more philosophical question at a very broad level. If so, at the very least, what types of environments or constructs do you think are going to make people? We want our entertainment to be automated, let alone software. That's the way things are going. **Geoffrey:** Bleak, huh? I can only speculate. I don't know the answer. **Jackson:** Or feel free to speak personally too, if that's easier. **Geoffrey:** I have an axiomatic belief that most people are pretty plastic on this dimension. Every experience that you have where you do something and it matters enforces the idea that what you do matters. Every experience that you have where you try to do something and you're not allowed to or it doesn't matter, reinforces the idea that what you do doesn't matter. That loop can be really, really strong. I've seen cute ideas. To take the DoorDash example, I've seen someone post a design for a DoorDash menu that had home cooking as one of the options in addition to all the restaurants. A tricky thing is that in the DoorDash example, what you're fighting against is a company who's incentivized to make you forget that delivery food is not the only option. They want you to think, "Oh, I have a thousand choices. How great. What I'm supposed to do is pick one of the thousand choices, and I'm good." **Jackson:** Same thing with Netflix. Same thing with TikTok. **Geoffrey:** There are very strong and heavy forces pushing that. I don't know the solution exactly, but I think for a lot of people, and probably for everyone to some extent, what you see on the menu and how people around you behave has an effect. I'll speak for myself: it has an effect on me. I don't think I was always a particularly high-agency person, and it's a trait I'm constantly trying to get better at. I have role models in my life; when I see them doing stuff, I want to do stuff more like that person. A lot of it is the way the education system works. "Get good grades, get into a good college, don't think too much about why you're here" is a philosophy that doesn't inspire a ton of agency. An alternate approach of, "What's a project you want to do and why, and what do you need to learn to do that project?" is a different way of thinking. Sadly, it took me until late college or after formal education to realize that's how the world worked. **Jackson:** I think that's probably common for most people. **Geoffrey:** It felt better late than never, but I wish someone had started this loop for me when I was 15 and not 25. **Jackson:** Our school system is actually probably inversely correlated to this. You're probably better off if you aren't in a traditional school system. **Geoffrey:** 100%. The more I reflect on my schooling experience, the more radicalized I get against grades and a lot of the way education works, because I think it is a very agency-robbing experience in many cases. It's tricky, though. There's a role for skill development and formal practice. I'll tell a story. One of the things that got me to where I am right now was I did this little one-week retreat at a place called Recurse Center, which they call a programmer's retreat. It's basically for a programmer who wants a place to get better at their craft or think about stuff. It's a room with a bunch of other curious, hungry people, and you go there and you do what you want to do. There's no curriculum; it's intentionally, extremely open-ended. Part of that process—most people go for about three months—is figuring out what you want to do, having some false starts, and having some existential angst. Nothing's being spoon-fed to you. It's very inefficient in that way, and you probably spend half the time in weird spirals. But at the other end, what you get is this renewed conviction and sense of, "I figured out what I actually want and where I want to go." Now I'm super, super motivated to pick up these 10 skills that I need to get there. Because I've learned. That's why I ended up going to grad school. I was not planning to do that, and I realized, okay, now I know. **Jackson:** Big gap too. Five or six years after school. **Geoffrey:** I wasn't going to do a PhD; it's like a terrible deal. You don't get paid much, you got to work really hard and write all this stuff. But I realized, okay, to execute this vision that I've come to, now I'm devouring these classes and books and stuff. For me, that sequence of: mess around, do it wrong, figure out where you want to go first, and then get a coach to help you learn, works way better. School just completely screws that up most of the time. **Jackson:** Because you're not doing it for you. **Geoffrey:** Exactly. You're doing it for someone else's made-up sequence that doesn't apply to where you are right now. **Jackson:** It's a little more tactical and less philosophical, but you've written about how we can make programming feel more like a video game. You have this little blog post about this Claude debugging GUI you built; maybe that's one very simple version of this. It's not surprising at all that lots of kids who are addicted to video games or Minecraft end up becoming programmers. But do you think that is one version of the type of context or environmental way that you can push people more this way? **Geoffrey:** An even better example in this context is Seymour Papert, an early pioneer of constructionism and educational philosophy focused on this idea: do projects, don't be fed facts. He had a whole web of related ideas. For instance, he discussed how a kid learns French: you live in France, and it happens automatically. How could you learn math like that? You just need Math Land. If you live in Math Land, you'll pick up math. **Jackson:** So, that's what video games can be. **Geoffrey:** I don't know if he would call it a video game exactly, but he thought of designing these environments, like Logo. This programming language. **Jackson:** Yes. **Geoffrey:** Not really to teach programming. Actually, his goal was more to teach math. And he thought that if kids lived in that place, immersed in these concepts, they would naturally pick up what they needed to do cool stuff. In Logo, it was, "I want to make a turtle draw a circle because that's really awesome." That's a huge inspiration for me. You mentioned some debugger projects I've done; I think that could be one ingredient. I believe that being able to intuitively visualize what the heck is going on is really powerful as one ingredient and feeling like you're seeing it, getting it, and you want to participate in it. It's literally being able to pop open the hood and see something. Yes. A big problem with software is that you can't do that. When I was in fifth grade, my teacher let me take apart a vacuum cleaner, and it was epic. I didn't really get what was going on, but it was, "Oh, my God, I can unscrew this thing and see what's there!" **Jackson:** Lots of engineers with stories like that. **Geoffrey:** Kids have a ton of agency from birth and are really naturally curious. A lot of this is creating prepared environments where you can let that take its natural course, and things will work out fine. **Jackson:** On that note, especially given the lack of connecting to the school part at least initially, you seem to be someone who has a very strong default towards hacking, tinkering, and experimenting. Where do you think that comes from? Maybe it was the vacuum cleaner. **Geoffrey:** I don't know. I think I'm the kind of person where once I get an idea for how cool something might be, I just have to try it and see, and I'll ride that wave for a day. And I get obsessed. I enjoy that initial seed of something and playing out all the implications. And where does that come from? **Jackson:** What you're tapping into is how you induce agency. You get somebody excited about something. Anyone will do anything if they're excited about it or they want something. But there's a curiosity there that is connecting in a way that this is what video games are good at: they help the kid connect in a way that they'll actually be motivated to do something. That's the question. We don't get to choose what we care about. But I'm curious: have you found a pattern in the things that are attractive to you in that way? **Geoffrey:** I love it when work that I do changes the way that other people see the world or think about stuff. To me, that part is really important. It's not just me doing something; it's other people reacting to it. As a kid, I would make weird animated videos and send them to my friends, and they would laugh at them. That was cool. Now, in the internet age, one thing I find powerful about living on the internet to some extent is you can just put stuff out there, and so many people can see it and react to it. Twitter is a medium that we both participate in a lot, for better or for worse. Seriously. The "for better" part is that I can assemble the hundred people in the world who are wanting to nerd out about this super niche thing, and we can talk about it. I can say something, and people will listen. It's global show and tell. That possibility of changing how people see stuff, I find that really exciting—the idea that ideas matter. Sure, making things real matters too, and execution matters, but ideas can be really, really powerful. It's just fun to get stuff out there like that. **Jackson:** There's obviously a risk of on this front, but ideas maybe have never mattered more, which is a cool thought. ## [01:51:09] AI's Impact on Malleable Software **Jackson:** A couple of things I wanted to hit quickly before we wrap up on the malleable software stuff: I brought up the AI thing hanging over so much of this. You had a tweet in March 2023. You said it seems likely to me that all computer users will soon have the ability to develop small software tools from scratch and to describe changes to make to existing software they're already using. By the way, one funny thing is that a lot of software is actually built like this under the hood. Engineers don't make things from scratch these days. They find libraries and assemble them into products where the seams are hidden from the user. We were talking about Foam. In the Foam model, the modules are exposed, giving me the power to open the hood. One view here would be that all software is just going to be malleable because all software is going to be amalgamated and not handmade. Maybe that's optimistic, or maybe that's likely. **Geoffrey:** I disagree. Or at least, I think that's not the whole story, and I'll explain why. In our SEI malleable software, we have this analogy to make this point: Imagine there's no more home cooking, like we've been imagining, and you can only eat at food courts. There are maybe 200 things you could order, but that's the list. AI is a sous chef. AI is this intern, maybe a brilliant chef, whom you've now hired for 20 bucks a month to help you out. You bring the sous chef to the food court; what are they going to do? You're still stuck ordering from the menu. They don't have access to the kitchen. The point here is that you don't control the production of the thing at all. What you're handed is a finished artifact. **Jackson:** Sorry to interrupt. Just to clarify, this shift doesn't necessitate more software being open source, and that's why it's still the food court. **Geoffrey:** Exactly. You're jumping ahead a little bit. My point is that there are other things that need to change in the software world to actually take advantage of AI, and software being open source is one of them. It's true, if all software was open source and you combine that with the fact that AIs can code, that's a pretty good start. Because now, if I want to change something, I theoretically just pop it open, edit it, and I'm done. **Jackson:** Yes. **Geoffrey:** I still think there is more to the picture. The more that you can do without coding, the better. There's a lot to figure out. If I want to share my version of the app with you, how do I send it to you? And then how do we decide? Are we going to keep using it together, or are you going to go make your own remix now? How does that work? Even open source is necessary but not sufficient. It's an awesome thing for the world. But if you've ever tried editing an app that you're using that was open source, it's not easy for a lot of different reasons. Even getting it running on your machine might be hours of work. There are all these barriers. I like to flip it: how do you design for everyone having a local developer or a sous chef? Assume they have a chef. **Jackson:** Maybe even a world-star Michelin chef, but who's still operating. **Geoffrey:** The answer is everyone has a kitchen. If all of my software was built in a way that I have little tools composing, and back to the earlier story, when I built that little tool while making this essay, we have a malleable environment. The whole point is that you do your AI coding, take the thing it built, bring it into your kitchen, and combine it with the stuff you already have. That's more of the ethos we need. We need to figure out how to do that well to actually take advantage of it. **Jackson:** It relates a tiny bit to the notion that we're all going to have an MCP server. There's more of this: how do I create my corner that can interoperate with the world? **Geoffrey:** You hit the nail on the head. How do you manage your own context? How do you manage all of your stuff? Because AIs love context. They love knowing about you. But it's creepy that Sam Altman knows so much about me. I want to have my own little thing that knows everything about me. I've made a personal assistant. My wife and I have an AI assistant that I made a couple of months ago, and we use it to help run our lives now. The more it knows, the better it works. I want to tell it how our weeks run, what food we like to eat, when people are visiting. That's a lot of stuff. This pattern where you have your little world of stuff about you and you bring tools to it is a really powerful thing you're pointing to. **Jackson:** With the thing you're referencing, which I'll link to: you made this little Telegram butler who speaks to you formally. That probably is pretty dang close to the prototype of our agent. We are going to have intelligences that do stuff for us and with which we have intimate conversations. And it's going to need to be able to control and internalize your world in a way that works. **Geoffrey:** 100%. And, this is an instance of malleability in my own life. I love that I control how that thing works. I know how it works. It doesn't do random stuff I don't want it to. It doesn't change what I don't want it to. I can constantly add new stuff to it. It's like I have a butler that I can train over time and make it do what I want. There are no limits to what I'm allowed to do with it. This is so different from Siri. The crazy thing is I did this whole project—I had a baby last month, and this was vibe-coded at 2 a.m. with one hand. The fact that I have software engineering training helped. It's not that anyone could do this right now, but the things you can do for yourself that are possible on the side are absolutely different than they were. This cost-benefit calculus is shifting really rapidly. It's something I'm feeling in my own work. **Jackson:** Two very empowering and interesting ideas there. One is, perhaps in what area will malleability be more important than our personal super intelligences? Number two, going back to the conversation we were having around agency and whether people are going to care: we've come back to this theme that increasingly more and more things can be automated for you. But also, we are moving more and more into a world where the floor is getting lower and the ceiling is getting higher, and the possibility space is getting wider. If you just lean forward, even the video game meme of leaning forward in your chair, if you lean forward just a tiny bit. If the non-technical person couldn't Vibe code that assistant two months ago or now, they definitely will be able to in six months. And that ramp is just going to keep going. **Geoffrey:** Absolutely. And I love what you said about these personal assistants are going to have huge influence on our lives. This is already happening. The Twitter algorithm affects my thoughts a lot. And that's one of the reasons why I think, for example, BlueSky, pick your own feed algorithm, is a great example of a certain kind of malleability. **Jackson:** Yes, yes, yes. **Geoffrey:** It's a lever you have over how you want to live, **Jackson:** It's a lever you have over your... the The person who's literally hardwired to your attention and has a switch on it. **Geoffrey:** Exactly. **Jackson:** Right now it's Elon. **Geoffrey:** Exactly. And I think a lot about my susceptibility to the feedback loops in my life and how I can. I want to figure out my values, pick my feedback loops, and then have my loops support me in enacting my values, not be constantly fighting. **Jackson:** That's agency right there. **Geoffrey:** That is absolutely. And I think it's like Odysseus tying himself to the mast. It's like, how do you set up the system in advance so you can get the outcome you want? **Jackson:** Brian Johnson talks about this in an old essay. Now he's gotten more extreme, but he literally was like, "I revoked evening Brian's eating privileges or decision-making privileges." **Geoffrey:** That's not a choice I'm going to make, but I like that he can make it. He can set up that for his life. **Jackson:** This is the possibility. This is this is the possibility. Yes, yes, yes. ## [01:59:03] Commercial Incentives and Ecosystem Change **Jackson:** My last question on this that we haven't really talked about, that is a huge boogeyman around everything you've explored around malleable software and whether it can really happen, is incentives, and commercial incentives specifically. As we talked about, you need tool makers, you need infrastructure makers, you need kitchen makers, exploiting all these metaphors. You need them to actually want to and be incentivized to enable this type of stuff. You give the example of Chrome early on: one decision made a long time ago enabled so much in the way of browser extensions. Before we explicitly talk about the commercial piece, which is super important, from a super baseline incentive standpoint, I've found myself coming across this a lot in technology where there's a temptation to believe that people will do something for an ideological reason. You run into this in crypto a lot. My favorite example is just the great genius of Tesla was: don't buy the electric car because you're a tree hugger; buy it because the car's sick. And that's how you, same thing with fake meat, whatever. You're not just relying on ideology. What do you think the most compelling reasons, we're talking abstractly and it's probably hard to be too specific here, what do you think the most compelling reasons for toolmakers, app developers, infrastructure builders to actually think in this way, or at least read the piece and want to buy into this? What are the ways that they should be selfishly incentivized to do this? **Geoffrey:** This part is really important, and it's a part that I have much less confidence on. It's less a narrow technical question; it's a very tricky, almost economics question. **Jackson:** It's a coordination problem. **Geoffrey:** There are some simple reasons to point out, though. You're a software developer. You have 10,000 bugs in your backlog or 10,000 feature requests. You're in a meeting, and your customer is saying, "Can you add this feature that competitor B has? And if you don't, I'm going to go with them." Your answer is, "We have restricted resources; we can't do it." Wouldn't it be nicer if they could just add that one thing they needed and stick with you? This is not a new idea. Plugin systems exist for lots of apps for precisely this reason: you can serve the long tail by outsourcing a bunch of stuff to external developers or even to users themselves. That's a win-win for users and devs if you let people do more on their own, so you don't need to do as much work. **Jackson:** Or even in some cases, you have a product like Obsidian where the product, in many ways, fundamentally one of its main selling points is accessibility. Those are maybe more\* rare today, but they do exist. **Geoffrey:** Obsidian takes it a step further: it's a more open system because the data is portable. Most plugin ecosystems have this catch: it's a proprietary plugin ecosystem that's trying to increase the lock-in around a single developer. That's why it's good for the developer. **Jackson:** Right. **Geoffrey:** But at the same time, that's a lot of what damages; the reason apps are siloed is because each app developer is trying to hold on to their moat. **Jackson:** Right. This is getting to the commercial part a little bit. But if I'm Notion or Figma, why would I possibly want people to use other clients? My entire business is built on the fact that I have lock-in. It's not exactly the plugin idea, but it's hard not to stare at this stuff. Google Docs. Maybe a little; that's not really where Google makes money. Maybe a little easier. Google has some of that ethos. But with most software, to the extent it's going to have a moat, it's not making the software anymore. It's the network effects. I want it to be less, or maybe not malleable, but I certainly want it to be less interoperable in theory. **Geoffrey:** If you look at the greatest success stories here, I think of the web and email. Email is not owned by any company. The web is not owned by any company. There are players in this ecosystem: email clients, email hosts, and different websites. The through-line there is that no one value-captured the whole ecosystem. It was created in a context where Tim Berners-Lee decided, "I'm not going to become a billionaire off this. I'm just going to let it happen and try to not capture the value." Ultimately, someone has to take that move: provide tremendous utility to the world in an open way where they're not capturing the value. Then other people can live on top of that. **Jackson:** But is there a more realistic template? Yes, hopefully, you have amazing things like that happen. Is there a more realistic template in commercial open source? Is that the closest thing we have here to commercial models that make sense here? You're incentivized to build a relatively open, malleable thing in part so that it will actually succeed. And by the way, you can, like a lot of open source companies, monetize laziness, monetize people hosting, and monetize being the best and first client for it. That was my best attempt in trying to think about it. **Geoffrey:** I think it's a great comparison. I'm not sure if I'm reassured or terrified by that comparison. Commercial open source is really hard to do well, and there's a lot of cases of people not getting it quite right, making awesome things, and failing to make great businesses out of it. I think there's some comparisons there that make sense. ## [02:04:17] Research and Ink & Switch **Jackson:** We haven't talked about it a ton. You have been a part of at least a couple of notable, interesting, research-oriented groups: first MIT in your PhD, now Ink and Switch. This is a strange path, especially relative to the world that you are often peripheral and adjacent to—though perhaps not broadly strange, given Ink and Switch is quite unique. My intuition is that most people doing really practical stuff in software aren't usually doing it in PhD programs. I'd love for you to talk about those two worlds and why they've been empowering for you. How do you think about research as the right long-term place for you? Is it a temporary thing? How do you think about that mode, having done it now for five or six years? **Geoffrey:** I did a Ph.D. for four years, and then I've been at Ink and Switch throughout that and since finishing. **Jackson:** What actually is Ink and Switch? **Geoffrey:** It's a bit hard to explain. Ink and Switch is an independent research lab. It's privately funded, and we think of ourselves as bridging academia and industry. We're not a startup; we're not shipping a product today, and we're not trying to make a huge return on a VC timescale. On the other hand, we're also not an academic lab. We're not just interested in figuring out the truth about the world. We want to change the way the industry works on a medium-term timescale—five to ten years away, not one year away. **Geoffrey:** The lab was started by the founders of Heroku, a successful startup. My understanding is they felt that a startup wasn't an appropriate vehicle, incentive-wise, for producing the outcomes they wanted to see in the world. That gets to how I think about what institutions I want to be part of or affiliated with. It's all about what incentive loops are closest to the ones I want guiding my work. Most jobs have strings attached, and you just got to pick the best strings. Where's the money coming from? What does the money want to happen? On one hand, startups are amazing because they have such a good feedback loop with reality. If the thing is not useful, it will die, and that's it. I love that. Reality check is amazing. The problem is you can't dream beyond. It's really hard to sell something that people don't want yet, but that you think they should want or that you imagine a world where they could want it. You don't have time to play that out. On the other hand, in academia, publishing papers is often the main goal. The number of good published papers you have is how your career is judged. There are a lot of problems with that, especially in HCI. **Jackson:** As part of your thesis, you built at least three—I don't know if you would call them apps—but very real software things. **Geoffrey:** Yes. **Jackson:** You wrote a nice thesis presumably, too. **Geoffrey:** But it takes a lot of work to do. Some of the problems I see with HCI academia are there's often a separate conversation happening in the academic world that doesn't influence what's happening in tech enough. **Jackson:** Right, right. **Geoffrey:** There's great work happening, but often the transfer function is poor. Especially when things are moving so fast, it can be hard. There are also certain ways the bar that a paper has to meet to be published is both too high and too low. They care about rigor that I don't care about, but they also don't care if it's interesting, for example. What we try to do at Ink and Switch is create incentive loops for ourselves that will lead to change in the world of the kind that we want to see. We try to think about how we can change the minds of designers and engineers who are building software today. What artifacts can we put out into the world, like open source or spin-out products, that people will actually use and build on top of? For example, the lab has this data layer called AutoMerge that is maintained as an open-source thing that people can build software with and build on top of. It's part of the local first movement, providing this as a public good to the community. We think of a portfolio of ways of influencing the world that hopefully nudge things the way we want them to. Our unique advantage is not that we're smart or talented—although I hope that we are those things—it's that we have a different structure. The fact that we aren't VC-backed and being a startup allows us to go places that other people can't go, and allows us to think about what would be the substrate, not what is the app. I think that's a cool opportunity to have a unique and differentiated contribution to the space of ideas people are talking about. **Jackson:** We briefly talked about ideas earlier and the importance of sharing them. To what extent do you think about the near, medium, and long term? Your research and writing is deeply inspirational, presumably is having real impact, and that could continue to compound. There's also a world where you could return to building production software, or however that's going to look in the future. You spent five years at a startup at the beginning of your career. I'm not asking for your imminent career plans, but how do you think about that trade-off? **Geoffrey:** I love both sides of it. I probably want to ship to users again eventually; I don't know when. There are real trade-offs. One of the biggest dangers of being in an idea space is, as I like to say, how often are you being slapped in the face by reality? At a startup, it's every day. You go try to sell stuff, people don't buy it, and you realize that's not how reality works. Your mental model is being really aggressively updated. **Jackson:** You get that from tweeting too, maybe. **Geoffrey:** Tweeting is a different loop. You need to have cool, interesting, novel, insightful ideas, for sure. There's a thing you're optimizing—sometimes not even novel. **Jackson:** You need to say things that seem contrarian, but that everyone actually already believes. **Geoffrey:** That's a perfect algorithm. You've cracked the code. In a way, that encapsulates one way I think about my own work. I have put in the work, and I know how to get Twitter likes. Does that matter? Is it aligned with the change I want? Sometimes, yes. There is value in knowing how to sell an idea and knowing how to get people to understand it. But I don't want that to be the only loop guiding what I do. You want a good mix of incentives driving you. Something we're trying to do more at our lab is have more loops with real people. We tried a project last year to have astrophysicists write science papers and do science in our tools. It didn't go that well. We learned a lot. They weren't able to do it yet, so that was a slap in the face from reality. We have a project ongoing right now where we're trying to help kids collaborate better on game development. We're running tests with teachers and, soon, students in classrooms. Again, we're saying we want to make collaboration more accessible. Is it? Let's find out. Even in a more researchy sense, that's totally essential. ## [02:11:46] ChatGPT as a Muse **Jackson:** Totally different topic. You have a post on ChatGPT as a muse and not an oracle. It was pretty prescient. It was back in 2023, you said. I've been wondering if this framing is missing a different opportunity: the notion of ChatGPT as an oracle or search. What if we were to think of LLMs not as tools for answering questions, but as tools for asking us questions and inspiring our creativity? Could they serve as on-demand conversation partners for helping us to develop our best thoughts as a creative muse? Any reflection on that piece? How has your use in that way evolved? **Geoffrey:** I'm glad you brought it up. I've been thinking about it recently. From my personal perspective, it's become increasingly true. I wrote that because I was frustrated by people thinking that an LLM was supposed to be used as a fact database, when it's actually really bad at being a fact database. They're getting better, but it's a boring way of using them. **Jackson:** Andrej (Karpathy) wrote that piece a while ago about hallucinations being the point. **Geoffrey:** Exactly. Some of the most interesting moments I've had with LLMs are treating them as a thought partner. Sometimes, if I want to talk to someone, it's late, there's no one around, and I want someone to shake up my thoughts and throw some new seeds into my thought space. An LLM can be a great way to get more entropy into my thoughts. Maybe it's as simple as "ask me questions." There's an idea I love from Gordon Brander, who has a great blog and has thought a lot about creativity. He points out that there's a deck of cards Brian Eno made called Oblique Strategies. I don't know if you've heard of it. When you want to be creative, it's a deck of cards, and you draw one. It has a very vague koan, a saying. **Jackson:** My friend Sean has this deck. **Geoffrey:** I don't remember them exactly. It's something like, "Flip it upside down." The point isn't that it tells you what to do. It's a seed that intersects with the current contents of your mind to shift your perspective. What's funny about that is a deck of cards can do that, and an LLM is a super, super-powered version of that thing. The bar is low for creativity support tools to shake up our thinking, and an LLM can probably do better than a deck of cards. **Jackson:** One of the things I noticed in that piece is that my intuition would say you are incrementally better than the average person at prompting this type of behavior. So much of LLMs, as we talked about, is adding in context. Have you found yourself prompting more aggressively in this way, or have the models just gotten better at this so you don't need to guide them? **Geoffrey:** I'm not sure. I probably still do the prompting reflexively and should test if I can remove it. A very common thing I do is say, "Ask me questions about what I'm thinking before telling me your thoughts." Or a simple technique I like is to say, "Give me 20 options, not one or three." Because with LLMs, the speed and breadth of generation is a huge asset. They're really good at generating options. zero cost to generating 20 options instead of three. It's instant. **Jackson:** With a person, it would be super high. **Geoffrey:** It'd be super. **Jackson:** We're not used to that. **Geoffrey:** Exactly. I love generating these long lists. Usually, most of the options are bad. Maybe there's one that's good, or more often, parts are good in a few of them. I see this as if we had a brainstorming session with 50 Post-its on a wall, and I apply my taste to extract the best ideas from that corpus. That's another example of that. **Jackson:** I like that. **Geoffrey:** You can definitely get pretty creative with playing to the model's strengths. **Jackson:** We could talk about that all day. ## [02:15:34] Working at MUBI and Solving the "Too Many Things to Watch" Problem **Jackson:** LinkedIn dive. You worked at MUBI from 2011 to 2013. **Geoffrey:** Whoa. **Jackson:** Throwback. Designing the streaming platform, it sounds like. **Geoffrey:** Yes. **Jackson:** They just raised $100 million from Sequoia. **Geoffrey:** Really? I didn't know that. **Jackson:** Billion-dollar valuation. Yes, Andrew Reed at Sequoia. **Geoffrey:** That's news to me. **Jackson:** News to you? **Geoffrey:** Wow. Any congratulations, MUBI? **Jackson:** Curious. I like movies. I actually am not a MUBI user, although I've played around with it a little bit. How did that happen? Any thoughts? That was a long time ago. **Geoffrey:** No one's ever asked me about this in an interview. I have fun stories from that time. I was in college and didn't know how to do much that was useful. I was looking for an internship. I applied to the big tech companies and bombed my interviews. They said, "You can't really program." So I found MUBI, which was a tiny startup then. At the time, it was under 10 people. They were based in London. What was really fun about my time there was that everyone was too busy to help me or teach me anything. I did get great support when I asked people for help, and they were really lovely people. But it wasn't an intern program. I just showed up, there was a desk, and I sat there wondering, "What can I do?" I accidentally ended up working on really important stuff because there weren't enough people. **Jackson:** Did they have technology people? **Geoffrey:** They did. They had great tech people. The part I helped out a little with—I don't want to overstate my contribution; it might have even been net negative, if you've ever worked with an intern—was this: the CEO had a really smart idea. It's hard to compete with Netflix because Netflix has a big catalog, and it's expensive to license all those movies. So, what if you competed on having a small catalog? The way you do that is by focusing on curation and taste, which Mubi was already doing. The key insight was that everyone's tired of Netflix because you can't pick what to watch; there are too many options. He had this brilliant idea: we're going to show you one new movie a day. It'll be there for a month, and if you don't watch it in that month, it's gone, off the platform. This created urgency, and people with taste were recommending movies to you. It was a brilliant win-win: users get a better experience, and it's more affordable for the company to license great content they couldn't afford otherwise. It was beautifully positioned against the existing player. I had a front-row seat to seeing that happen and watching this company go through what felt to me like bleak times. To see that creativity spark a reinvention of the thing was really cool to me. **Jackson:** I think they have 20 million users now, and the CEO just got a crazy profile in Variety\*. **Geoffrey:** Go Mubi! **Jackson:** You'll have to check that out. **Geoffrey:** They're doing great. ## [02:18:27] Japan's Culture of Care **Jackson:** You grew up a little bit in Japan and presumably go back there. It's one of my favorite places. You have a Japanese mom. I'm curious how Japanese culture or living there has inspired you, how you think about things. Any influences there? **Geoffrey:** It's a big part of my identity. I spent middle and high school there. I love having been exposed to both US and Japanese culture as a kid because they're really different. The level of care you experience in Japan is ridiculous, and I think a lot of people see this when they visit. It shows up in the cleanliness of the benches in the train station. In my experience, it's pretty hard to get a bad meal in Tokyo because even at an average place, they care, they give a shit, they lock in the details, and they take pride in the work. It's common in Japanese society for people to take pride in labor that might be considered low status in the US. The janitor has an important job to do, and they're part of a society. **Geoffrey:** I had some cool experiences. In elementary school, I was living in the U.S., but my mom would take me and my brother to Japan every summer, and we spent a whole month at a Japanese elementary school, which was so cool. It's different. A recent documentary came out about this. You serve lunch to your fellow students, you clean the school, and the students are running 'round the floor with rags. There's much more to the agency point and what we were talking about with education. In some ways, Japanese education is worse in agency development, I would say. But in these interesting ways, it also has these amazing glimmers of developing a sense of responsibility and membership as part of the whole community. This is vital, I think, to holding the whole—the way society works—together. **Jackson:** When you take the most extreme examples that seem totally insane in Japan, like the little kids who are able to ride the subway alone or the little kids in the little hats—American society could never support that. It seems to be rooted in a great deal of every person in that society feeling they have some kind of responsibility to care for **Geoffrey:** On the theme of agency, there's a stereotype of Japan: a bunch of salarymen, business guys in boring meetings, listening to their elders tell them what to do. Which is partially true. There's a vibrant culture of indie businesses and entrepreneurship. The restaurant scene in Tokyo is amazing because there are all these restaurants and bars that have six seats. Many of them aren't famous and aren't the kinds of places tourists would go. There are these tiny businesses, and they're popping up everywhere. Some of it's zoning law. **Jackson:** One of the best things about it is the three-dimensionality of it. **Geoffrey:** You go up to the eighth floor of an office building, in the back of the hallway, and there's a hidden place. There are so many cool no-sign spots. There are a lot of reasons why that is, around urban design. There is an ethos of taking pride in your work and doing something not to make the most money. The goal isn't to delegate it or scale it. The person wants to run one bar with six seats, that's what they want to do, and they're fine with that. **Jackson:** The goal is to keep doing it. **Geoffrey:** The goal is to do it well, to do it excellently. American culture, especially in tech, is all about growth and scale. There's something beautiful about that: keeping it small. **Jackson:** All about ends, rather than means. ## [02:22:15] Mastery and Variety **Jackson:** We were having a conversation at lunch before this. It sounds like you are a person who is dispositionally someone who wants to do lots of different things; I am as well. I'm curious how you have related to various levels of depth in your life. I was going to ask about music because you seem to have a lot of instruments in the background on Zoom calls. It sounds like you play the cello. That's one area, for example. Software has been another area. You're also someone who's not staying in one place. I'm curious how you relate to that, especially in the context of the Japanese philosophy of mastery. How are you holding that? Have you found the right balance? Do you hope to have more mastery in the future? Are you trying to go wider or deeper? **Geoffrey:** It's something I struggle with. I love doing different stuff. I hate job descriptions or job titles; it's always so funny. **Jackson:** We hate legibility. **Geoffrey:** People try to ask, what's the difference between a UX designer and a product designer? Who cares? Just make great stuff and learn the skill set you need to do it. **Jackson:** Making stuff is a strong consistency for you. **Geoffrey:** I do respect masters of crafts a lot, and there are degrees to it, of specificity. I aspire to master in depth a cluster of skills around designing, developing, writing, and communicating. That is the bundle necessary to do the work I want to do and see the change I want to see in the world. A PhD program in grad school in computer science is not about being a better programmer. That's taken as a prereq. A lot of research is about a bundle of skills: thinking, writing, reading—reading papers is a skill—making stuff, and prototyping. I love that it's this coherent basket that you need to have. You need to be a Mario in Mario Kart; if you're well-rounded in all those places, you can do a great job at this craft. This holistic craft. **Jackson:** This holistic craft. ## [02:24:34] Joy and Clarity as a Parent **Jackson:** You had a child for the first time very recently. What do you know now that you didn't a few months ago? **Geoffrey:** It's been really fun, almost more fun than I could have expected. The highs have been so high. It creates a really interesting perspective to know that there's this really important thing that matters most to me now. A lot of stuff that mattered before to me still really matters, but in a slightly different way. I can care about it, but ultimately there's something more important. I think that can sometimes be a freeing way to think about something. You take a little bit of the pressure off. I want to do great work. I want to change how the world thinks. But also, if I see my daughter smile and play with her, that's a good day. That's been different for me. ## [02:25:30] Expressing Care Through What we Make **Jackson:** We were also chatting at lunch about how you had observed there's something kind of pure in a pro athlete, like a basketball player. In terms of, does what they do actually matter? It's just a ball game, just a sport. But they're perfectly honest about how much they care about this thing, regardless of how, quote, unquote, important it is for the world. We were also chatting about your brother, who's a doctor. An athlete, a doctor—they know right away, when they're 20 years old, what they're going to invest all their time in and what they're going to care so much about. You also tweeted, I think today, "Working with people who have high standards is tiring in the best way." I was marinating on that combination of ideas. We had also talked about my conversation with Nabeel (S. Qureshi) and this notion going to care about, what you are going to choose to care about. You only get so many things to care about. My final question is, what else is tiring in a good way? Why do you labor and tinker and experiment and iterate and push beyond the default? It's a little bit like what I asked you earlier. **Geoffrey:** I'll tell you the story of where that tweet came from today to get at this. The secret is we're recording this before this essay about malleable software comes out. You've seen a draft, but it's not done yet. It's been a long process. We've been working on this thing for months. There have been a huge number of drafts, and we're close, but we're not done. One of the things that we believe at Ink and Switch, and that I've learned from working with the people I work with now, is there is tremendous value in just sweating the details and caring way more than everyone else. It's hard, and it sucks. You can feel the finish line get closer and then further away, and closer and further away. **Jackson:** 90% done, so I only have 50% left. **Geoffrey:** You wish it was over. Sometimes I personally fall prey to just getting it out the door, which is what the startup does. With a startup, you don't have time; you just have to cut it, go, and iterate. One of the nice things about writing essays at a lab is that you can do it differently. You can say, "No, we're going to wait until this thing is perfect, then we're going to put it out there, and people will notice." That's something I've learned from some of my collaborators too: every detail, from the photography to the margins to every single word. The individual choices won't be visible, but you can feel when it's all there. It's a very special feeling. You see that in Japan at a restaurant often. It's incredibly rewarding to try to do that and aspire to it yourself. It's tough, but having people around you who enforce that for you is helpful. Some of this is picking the right people. I've sent this essay to a number of friends whose opinions I really respect, and I've gotten some pretty negative feedback, or feedback like, "This part is great, but here are 10 deep flaws." It makes my day because it's so cool that I found people I trust. When I read what they say, I nod. I respect their taste enough to say, "Okay, we gotta try again." That's really rewarding, and it's something I want to keep getting better at. **Jackson:** If you're going to make anything, why not make it wonderful? **Geoffrey:** Go for it. Exactly. Steve Jobs has a lot of good stuff to say about this. Showing people you care or expressing your care through things you make—I forget who, Steve or Jony—is a lovely way to live. **Jackson:** That's all I got. **Geoffrey:** Thanks. This was really fun. **Jackson:** This was wonderful.