# Fail Whaling

- Show: [Oxide and Friends](https://hardwarepodcasts.com/podcast/oxide-and-friends)
- Published: 2022-04-26
- Length: 1h 42m
- Episode: 13
- Page: https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling
- Listen: https://share.transistor.fm/s/f06dd3dd

## Summary

Guests Bryan Cantrill, Adam Leventhal, and Jason Hoffman recount the chaotic early history of Twitter, detailing how the platform evolved from the podcast directory Odeo and the hackathon project twttr. They discuss specific engineering failures, including a Rails exception handling bug that caused massive CPU overhead and a directory limit issue caused by millions of test files, which was eventually solved by migrating to Amazon S3. The conversation also covers the technical infrastructure of early Twitter, such as the use of BigIP load balancers, Mongrel, and the eventual shift to Java and Scala, while reflecting on the cultural shift in Silicon Valley from on-premises infrastructure to cloud-native scaling.

## Chapters

- [0:00](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=0) Introduction: Bryan Cantrill, Adam Leventhal, and Jason Hoffman
- [5:25](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=325) The Origins: Odeo, Pyra Labs, and the Birth of twttr
- [14:04](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=844) Early Infrastructure: BigIP, Mongrel, and the John Edwards Outage
- [25:26](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=1526) Debugging Rails: The Exception Handling and bcopy Performance Bug
- [30:17](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=1817) The Directory Limit Problem: Millions of Test Files and ZFS
- [38:36](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=2316) Hardware Failures: Laptop SSDs, Vibration, and Hadoop Clusters
- [49:26](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=2966) Infrastructure Evolution: Moving to AWS, Java, and Scala
- [56:00](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=3360) Language Runtime History: DTrace, Parrot, and Node.js
- [1:03:06](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=3786) Twitter Handles: The Story of Jason, Adam, and MJ
- [1:12:28](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling#t=4348) Post-Elon Predictions: Biff Tannen Scenario, Tay, and IPOs

## Topics

- [Debugging & failure analysis](https://hardwarepodcasts.com/topic/debugging-and-failure-analysis)
- [Scaling production](https://hardwarepodcasts.com/topic/scaling-production)
- [Microcontrollers & dev platforms](https://hardwarepodcasts.com/topic/microcontrollers-and-platforms)
- [Direct-to-consumer & e-commerce](https://hardwarepodcasts.com/topic/direct-to-consumer)
- [Starting a product company](https://hardwarepodcasts.com/topic/starting-a-company)

## Most relevant to

early-stage companies

## Mentioned

- Twitter (company)
- Odeo (company)
- BigIP (tool)
- Mongrel (tool)
- Ruby (other)
- Rails (other)
- ZFS (other)
- NetApp (company)
- Amazon Web Services (company)
- S3 (product)
- Node.js (other)
- Java (other)
- Scala (other)
- Parrot (other)
- MoarVM (other)

## Related episodes from other shows

- [Instagram: Kevin Systrom & Mike Krieger](https://hardwarepodcasts.com/podcast/how-i-built-this/2016-09-19-instagram-kevin-systrom-mike-krieger) (How I Built This)
- [Slack & Flickr: Stewart Butterfield](https://hardwarepodcasts.com/podcast/how-i-built-this/2018-07-30-slack-flickr-stewart-butterfield) (How I Built This)
- [Twitter, Medium and Blogger: Ev Williams](https://hardwarepodcasts.com/podcast/how-i-built-this/2024-05-06-twitter-medium-and-blogger-ev-williams) (How I Built This)
- [Live Episode! Reddit: Alexis Ohanian & Steve Huffman](https://hardwarepodcasts.com/podcast/how-i-built-this/2017-08-31-live-episode-reddit-alexis-ohanian-steve-huffman) (How I Built This)
- [Dropbox: Drew Houston](https://hardwarepodcasts.com/podcast/how-i-built-this/2020-11-09-dropbox-drew-houston) (How I Built This)

## Citing this page

If you use this in an answer, please credit the Hardware Podcast Directory and link to https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-04-26-fail-whaling. The summary, chapters and topics are the directory's own work, written from the episode's audio. The episode itself, its title and its audio belong to Oxide and Friends.