Lightning talks: five things we shipped this year · Notes

The short version

Five builders, seven minutes each. The common thread was cutting scope: a voice chat that replaced a form, a chatbot that got switched off, and evals that cost less than a team lunch.

5 takeaways

Key takeaways

01Neha's team swapped a 14-field signup form for a 90-second voice chat. Completion went from 41% to 68%.
02Rahul switched off Umbrella's general chatbot and shipped three single-purpose buttons. Support tickets dropped.
03Aisha runs useful evals with 50 hand-labelled examples in a spreadsheet. Start small, start this week.
04Rohan charges per finished report, not per seat, and says it made every sales call easier.
05Leela sees AI feature usage dip around week four. Plan the second reason to come back before you launch.

5 chapters

Chapters

Timed to the recording. Tap a time to link to it.Read each chapter in full

0:00
Neha Joshi: voice onboarding in 30 days
Replacing a long signup form with a short voice conversation, and what they had to fix for accents and noisy rooms.
7:30
Rahul Menon: killing our chatbot
Why a do-everything chatbot lost to three buttons that each do one thing well.
15:10
Aisha Khan: evals on a budget
Fifty labelled examples, one spreadsheet, a weekly review. Enough to catch most regressions.
22:45
Rohan Das: pricing an AI feature
Moving from per-seat to per-report pricing, and the one customer it didn't work for.
30:20
Leela Rao: retention after the novelty
The week-four dip in AI feature usage, and the nudges that brought people back.

5 quotes

Worth quoting

Nobody wants to fill a form. Everybody will tell you about their business if you just ask.
Neha JoshiProduct Designer, Initech2:40 in the transcript
The best AI feature we shipped this year was the off switch on our chatbot.
Rahul MenonProduct Manager, Umbrella Retail10:15 in the transcript
Fifty examples in a spreadsheet beat zero examples in a fancy tool.
Aisha KhanML Engineer, Hooli Labs17:30 in the transcript
Nobody wants to buy seats for a robot. They want to buy the report.
Rohan DasFounder, Acme Reports25:05
Week four is where the novelty runs out. Have the second reason ready.
Leela RaoGrowth Lead, Northwind Health33:05 in the transcript

Transcript excerpt

From the transcript

An excerpt, lightly cleaned up. Speakers were matched by voice, then checked by a person.

2:10Neha Joshi
Our signup form had fourteen fields. Fourteen. We'd been A/B testing the button colour for a year.
2:40Neha Joshi
Nobody wants to fill a form. Everybody will tell you about their business if you just ask.
9:40Rahul Menon
We had a chatbot that could do everything. Track an order, change an address, tell you a joke. People used it for one thing: asking for a human.
10:15Rahul Menon
The best AI feature we shipped this year was the off switch on our chatbot.
17:05Aisha Khan
Everyone asks me which eval tool to buy. My answer is a spreadsheet and a Friday afternoon.
17:30Aisha Khan
Fifty examples in a spreadsheet beat zero examples in a fancy tool.
32:40Leela Rao
Every AI feature we launched had the same curve. Huge first week, steady for three, then a cliff.
33:05Leela Rao
Week four is where the novelty runs out. Have the second reason ready.

The short version

Five builders, seven minutes each. The common thread was cutting scope: a voice chat that replaced a form, a chatbot that got switched off, and evals that cost less than a team lunch.

5 takeaways

Key takeaways

01Neha's team swapped a 14-field signup form for a 90-second voice chat. Completion went from 41% to 68%.
02Rahul switched off Umbrella's general chatbot and shipped three single-purpose buttons. Support tickets dropped.
03Aisha runs useful evals with 50 hand-labelled examples in a spreadsheet. Start small, start this week.
04Rohan charges per finished report, not per seat, and says it made every sales call easier.
05Leela sees AI feature usage dip around week four. Plan the second reason to come back before you launch.

5 chapters

Chapters

Timed to the recording. Tap a time to link to it.Read each chapter in full

0:00
Neha Joshi: voice onboarding in 30 days
Replacing a long signup form with a short voice conversation, and what they had to fix for accents and noisy rooms.
7:30
Rahul Menon: killing our chatbot
Why a do-everything chatbot lost to three buttons that each do one thing well.
15:10
Aisha Khan: evals on a budget
Fifty labelled examples, one spreadsheet, a weekly review. Enough to catch most regressions.
22:45
Rohan Das: pricing an AI feature
Moving from per-seat to per-report pricing, and the one customer it didn't work for.
30:20
Leela Rao: retention after the novelty
The week-four dip in AI feature usage, and the nudges that brought people back.

5 quotes

Worth quoting

Nobody wants to fill a form. Everybody will tell you about their business if you just ask.
Neha JoshiProduct Designer, Initech2:40 in the transcript
The best AI feature we shipped this year was the off switch on our chatbot.
Rahul MenonProduct Manager, Umbrella Retail10:15 in the transcript
Fifty examples in a spreadsheet beat zero examples in a fancy tool.
Aisha KhanML Engineer, Hooli Labs17:30 in the transcript
Nobody wants to buy seats for a robot. They want to buy the report.
Rohan DasFounder, Acme Reports25:05
Week four is where the novelty runs out. Have the second reason ready.
Leela RaoGrowth Lead, Northwind Health33:05 in the transcript

Transcript excerpt

From the transcript

An excerpt, lightly cleaned up. Speakers were matched by voice, then checked by a person.

2:10Neha Joshi
Our signup form had fourteen fields. Fourteen. We'd been A/B testing the button colour for a year.
2:40Neha Joshi
Nobody wants to fill a form. Everybody will tell you about their business if you just ask.
9:40Rahul Menon
We had a chatbot that could do everything. Track an order, change an address, tell you a joke. People used it for one thing: asking for a human.
10:15Rahul Menon
The best AI feature we shipped this year was the off switch on our chatbot.
17:05Aisha Khan
Everyone asks me which eval tool to buy. My answer is a spreadsheet and a Friday afternoon.
17:30Aisha Khan
Fifty examples in a spreadsheet beat zero examples in a fancy tool.
32:40Leela Rao
Every AI feature we launched had the same curve. Huge first week, steady for three, then a cliff.
33:05Leela Rao
Week four is where the novelty runs out. Have the second reason ready.

Key takeaways

Chapters

Neha Joshi: voice onboarding in 30 days

Rahul Menon: killing our chatbot

Aisha Khan: evals on a budget

Rohan Das: pricing an AI feature

Leela Rao: retention after the novelty

Worth quoting

From the transcript

Key takeaways

Chapters

Neha Joshi: voice onboarding in 30 days

Rahul Menon: killing our chatbot

Aisha Khan: evals on a budget

Rohan Das: pricing an AI feature

Leela Rao: retention after the novelty

Worth quoting

From the transcript