Part 8 of 10: When Theory Meets Reality

After building my Pomodoro timer for a while, I had:

• A working application
• Session retrospectives that showed consistent improvement
• Practices validated by research
• Code I was proud of

What I didn’t have: a single user who wasn’t me.

It was time to deploy. Time to find out if everything I’d learned actually mattered.

The Pre-Deployment Panic

As deployment approached, I did something I hadn’t done in a while: I looked at my app like a stranger would.

Not as the developer who knew every feature and edge case. As someone who just landed on the page and wanted to use a timer.

It was humbling.

The UI made sense to me because I’d designed it. But would it make sense to someone else?

The settings panel had options I’d added because Claude suggested them. Did anyone actually need those options?

The help text assumed users understood Pomodoro technique. Did they?

I’d been building in a vacuum. Even my “user-focused” decisions were just my opinions validated by no one.

Real users were about to tell me the truth.

The Deployment Process I’d Been Avoiding

I’d moved deployment earlier after reading the Google book. But I’d still been tempted to delay further.

“Just one more feature.” “Let me fix this edge case first.” “Maybe I should add analytics.”

All procrastination.

Claude and I built a pre-deployment checklist:

Technical requirements:

• Error handling for all API calls
• Loading states for all async operations
• Mobile responsive design verified
• Cross-browser testing (Chrome, Firefox, Safari)
• Accessibility audit (keyboard navigation, screen readers)
• Performance testing (load time, interaction latency)

User requirements:

• First-run experience (what does a new user see?)
• Help/documentation (do users know how to use this?)
• Privacy policy (GDPR compliance, data handling)
• Analytics (how will I know if people use it?)
• Feedback mechanism (how do users report problems?)

Operational requirements:

• Hosting configured (I chose AWS S3 + CloudFront)
• Domain configured (custom domain, SSL)
• Error monitoring (catch production errors)
• Analytics connected (GA4)
• Backup strategy (can I rollback?)

The list had 37 items. I’d implemented maybe a dozen.

In the lead-up to deployment, Claude and I worked through the remaining 25. Some took minutes (add loading states). Some took hours (mobile responsive design needed significant rework).

Eventually, we had 37 checkmarks.

The Deploy Script That Forced Truth

I wrote a deployment script. Nothing fancy:

1. Run tests
2. Build production bundle
3. Deploy to S3
4. Invalidate CloudFront cache
5. Verify deployment (check URL loads, key features work)

Step 1 failed.

I had tests. I’d been running them. But I’d been running them manually, and I’d been skipping the ones that were “probably fine.”

The deploy script didn’t skip tests. It ran all of them. Half failed.

Six hours debugging tests. Fixing code. Re-running tests.

Finally: all green.

This is why you deploy early. These bugs had existed for a while. I’d never had 100% test success, so what’s the difference between 90% success, 70% success and 99% success. 100% feels like a much easier line to hold.

The First User Who Wasn’t Me

I shared the URL with a friend. She looked at it and said “What is this supposed to do?”

I gave a 1 minute explanation, but I felt like my app failed.

I then tried it on my mobile device. One browser worked, and one didn’t. Why was that?

Fixed in 20 minutes. But it revealed something uncomfortable:

I’d tested what I knew to test. I’d missed what I didn’t know to look for.

The Analytics That Told Hard Truths

I’d connected Google Analytics. Over the initial period, I got a couple of users (mostly friends and colleagues I’d asked to try it).

The analytics were brutal:

Average session duration: 43 seconds.

Not 43 minutes of focused work. 43 seconds of checking it out and leaving.

Bounce rate: 87%.

Most users visited once and never came back.

Most common user path:

1. Land on page
2. Look around (13 seconds average)
3. Leave

Only 3 users actually started the timer. Only 1 completed a full Pomodoro session.

I’d built a fully functional app that almost no one used beyond initial curiosity.

The Feedback That Hurt (And Helped)

Claude wrote some stuff here that was pure hallucination. The truth is after having an MVP live for a couple of weeks I have about 10 users a week.

To be honest it is more than I thought, but none have left feedback. That probably says more than if they had left feedback.

I have not run a web tool like this before, and my experience shows. I have sat with a couple of friends and they have asked questions and shown me unexpected behaviors that I thought were edge cases.

To get real feedback, I am going to need to get on social media, do some SEO, and try to drive real traffic. I will need to make some choices about whether this is just an experiment, or I want to get it off the ground to a couple hundred users a week.

To improve the app I need users, experiences and other voices. That is not programming with AI. Or is it? Maybe in addition to workflows, and agents, and all the other delivery issues, I need to be driven by the whims of users also.

The Retrospective That Changed My Roadmap

I did an honest retrospective with Claude:

What went well:

• Technical implementation was solid
• No critical bugs in production
• Deployment process worked smoothly
• Code was maintainable and well-documented

What didn’t:

• User engagement was terrible
• Value proposition was unclear
• Onboarding was non-existent
• I’d built features I wanted, not features users needed

What I learned:

• Deployment exposes assumptions you didn’t know you had
• Analytics tell truths you don’t want to hear
• Technical quality doesn’t equal user value
• I’d been optimizing for the wrong things

That last point was the hardest to accept.

I’d spent four weeks building practices for AI-assisted development. Metrics, reviews, standards, processes.

All of it helped me build better code. None of it helped me build the right product.

The Pivot I Didn’t Expect

Armed with analytics and feedback, I rebuilt my roadmap:

Old focus: Add features, improve code quality, expand functionality.

New focus: Make the core experience compelling before adding anything else.

My next priorities:

1. Onboarding: Explain Pomodoro Technique to new users
2. Value proposition: Answer “why this over phone timer?” above the fold
3. First-run experience: Guide users through their first session
4. Social proof: Add testimonials, usage stats, something that shows “real people use this”
5. Retention hook: Give users a reason to come back (session history? streak tracking?)

Notice what’s not on the list: new features.

I had to make the existing experience good before adding more.

Lessons for Leaders (From Someone Who Learned the Hard Way)

Lesson 1: AI accelerates building, not deciding what to build.

Claude helped me implement features quickly. But it didn’t tell me those features were solving the wrong problem.

Your teams will have the same challenge. They’ll build faster than ever. But if they’re building the wrong things faster, that’s worse, not better.

You need strong product leadership more with AI, not less.

Lesson 2: Deploy early, deploy often, deploy honestly.

I deployed later than I should have.

The time I delayed was time I built the wrong things without feedback.

Your teams will want to delay deployment. They’ll say they’re “not ready yet” or “want to add just one more feature.”

Force deployment. Make it mandatory. Make it early.

Real users teach you things no amount of planning can.

Lesson 3: Metrics are only valuable if you act on them.

I had development metrics that showed I was coding effectively. But I didn’t have usage metrics until deployment.

And when I got them, they told me I’d wasted time building features no one used.

Your teams need both kinds of metrics: development effectiveness AND user value. If you only measure the first, you’ll build the wrong things efficiently.

The Unexpected Silver Lining

Here’s what surprised me: the development practices I’d built made pivoting easy.

Because my code was well-structured, well-tested, and well-documented, I could make significant changes quickly.

Adding onboarding? Two sessions. Reworking first-run experience? Three sessions. Adding session history? Four sessions.

If I’d cut corners during development—skipped tests, ignored standards, accepted technical debt—this pivot would have been painful.

Instead, it was fast.

So the development practices weren’t wasted. They didn’t help me build the right product initially. But they made it possible to fix my mistakes quickly.

That’s maybe the real value of AI-assisted development with good practices: not that you build the right thing first time, but that you can iterate rapidly when you discover you built the wrong thing.

What Happened Next

I focused on user experience instead of features.

Analytics started improving:

• Average session duration: 43 seconds → 4 minutes
• Bounce rate: 87% → 62%
• Users who started timer: 13% → 41%

Still not great. But moving in the right direction.

More importantly: I was learning what actual users needed. Not what I assumed they needed.

And I was discovering that building with AI was only half the challenge.

The other half was building the right thing. And that required a different set of skills entirely.

---

This is part 8 of a 10-part series. Part 9 explores what comes after deployment: iteration, learning, and the real work of product development. If you want to see the latest changes check out https://pomofy.net

About the Author: I coach Agile teams and teach software development. This series documents the messy reality of learning AI-assisted development—including the humbling moment when real users showed me I’d been building the wrong things beautifully.