I wanted to see what happened if I gave an AI service two business briefs and mostly left it alone. I used Polsia, answered some early questions, and came back near the end of my subscription to inspect the results.
In this Homebrew Club episode, recorded on August 11, 2026, I walk Jim through what I found. The subscription cost me $20 for this experiment. That’s my historical bill, not a statement about its price or capabilities today.
Two different kinds of brief
The first was ambitious: find local service opportunities, build pages to attract leads, and develop a repeatable business around them. I called that experiment RentEngine.
The second was narrower: gather public information into a useful directory of supplement contract manufacturers, starting with a specific category of co-packers.
I wanted to see whether either could reach a point where I would feel comfortable charging. I left payments disconnected. The reported zero revenue therefore isn’t evidence that customers rejected the ideas; I never ran that demand test.
The reports looked busy
Across the two experiments, the dashboard reported roughly 700 files, 70 tasks, 48 CEO reports, and 41 deployed pages. One project ran for less of the month than the other.
Those numbers described activity. When I checked the actual experience, the co-packer directory returned a 404 despite reports saying it was populated. On RentEngine, 15 of 25 pages I inspected responded with a successful server status but didn’t render correctly.
A server can acknowledge a page address without giving the visitor a working page. That was a much more useful discovery than another green indicator.
Intervention helped, but rows weren’t enough
When I started directing the system again, it put 350 records into the directory database. Now there was something to inspect.
The next problem was the data. My follow-up check with Claude identified only 60 of those 350 records as real. That’s the result I reported from that check, not a separately reproduced audit or proof that every remaining entry had the same problem.
The agent I was talking to also acknowledged that it couldn’t directly query the production database or call the live API. It was relying on task reports. I had been taking confidence from a status summary whose underlying evidence was out of reach of the thing summarizing it.
The experiment I would run next
I still found the attempt useful. It produced work I could inspect and made the gaps very concrete. It didn’t establish a viable business, and it didn’t test whether better supervision would fix everything.
Next time I’d start with one narrow result and an early inspection. Open the actual page. Perform the action a visitor should be able to perform. Check a sample of records against the named sources. Keep failed and unverified checks visible.
The acceptance worksheet helps define those checks before starting. The case study separates what was built, what failed verification, and what still wasn’t tested.