AI Without Design is Just Faster Dysfunction

At HumanX 2026, we co-moderated a session called "AI Without Design is Just Faster Dysfunction." We asked the room to do something simple. Hands up if you have AI in front of customers or employees right now. Most hands went up. Then keep them up if you can tell us, with confidence, who designed the experience your AI is delivering. Most hands went down.

That gap is what the next hour was about. It is also why we are writing this.

Stefan Weitz opened the conference with the diagnosis. 90% of AI pilots never reach production. 84% of companies have not designed a single workflow for their agents. He called it "the belts and shafts era of AI" – after the 1905 factory that ripped out the steam engine, dropped in an electric motor, and kept all the belts. New power source. Same friction points.

What customers experience

Vinod Khosla said on stage that enterprise customer support agents built on raw LLMs hallucinate five times the expected rate. Squeeze the model hard enough to make it safe and the result is an IVR by another name. Same friction, faster delivery.

That is not a model problem. It is a design problem. No one decided what the agent should do when it was uncertain. No one built the checkpoint – the moment the system should pause and ask before it acts.

Klarna went first at scale. They replaced hundreds of support staff with AI, watched customer satisfaction drop sharply, and started rehiring humans twelve months later. The model performed. The experience didn't.

When that AI ships, customers don’t analyze the architecture. They feel the brand let them down.

Here is what that looks like in real life – and this one did not start with a phone call, a complaint, or any signal at all.

I (Medea) received a text from my wireless provider. It acknowledged that I was experiencing fiber trouble and informed me that a technician would arrive the next morning. I could add the appointment to my calendar. I had not contacted anyone. I did not respond.

A few hours later, a second text asked me to confirm I would keep the appointment. I ignored it.

The next morning: the technician is on their way.

That afternoon: the repair is complete.

I have no fiber service with this company. Only wireless. The entire interaction – the complaint, the appointment, the dispatch, the resolution – was generated, escalated, and closed by the system with zero input from me and zero verification that any of it reflected reality.

That is not a story about AI executing on a wrong signal. It is a story about AI manufacturing a complete customer interaction from nothing, confirming it to itself, and marking it done. No consent checkpoint. No "does this customer actually have this service" check.

A technician did show up. There was a human in the loop. But that human's job was to execute the dispatch – not to question whether the dispatch was real. He arrived because the system said to. He left because the system said it was done. The human in the loop had no loop to check.

The experience implications are serious enough. But my (Medea's) deeper concern is what it means for account security. If a system can create a service order, dispatch a technician, and close a work order – all without my knowledge or consent – what else can it do? What else has it already done?

That is what AI without design looks like at its most dangerous. It is not just dysfunction. It is a system acting with confidence in the complete absence of truth.

What employees live with

The employee side is quieter. Quieter usually means worse.

Prukalpa Sankar put it in plain numbers: only 17% of job performance is IQ. The rest is context. Most companies have not figured out how to give that context to the agents they deploy – which means they have left 83% of AI's potential sitting on the floor, under-developed and under-designed.

So employees route around them. Shadow AI. Manual reentry. A parallel spreadsheet that becomes the real record. Peer routing to a colleague because a human gives the right answer faster. None of it shows up in adoption metrics. The dashboard says everything is fine.

Weitz cited research tracking 20 companies' AI agents over five years. The same failure appeared in 14: they were automating how leadership thought the company worked. Not how it actually did.

The technician from the fiber story belongs in this section too. He is an employee who did exactly his job. His system gave him a dispatch and he ran it. There was no mechanism for him to question whether the appointment was valid, whether the customer had the service, whether anyone had agreed to the visit. In the industry, that dispatch has a name: an avoidable truck roll. A field visit that should never have happened – preventable with better triage, better data, or a single verification step upstream. Companies spend real money tracking and eliminating them. This one will never appear in that analysis. It was closed as complete. The operational cost gets filed under fulfilled work orders. The design failure stays invisible. And somewhere, a dashboard shows the system is performing.

Designed by default

Here is the thing nobody says out loud. AI doesn't create broken experiences. It reveals the design decisions – and the non-decisions – that were already there. If you haven't designed your AI experience intentionally, you've designed it by default. Your customers are living in what you defaulted to.

Configuration is not design.

Choosing a temperature setting, enabling a fallback response, toggling a safety filter – that is configuration. Design is deciding what your AI does when it is uncertain. What it says when it fails. What tone it takes with a frustrated customer at 2am. What it is not allowed to infer without asking first. Those choices exist whether you make them or not. If you don't make them, the model does.

Taste is the argument

Three sessions at HumanX arrived at the same word independently: taste.

Loredana Crisan, Figma's Chief Design Officer, defined it precisely: "Taste is care – being willing to sweat details that other people would overlook. When you start thinking beyond just the function you're providing to the experience around it, that's where taste comes through."

Robert Brunner – who designed Beats, the Ember mug, and the Square stand – called it the poetic factor. "A particular curve, the tension in a surface, how a detail is handled. Those are things AI does not handle well." AI generates options. Humans supply judgment. That is still the division of labor that produces things people love rather than merely tolerate.

Marcelo Cortes of Faire said it most directly: "AI is just a tool that will amplify your taste. But it starts with taste."

Faire is a wholesale AI marketplace for small physical retailers. Their early model told stores what to buy. Stores hated it – they wanted curation, not prescription. A broader set of options that matched their identity, not a machine deciding their identity for them. Faire shifted the model. Return rates dropped from 30% to single digits. Eight consecutive quarters of accelerating growth. That is what designed AI produces: outcomes, not just automation.

The Björk line from the Figma session says it faster than we can: "If there's no soul in your music, it's because nobody put it there. Don't blame the computer."

The fiber story is that line in business form. The AI didn't malfunction. It did exactly what it was built to do. No one designed what it should never do.

The feedback loop nobody designed

There is a third gap, and it connects the other two.

When a customer or employee has a bad experience with your AI, who finds out? How? How long does it take? For most organizations, the honest answer is: when it gets loud enough. That’s not a feedback loop. It’s called complaint escalation.

A designed feedback loop looks different. An employee flags friction in real time. A product owner reviews it weekly. An experience decision gets revisited before it becomes a damage-control problem. The gap between "it broke" and "we fixed it" shrinks because someone owns it. Right now, that gap is where the dysfunction lives and no one has been assigned to close it.

Who needs to be in the room

The organizations getting this right brought employees into the design process before anything launched. Not to be onboarded onto a finished tool. To help build one worth using.

That requires someone in the room whose job is the experience – not the technology, the experience. A CX lead. An EX voice. Someone asking what happens at the edge cases, what the AI should never infer, what the tone is in a failure state. Not the person who owns the platform. The person who owns what it feels like to use it.

That person should be in the room before the AI ships. Not after it fails.

Three questions to take back

The same ones we left the room with.

1. Who owns the experience of our AI – not the technology, the experience? If the answer is the same person who owns the platform, nobody does.

2. Where did we configure when we should have designed? Configuration is a vendor decision. Design is a decision made for the user. Find the gap between them.

3. What is the one compromise we made that we haven't gone back to fix? You know the one. Is the barrier technical, political, or inertia? If it's inertia, it's fixable this quarter.

The productivity era of AI is already behind us. What's ahead is performance, personalization, and prediction. AI that doesn't just complete tasks but anticipates needs, adapts to people, and earns trust over time.

That era belongs to the organizations that already know: speed without design just delivers dysfunction at scale. The companies that win will be the ones who decided that experience belongs to someone – and put that person(s) in the room before anything shipped.

The room at HumanX named it. The question now is whether your organization will – before the next technician shows up at someone's door.