I think the first time I heard about how software agents were going to be useful to help perform tasks for us must have been with General Magic in the 90s. It was all the same use cases we hear about today, flight booking, recipe generation etc. Just taking out all that drudgery from our lives.
I think a big part of this appeal was it makes you feel like you are important. Like wealthy, important people have assistants that go out and do things for them. They aren’t bogged down by the drudgery of searching through airline flights to get the best deal or something. They have people for that. If my little device can give me the same kind of power then I feel a little bit more like a wealthy person.
The key problem here, and one that we all should know really well by now is that isn’t at all how The System works. None of these digital agents will be making these choices for our benefit. They will be made for the benefit of the service providers.
Here’s what happens today when you search for hotels at hotels.com:
Clicking on the ⓘ actually gives you a good explanation of what they do here.
Also, the dropdown has options:
The default option is the one that maximizes the revenue for Hotels.com. Once a PM runs an A/B test which shows that putting that disclaimer there increases the number of people who switch the option, the disclaimer will go away. Maybe they will end up hiding the options to choose other search criteria entirely.
Agents, particularly those which use a simple voice UX allow the system to just speed run all of that. Here are your options. I’m sure you will love them.
Personal computer technology started with some brave ideas like building bicycles for the mind. That made a lot of people a lot of money, but it wasn’t enough.
When we got smartphones we got a lot more capabilities for sure but we also got attention manipulating media which was less a bicycle for the mind and more of a skinner box for the culture. The people who built these systems knew they were hurting us but they did it anyway so they would have enough money to buy parts of Hawaii.
AI does have great potential, but most of these systems which want to turn it into some kind of agent for us, are really about removing our agency. It’s about trapping us in a set of very nice options which are chosen for us by someone else. The problem won’t be that the AI hallucinates. The problem will be that we will believe in the illusions of choice it creates for us.
Leave a Reply