Why We Need Human Workers to Make Robots Look Good
Matthew Beane on the countless "little rescues" by humans that make automation pay off, and how robots could be used to really improve workers' lives
In media chatter about robots there's too much speculation, hype and anxiety, and too little actual information about how humans and intelligent machines are getting along. So it's a good idea to keep up with the work of people who are actually finding things out.
One of those scholars is Matthew Beane, an assistant professor in the Technology Management program at the University of California at Santa Barbara and a Digital Fellow at Stanford’s Digital Economy Lab. He conducts extensive and painstaking research on the ways people work with real robots in real places. That involves more than getting good answers to one's questions. It's also about figuring out what questions to ask. Common yardsticks — how well robots do the work for which they were engineered, or how (if) they've paid off financially — don't suffice to describe how people reckon with the machines. What Beane calls "the full range of value that we seek from robots" also involves aspects of life that neither investors nor engineers seek to measure, including (as I discussed last week) how people use robots politically, socially and psychologically.
I first contacted Beane a couple of years ago about research he had done with surgeons-in-training, as they coped with the challenge of learning how to use robots in the operating room. Those robots allow experienced surgeons to do much more of their work without human assistance, which sounds great, until you consider that assisting master surgeons is how trainees get the experience they need. Some trainees found a path around this obstacle, but they had to break the rules to do it. In the real world of robot adoption, where the machines have to be fit into an education system that wasn't built with them in mind, successful robot-human interactions take place through what Beane calls "shadow learning" — practices that instill skill but that break or even bend the official rules of an institution.
I've interviewed people who wave such questions away, convinced that in the long run things will sort themselves out. But as Harold Ickes said, "in the long run, we're all dead.” In the short and medium runs, the years of my actual life and my son's too, I expect, people won’t be coping with general trends. They’ll be coping with such robots as can be made in such spaces as actually exist, with such contradictory thoughts and feelings as real people have. Making these often-incompatible parts fit together will take a lot of gnarly, imperfect work. That's how the robot future will be made.
With that in mind I recently asked Beane to speak with me about robots in warehouses, where (as I wrote earlier this month) robots are making some workers' lives harder. For the last couple of he and his colleagues have been studying deployments of eight different companies' AI-enabled robots in warehouses and packaging centers.
I spoke with Beane about this work and more generally about what is known about the rocky path from a new technology's invention to its integration into people's lives. This conversation has been edited for length and clarity[1].
Q: I want to ask about a really surprising (to me, anyway) figure I learned from you: A 2018 survey of 850,000 U.S. firms found fewer than 2 percent had robots — 1.3 percent were using them and 0.3 percent were testing them for possible future use. That figure suggests that we're still in the very early days of adoption of this new technology. In other words, the days of failures and pain. Not what you'd expect from all those robot product announcements!
A:Yes. It's real cheap […] to show a robot with a worker doing a thing. On a video that you can play on infinite loop on the internet. But the facts on the ground are that AI-driven robots are really rare.
Q: Are people being irrationally exuberant about robots?
A:Right now, multibillion dollar decisions are being made based on the perspectives that are informed by this sort of smoke-and-mirrors way of processing new technologies, [and] millions of lives are affected. And robots tend not to deliver the kind of price performance that, you know, an organization — one that's not trying to be fancy — needs.
Q. But, to mention a point I've heard you make, all new technology — going back to mechanical looms, steam power, railroads — involves, you've said, "a lot more time, a lot more money and a lot more failure than anyone would hope.” You don't think robots are going to be an exception.
A:Right. So the first really cool study on that front, that I'm aware of, is from 1951. It was about the long wall coal mining method. [The introduction of long-wall coal mining techniques improved efficiency but wiped out miners' practice of working together in small tight-knit groups, leading to widespread anger and resistance to "progress."] Since then there have been, I'd conservatively say, at least five studies every decade that just show the same thing over and over and over again. If you make the claim that things are not generally going to be difficult, slow, expensive and painful with any new technology, you have to have extraordinary evidence for that claim.
Q. On this point, let's talk about non-engineering motives people might have to invest in robots. Like the hospital you studied, that didn't get much out of the uses planned for its robots, but did get other uses out of it.
A:Yes, that study examined the dynamics of the messages that you share with the world about a new technology. Those messages are quite consequential for the value you get out of the technology. At BMW Spartanburg plant, they have these collaborative robots holding a door in front of a worker who's putting the handle on. You know, there is no way that's a scalable process that is profitable for that organization.
Q: So what is going on there?
A: It's a demo. It's a way of exploring what's possible, which is great, cool and fantastic. And I would say for human society, too, probably great, cool and fantastic. But that deceives us all into believing that more is possible, or going on, than is actually possible, or going on. And we're all kind of taking that drug a little. So people who say the synthesis of AI and robotics is soon going to make lots of things better [because] this time is different, well, where's your extraordinary evidence?
Q: But doesn't digital technology speed up the rate at which people learn about new things, and learn how to use them?
A: If you look at one of those simple charts, that just shows the time for a tech to diffuse into the economy, those diffusion curves are getting shorter over time. People often make a quick logical step from that, though, to presuming that "diffusion" equals "optimal use." And that's not obviously so. No one has shown that.
Q: I'd like to talk about the difference between the imagery of robots and the reality you've found in the places you've observed. What you found in warehouses is not a world where robots work perfectly and humans are always just cogs in the system.
A: Sure. So, the last two years, I've been focused on the parts of an organization where you have the jobs that require the least skill, pay the least, involve the most people and are most repetitive. [In those jobs] there's an enormous amount of micro repair that people are doing [at] work to ensure that things stay on the rails. [Workers step in] when technology would have otherwise missed an edge case.
I'll just give you, one concrete example. [In warehouses] there's often a person who will put a package on a conveyor. And one in 40, or one in 80, times, the package will get caught in the conveyor, it will get pinched, it will slide off. Or the plastic that encloses the shipping manifest — that little plastic sleeve on the outside of a FedEx envelope — will get stuck in one of the wheels on the conveyor. When it does, that person fixes that problem 99 percent of the time. That fix is invisible to anyone except for that person (unless you have an absolutely exceptional manager present). So [that person is] important glue that allows for that system to automate that work.
Q: I've seen just that in a factory in Japan that makes cash registers and ticket machines. Every once in a while a part lands just a bit off its proper place, and the robot assembler calls a human over to seat it correctly.
A: They're always these little rescues that humans are doing. Our data show that overwhelmingly, humans are propping up automation in these tiny ways that go unrecognized, uncompensated. I think this applies not just to physical systems like a warehouse conveyor but also to services — for example, in a bank's automated chat system, where a human [gets involved chat bot can't proceed].
There's a tradition in the social sciences that calls this “invisible labor” and "articulation work" — humans helping at the "articulation points" in every process, helping things move fluidly. This [invisible human labor] makes it easy, then, for salespeople to claim that their system works. Because they can trust that after it's thrown it into an organization, the people there will make it work.
Q: This suggests that people who are supposedly unskilled, low-value, interchangeable employees are, in fact, not so unskilled after all.
A: Yes. And that is one of the three things that I'm mostly focused on these days. [We have studied] these frontline workers getting paid close to minimum wage, or at minimum wage, no training, and also often not even high school diploma. [We wanted to know:] under what conditions did they adapt particularly constructively and advanced their lot, and the organization's lot, because automation arrives. There are people who do this. We have found that across organizations of a variety of sizes, across all of the technologies in the sample, all the demographics. It happened with men, women, west, east, north, south, all the technologies.
Q: That's the opposite of the story so many people are telling each other — that more automation makes these jobs less engaged and less rewarding (in every sense of the word).
A: That's why we did this big study. We could have just picked one or two organizations, and one or two technologies, and shown pretty easily that things are hard. Because that's what you would expect. And, trust me, they are. [This work is] difficult. And that's an understatement.
But still, there are some rare individuals and organizations in our sample that, at least in part, get things right. [Not in spite of the difficulties of automation, but] because of them. That's what my surgery study showed. Most surgical trainees are sucking wind, and it's terrible. We should all be very scared. But there's this very small minority that [is working on the edge and succeeding].
Q. And what was the secret of these [warehouse] workers?
A: The workers who got ahead are the ones who had the right sensitivities. That person with the conveyor is a good example. The person who catches those exceptions, keeps things moving. Not all workers do that. It's going to be a pretty small minority. And that will vary by work domain.
Q: So one person's good with mechanical stuff, another person's good with computer glitches, that kind of thing?
A: Some people have a sensitivity with respect to the process they're involved in. Other people have a sensitivity with respect to the technologies they're dealing with. Others have a sensitivity to people. So where, and how, this invisible work gets done has something to do with the capabilities that the individual is bringing to the table. But in general, [as one manager told me] "talent flows through the building like water." [Companies are] getting the benefit of all this. Not because they understand it [in detail]. They know it's there in aggregate, but they aren't measuring it, they don't have data on it.
Q: Do workers themselves appreciate that they have skills, even if they're getting paid minimum wage for a supposedly unskilled job?
A: The number of workers who were absolutely unaware that they had some capability — one that was quite unusual, and that they could develop as a skill and say, parlay into a W-2, capital-J job — was stunning. Others identified them. Colleagues or managers would say, "oh, he's the fix-it guy." Yeah, you know, conveyor goes down, just go call Marty. And Marty would be like, "what do you mean, [I have a skill]? Sure, whatever, I can fix conveyor problems. Who cares?"
Q: OK, so how can this knowledge — that some workers have skills that are invisible to their bosses, and even to themselves — be leveraged to improve conditions in warehouses?
A: In our work, the next couple of years [is going to be finding] ways to help individual workers, managers, technologists, reliably identify this kind of potential to build skill. Find ways to see it more clearly and advance it.
Q: Robots are often sold as a way to standardize and regularize things so that differences from one person to another won't matter. — for example, the hole will be dug precisely the same way each time, instead of differently by Bob than by Sue. Yet you’re saying a company with robots will end up depending on human differences. They’ll need employees who have a "feel" for robots that other people don't have. That implies that it won't be so easy to de-skill workers into being interchangeable cogs.
A: Good managers already know how to [find the right people]. [What they do] is like the Harry Potter sorting hat. What's your swim lane in an organization? Organizations are better at finding that and capitalizing on it with white collar workers. They have to, [dealing with those] more specialized, and ostensibly higher value, jobs. But that problem [of finding people who have the right fit] is equally applicable at the front line. And it's particularly relevant with new automation.
Q: Why?
A: Because new kinds of tech disturb organizational operations. And the more you do that, the more you need that kind of human glue, that local ingenuity, to make things work. To figure out how the tech need to be changed, so that it meshes better with organizational operations. Or how you get a team motivated to use it.
Q: OK, let's pull out and talk about people's biggest anxiety about robot adoption, which is that it will be done in a way that eliminates meaningful work for a decent wage. What can we all do to encourage automation that doesn't do that — that instead enhances human flourishing ?
A: That is a fantastically complicated question. Part of that is political. And legal. [For example], what is your capital gains tax versus your payroll tax in your country? That's going to bear directly on whether you're motivated as a corporation to invest in capital or in people. [Tax policies in the U.S. and other nations make it more costly to hire more employees than it is to install robots and other equipment.] So there are [national] policy concerns that are going to bear on that. But then there's internal organizational policy, too. For instance, what's the right ratio of CEO pay to frontline worker pay? There are things you can do around policies and rules and law in particular, that could improve things, without changing any of the technology.
Q: And in the tech realm, what should people who make robots and buy robots be thinking about?
A:There's an important role for the people building and selling and marketing the technology — to insist on human flourishing as a design criteria. Fine, you can design a system that will improve productivity as quickly as possible. But could you design that same tech in a way that enhances the well being and skills of the people who have to use it — as well as generating the productivity outcomes that you're hoping for? Can you give me that twofer?
You're not always going to get a yes, there. Sometimes you can't. Or sometimes you have to trade one for the other. But we basically aren't even trying there. So a technologist, though, could say, "we're going to try to get both."
Then, as a manager, you could tell the technologist, “Fine, you're going to bring a robot in here that's going to help sort the packages. What could you do with the design, in the time we have, [so that] my workers are better off? So that they're more motivated, they've learned something, and [your technology] has helped me identify their capabilities?” You can start to insist in those directions, I think, if you're a purchaser.
But if you did all that, and there was no policy change, [it won't alter how business is done]. We have this entire system that is laser focused on "$1 in, and $1 and one cent out." We're oriented to razor thin, productivity-enhancing investments in technology. The pressures there are so intense, that it's extraordinarily difficult — especially here in the United States — to say, "we want a little breathing room in time, space and money, to try to [have both productivity gains and better-off workers].
Q : Are you seeing any successes along those lines, in the warehouse industry?
A: Oh, yes, absolutely. We're going to be publishing papers in the next year and a half on successes in three bins: organizations, workers, and technologists. In each of those categories, there are those who, in spite of all these challenges, had early, disproportionate, surprising success — in achieving both aims (gains in productivity and improvements in workers' lives).
Not all AI enabled, robotic vendors are created equal, some of them are doing things that allow for high productivity plus high humanity. What are those things? How is it that they're doing that? Actually, [the company in our study that is] making the most money, and is most successful in our sample, took that strategy. So [we're preparing] at least one paper in each of those three categories, identifying what is working from the viewpoint of a worker, from the viewpoint of an organization, from the viewpoint of a technologist.
Q: Which you can't reveal here, right?
A: I had enough people put the fear of God in me about how you want to get it published first before you before you spill the beans. But I'm very motivated. Because, well, my team and I thought life as a frontline worker in a warehouse, especially when automation comes in, would be quite rough. And it was really a lot worse than we expected.
You know, life is very intense and difficult at every single warehousing site. If you want to go for a lunch break, within five miles, you're going to get your food through bulletproof glass. And then right next to that place is the payday lender who will take 40 cents on $1 off your paycheck. [Yet] there are people who get ahead, there are beautiful, enduring and supportive relationships that people form. There are many beautiful things about it, some of the most touching human behavior I've ever seen in my life I've seen from these workers. But [their working life]? It's not okay. If I can, through something I write, contribute to [someone making] a more level-headed decision, or thinking to include worker welfare in their technological design requirements, then, the world becomes more of the kind of place I want to live in.
[1]. If you're a subscriber (remember, it's free!) you can request the full pre-editing transcript and I'll email it to you. (Of course, in an unedited transcript Beane will read more rambly and I will sound like a complete idiot, so caveat lector on that.)
This and That, Week of April 19
Robots on Mars: Let's take a moment to salute all the roboticists who contributed over decades to an epic moment in human history — the day we flew in an alien sky. The 30-second flight of Ingenuity on Monday was, among many other technical firsts, a triumph of robotics. As it takes about 15 minutes for radio waves to travel these days between our planet and Mars[1], you can't teleoperate this one. Instead, the robot chopper autonomously made the second-by-second adjustments required. (It also autonomously kept itself warm enough in the Martian cold, and recharged its batteries as needed.)
Robots made of living cells: The other mind-blowing robot news this week was the publication of this paper (pdf available here), which describes a robot swarm made from living cells. The cells were developed from frog stem cells. They turned into skin cells that used their tiny hairs, called cilia, to move about. The authors — Douglas Blackiston, Emma Lederer, Sam Kriegman, Simon Garnier, Joshua Bongard and Michael Levin — got these cells to work together to gather objects.
This got the attention of biologists as well as roboticists. That's because cilia normally function on frog skin to move mucus around and push away parasites. That these cells used their cilia for a different purpose — to move around — was surprise. It suggests that cells aren't as constrained by their genetic "instructions" as many supposed. The team has a great website explaining their work here. And Philip Ball has a fine piece about this research here.