Health care workers are not the only unwilling essential services frontline workers at increased risk of COVID-19. According to the Washington Post on April 12, “At least 41 grocery workers have died of the coronavirus and thousands more have tested positive in recent weeks”. At the same time, grocery stores are seeing a surge in demand and are currently hiring. The food industry is also seeing increasing adoption of robots in both the back end supply chain and in the food retail and food service sectors.
“Grocery workers are risking their safety, often for poverty-level wages, so the rest of us can shelter in place,” said John Logan, director of labor and employment studies at San Francisco State University. “The only way the rest of us are able to stay home is because they’re willing to go to work.” [Washington Post April 12 2020]
In our April 7th edition of “COVID-19, robots and us”, we heard from Linda Pouliot, CEO and founder of Dishcraft Robotics and Vipin Jain, CEO and founder of Blendid. Both provided us with insights into robotics in the food industry and the difficulties and joys of adapting robots to COVID-19 work.
Dishcraft Robotics is Linda Pouliot’s fourth startup, and second robotics startup. She previously cofounded Neato Robotics, the number two player in automated vacuuming. Dishcraft Robotics is a clean dish delivery service for cafeterias, or large kitchens. On a daily basis they deliver customized clean dishes to the customer, they collect the dirty dishes, and return them to a central hub where custom robots wash them.
“We initially thought we would develop the robot and install it directly into dish rooms. And then we found that it was a really high hurdle for an operation to have a big capital expense, and that we could solve their problem in a really frictionless way, by simply doing the delivery because there’s no upfront cost for them. There is no risk for them to try it. And in fact, we give a free couple week trial and we have found that once you start with dish craft, you immediately convert because the service solves all their labor problems, it also is more sustainable. And in today’s environment, it’s very, very sanitary.”
“We give them carts full of clean dishes, they are our dishes that are proprietary. The way our robots work is using magnetics. And so it requires using our specialized wares. And so we just bring in flexible, clean dishes and we give them a collection system where they can drop off. It’s very organized, it’s ADA compliant. It’s pretty space saving compared to where people normally store their dishes. So it’s worked really well for us. And that means for a customer, there’s no construction costs. There’s no months of planning. You know, it’s just been pretty delightful for both us and them.”
“Currently we service essential businesses and we have started to offer our services to help in this time of need to hospitals. Dish rooms are very small and cramped spaces and we realized that workers in them were unlikely to be able to be socially distance. Also today, the hospitals are overloaded. Some have twice the volume of what they normally expect a dish room to have to process. And so this was a great opportunity for them to use robots and use our service and keep their existing staff very safe.” [Linda Pouliot April 7 2020]
Vipin Jain is the CEO and founder of Blendid, a food robotic kiosk startup. After about 5 years of R&D Blendid opened their first kiosks just over a year ago in March 2019 at the University of San Francisco, followed by Sonoma State University, and while they were initially doubtful that people were ready to have their food or drink made by robots, they’ve seen great interest.
“We were told that automating food was too hard. And people won’t like it. People don’t want robots to make the food, people said. But people have different tastes and and standardized food doesn’t cut it anymore. So you need a better solution so that I can get what I like to eat anytime of the day wherever I am. And that can only be done using a lot of data, AI and robotics.”
“We were just getting ready to start deploying into retail, corporate and hospital environments when the COVID hit. So I wish we were a little bit ahead further ahead in terms of deployment because as you can imagine, food is an essential service. We all want food, we all want access to food, but in this environment, we will be fearful about the safety of people working on food preparation, because we don’t know how the food was handled.” [Vipin Jain April 7 2020]
Rich Stump is one of the cofounders of Studio Fathom for prototyping, product development and production services. Fathom specializes in 3d printing additive manufacturing, dealing with multiple materials and leveraging traditional manufacturing alongside, allowing them to solve some interesting engineering challenges or product development challenges. Fathom has been very active in COVID-19 response.
“There’s three main initiatives that we have going right now with COVID-19. And some of them are, are potentially revenue generating, but most of them are just just trying to help us with our expertise and capabilities. The first is obviously with PPE challenges we have. We have a facility in Asia and southern China and so we have a vast amount of supply chain resources. So, we called in our colleagues over there and we found a number of factories that had FDA approved PBE supplies that that were overrun from the flattening of the curve in Asia.”
“The news has been talking about investment into 3M and Honeywell ramping up production in the US, which is absolutely great. The problem is, it’s just not going to get done in time, right? Anytime you buy 150 thermoforming machines and you try to ramp up production to make n95 masks, it’s going to take weeks or potentially months to get any volume. So what we were trying to do is match hospitals and senior homes and folks that needed PPE with supply chain resources that we had in Asia.”
“We’ve had a bunch of roadblocks to the point where we’ve tried to reach out to the state government officials to try to remove some of the roadblocks with customs and tariffs and freight constraints. And so that’s been a an interesting challenge. But we’ve been able to connect, I think over 1.5 million and counting supplies into the US from Asia. It’s not our everyday business, but we have resources, so we tried to help there.
“The second initiative was around the test kits. Obviously there’s been a shortage of test kits. And it turns out, interestingly enough, one of the big shortage items is the nasopharyngeal swab that goes back into your nasal cavity. The main manufacturer was in Italy, and obviously with everything happened in Italy, that that shut down a lot of the supply. There’s been a community of about 50 folks that have come out of the 3d printing community to see if we can 3d print these swabs. And we’ve made a lot of progress.”
There’s now four manufacturers approved to 3d print nasopharyngeal swabs. Next week, we’ll get into production of these in at least hundreds of thousands, if not millions, in order to support the short term need. Long term, it probably doesn’t make sense to 3d print them given the cost base. But using 3d printing technology, we can design swabs that could potentially perform better than the traditional swab. So that’s been a fun and challenging project at the same time.”
“And the third initiative is around the ventilators. You have folks like Lawrence Livermore developing a ventilator with spare parts that are available today. And you have JPL, the NASA team, trying to develop their own ventilator. Then you have the large automotive manufacturers who have partnered up with various existing ventilator manufacturers and are trying to ramp up production. By today I think more than 270 people have come to us with either a ventilator design, a mask design, or some type of shield design, just over the last 10 days. There’s just a tremendous amount of activity. And it’s great because everyone wants to help, but at the same time, it presents a lot of challenges because so much effort going in so many different directions, that you you worry that, you know, all these efforts won’t end up being impactful by the time we need the supplies.” [Rich Stump April 7 2020]
Mark Martin is the Director of Industry and Workforce Development for California Community Colleges and founder of the Bay Area Manufacturers Discussion Forum. He’s very connected with manufacturers in the region and has a lot of insights into what’s going on in the manufacturing sector.
“Many of the manufacturing companies are being hurt and having to furlough people. There’s about 8000 in the Bay Area and we’re trying to help them repurpose. In some things, it’s relatively easy, maybe you have an injection mold facility, and you can injection mold these face shields, or some of these parts of the face shield. You have to make the molds but it’s still within your core business. Others are contract manufacturers that could theoretically assemble ventilators because they’re already doing medical products. Ventilators are a complicated product, but not necessarily more complicated than some of these people already doing.”
“But for others, you might have to get specialized machines, which can take a long time. And then you need to have expertise around. If you haven’t done medical products before, do you know how to handle the equipment and ensure QC? And the FDA approves factories, not just the designs. I created a list of Bay Area manufacturers and I got almost 200 responses with things they think they can manufacture and I’ve supplied that list to the state government.”
“And community colleges have makerspaces or fab labs. So where I’m located at Laney College in Oakland, we started working on face shields a week and a half ago, because that was relatively simple for us to be able to do. And we printed up 500 face shields to Highland hospital. It took us a day to do it with six people. And I asked the hospital what their daily use was for face shields and they said 600 a day. So I was like, all right, that was a day’s worth for this one hospital.”
“And then we actually took a design that was done by Stratasys for 3D printing and modified it a little bit. And in like two days, we got a molder to set up injection molding, although it’s going to take a couple of weeks to finish. And honestly, I have no idea if we’re going to need in a couple weeks. Because I have no idea what the demand is.”
“Apple’s bringing in over a million a week and others are doing this hundreds or even thousands a time in basements and little shops. But we don’t know if the demand is 5 million a week or 50 million. “
“So that that’s the thing the manufacturers all trying to figure out. I kept wondering why the government didn’t take a little bit of control. Even just financial incentives to say we will backstop you if you supply PPE, so that if the demand falls out, you don’t lose money.” [Mark Martin April 7 2020]
Ken Goldberg is the William S. Floyd Jr Distinguished Chair in Engineering at UC Berkeley, Director of CITRIS People and Robots Initiative and a regular guest on COVID-19, robots and us. Today in keeping with the theme of food robotics, Ken explained the reasoning behind one of his latest projects, AlphaGarden.
This is a farm bot that you can buy from a local company, in San Luis Obispo for about $3,000. Our garden includes 14 species of edible herbs and plants and they’re all growing in close proximity. The challenge is that they’re competing for increasingly scarce light and water. So what happens is that this starts to get quite out of control. Right now, the garden has been acting very autonomously because we haven’t been able to get into the greenhouse, and so it’s starting to decay, and certain invasive species like mint have essentially taken over.”
“We’re trying to optimize diversity, that is to maximize the number of different plant types that are growing. But the key idea here is that we’re using AI to simulate the garden, because a grow cycle takes three months. So we have a simulator that can simulate 10,000 times the speed of nature. We have 64 gardens but thousands of parameters to fine tune the system. Then we evaluate each of those gardens to figure out how to tune the parameters for the watering and pruning devices.”
“This is an ongoing project but the end goal is to be able to learn a policy for successful gardening because polyculture is much more labor intensive than monoculture. The reason that I call it an art project is because it’s extremely difficult for AI and robotics. It’s a big challenge. I’m not at all convinced we’re actually going to succeed. We’re really putting AI to the test. But just this week we did a deep learning method based on the data that we collected from our simulator. And in time we hope to learn how to be able to automate some of these very difficult tasks like organic gardening.” [Ken Goldberg April 7 2020]
The moderator of the weekly ‘COVID-19, robots and us’ discussions, Andra Keay, is the Managing Director of Silicon Valley Robotics, supporting innovation and commercialization of robotics technologies. Andra was also the Industry Co-Chair of the Human-Robot Interaction Conference 2020, which is taking place online. In the April 3 Industry Talks Session, Chris Roberts from Cambridge Consultants described the process of working out what sort of robotics or automation made sense for food production.
“Chris described a project with the food service industry to look at using robotics and AI. And of course what he wanted to build was a robot dipping chips and flipping burgers and all the rest, but on evaluation, that was not the best direction to go. In this instance AI optimization of locations was the best option. And this is something I’ve heard from a number of other people who’ve been doing robots for the actual preparation of food itself. For example, Creator, who make those fantastic hamburgers, didn’t have a robot arm, imitating the way humans cook. Instead, they completely re architected the entire process of constructing a burger, so that it could be done mechanically.” [Andra Keay April 7 2020]
There’s much more to hear in our weekly discussion on ‘COVID-19, robots and us’ from April 7 2020 with guest expert speakers: Linda Pouliot, CEO of Dishcraft Robotics Vipin Jain, CEO of Blendid Rich Stump, Principal at Studio Fathom Mark Martin, Industry & Workforce Development California Community Colleges and Bay Area Manufacturing Discussion Forum Ken Goldberg, Director of CITRIS People and Robots Initiative and William S. Lloyd Jr Distinguished Chair in Engineering at UC Berkeley; moderated by Andra Keay, Silicon Valley Robotics.
Community, craft, and the vernacular in artificially intelligent systems take the position that everyone participating in society is an expert in our experiences within the community infrastructures, which inform the makeup of robotic entities.
Though we may not be familiar with the jargon used in specialized professional contexts, we share the vernacular of who we are as people and communities and the intimate sense that we are being learned. We understand that our data and collaboration is valuable, and our ability to successfully cooperate with the robotic systems proliferating around is well served by the creation of qualitatively informed systems that understand and perhaps even share the aims and values of the humans they work with.
Using her art practice, which interrogates a humanoid robot and seeks to create culturally specific voice interactive entities as a case in point, Dinkins examines how interactions between humans and robots are reshaping human-robot and human-human relationships and interactions. She ponders these ideas through the lens of race, gender, and aging. She argues communities on the margins of tech production, code, and the institutions creating the future must work to upend, circumvent, or reinvent the algorithmic systems increasingly controlling the world, including robotics, that maintain us.
Publication:HRI ’20: Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot InteractionMarch 2020 Pages 221https://doi.org/10.1145/3319502.3374844
Wired Magazine recently called for us to, post pandemic, “ditch our tech enabled tools of social distancing”. But are our telepresence robots creating emotional distancing or are they actually improving our emotional lives. This week in our weekly “COVID-19, robots and us” discussion with experts, we’re looking at the topic of virtual presence and emotional contact as well as many other practical ways that robotics can make a difference in pandemic times.
Robin Murphy, Raytheon Professor at Texas A&M University and founder of the field of Rescue Robotics, was involved in the very first use of robots in a disaster scenario in 9/11. Since then she’s been involved in multiple disaster responses worldwide, including the Ebola outbreak in 2014-2016. During the US Ebola outbreak, the White House Office of Science, Technology and policy, and then later NSF did a series of workshops, and myself and Ken Goldberg, are among those who participated in work with various public health officials in groups such as Doctors Without Borders.
“Some of the lessons learned about infectious diseases in general, and for COVID, in particular, are that there’s really five big categories of uses of robots. Most everybody immediately thinks of clinical applications, or things that are directly related to the health care industry, but the roll of robots is far broader than that. It’s not always about replacing doctors. It’s about how can robots assist in any way they can in this whole large, complex enterprise of a disaster.” [Robin Murphy March 31 2020]
Ross Mead’s company Semio develops software for robot operation that focuses on how people will live, work and play with robots in their everyday lives. “We’re building an operating system and app ecosystem for in home personal robots. And all of our software relies on natural language user interfaces, just speech and body language or other social cues.”
“Right now as it pertains to COVID-19. We are working with a team of content creators from a company called copy to develop conversational content similar to chatbots, or voice voice based skills that’s geared towards informing users about or helping mitigate the spread of COVID-19. We’re also developing socially aware navigation systems for mobile robots, natural human environments. I would love to talk about use cases for social robots, even telepresence robots, as well as the impacts of social isolation in these times.” [Ross Mead March 31 2020]
Wendy Ju studies people’s interaction with autonomous cars, systems or robots. She focuses on things like nonverbal cues, and the things people do to coordinate and collaborate with one another. Her PhD dissertation was on the design of implicit interactions, something a lot of us take for granted or consider static, not dynamic. Through her work on autonomous cars, she’s been exploring the subtle cues that convey critical information.
“if we get these things wrong, or it’s life or death. I think we’re starting to understand that a lot of the things that we think of as interaction are only the top layer of what we’re actually doing all the time with other people. And if we don’t understand those lower layers, it could kill us.”
“So last week, I put together a proposal to study how people are interacting with one another in the city around the social distancing policy. And I agree the name is not perfect, but I think it also gets to the heart of what’s important to do. In this epidemic, there’s a halo effect around our social interactions because we know they’re necessary and good for us. And so people think, well, I shouldn’t go to the grocery stores and go to the hospital. But surely, it can’t be bad to go visit my neighbor or surely it can’t be bad to go see my grandmother, these kind of inclinations will kill us, when taken to scale.”
“When we say social distancing, we’re saying like, yes, school is good, but school is bad in the situation, churches good, but churches are bad in the situation, really getting at the thing that we are so tempted to do that is literally the thing that we’re trying to stop right now. I think that’s why they call it social distancing. And it does definitely have a physical corollary. I’m interested to see afterwards, if those people who were playing basketball and other people who were playing soccer, are those places where people got more sick or not? We don’t actually know all the different mechanisms for transmission for disease. And I think later on, we’ll be able to figure it out.” [Wendy Ju March 31 2020]
Cory Kidd is the CEO and founder of Catalia Health, which uses social robots for medical care management. Catalia Health has done extensive clinical trials prior to commercial roll out and leads the world in understanding robots for medical care.
“The concept of chronic disease management of course is not new, it’s just that the usual model is very human powered. We do it in clinical settings, in the doctor’s office, in the hospital, and we send people out to homes, and a lot of the work is done by calling patients on the phone to check in on them. We replace all of those by putting actually a physical robot in the patient’s home to talk to them.
“So what we’re doing on the AI side is generating conversations for patients on the fly, for whatever condition they’re dealing with, and we build these around specific conditions. The robotic piece of it is really driven by the psychology around why we would rather be all in a room together, as opposed to, gathering around our computers and staring into the the screen at zoom. We intuitively get that physical presence is different. “
“When we’re face to face with someone, they’re more engaged, we create stronger relationships and a number of other things. Research showed that those differences actually carry over into the future. When you put a cute little robot in front of someone that can look them in the eyes while it’s talking to them, we actually get a lot of the effects of face to face interaction. And so we’ve leveraged that to build chronic disease care management programs. Over the last couple of years, we’ve been rolling these out largely in specialty pharmacy, so we work with some of the largest pharma manufacturers in the world, like Pfizer. We’re helping patients across a number of different conditions really keep track of how they’re doing, to stay on therapy and stay out of the hospital using our AI and robotics platform.”
“The current situation around the world is really highlighting the need for more of this kind of technology.” [Cory Kidd March 31 2020]
Ken Goldberg is the director of CITRIS People and Robots Initiative, and the William s. Floyd Jr. Distinguished Chair in Engineering at UC Berkeley. Both Ken and Robin are amongst the authors of recent editorial in Science Robotics “Combating COVID-19 – The role of robotics in manging public health and infectious diseases”.
“There’s this fundamental issue that I’ve been thinking a lot about, which is protecting the health care workers, especially where they’re now having to provide these tests for huge numbers of people. Swabbing is quite an uncomfortable and invasive process. And is there any way that that that we might be able to automate that at some point? I don’t think that’s going to happen anytime soon. But it’s an interesting goal that we could move in that direction.”
“The other is the idea that intubation is an incredibly difficult process and very risky because a lot of droplets vaporizing happens. That’s another area where it would be very helpful if that could be teleoperated. Right now, the state of the art in telemedicine, tele surgery in particular, and these type of procedures is not ready for the situation we’re facing now. We are nowhere near capable of doing that. And so I think this is a really important wake up call to start to develop these technologies.”
Also, in the discussion, Jessica Armstrong who is a mechanical engineer at SuitX and local coordinator for Open Source COVID-19 Medical Supplies gave us updates on local PPE activities and how community grass roots initiatives like OSCMS and Helpful Engineering have been part of catalyzing networks of people to sew masks and gowns, to laser cut face shields and 3D print parts for PPE and medical equipment, and developing new designs for emergency ventilators and respirators, while we’re still waiting for manufacturers and the supply chain to meet the demand.
Perhaps most critically, groups like OSCMS and Helpful Engineering validate and share designs for PPE so that people aren’t wasting time designing their own solutions, nor putting health care workers at risk with badly designed homemade PPE.
Our second weekly discussion about “COVID-19, robots and us” from March 31 is now available online and as a podcast. You can sign up to join the audience for the next episodes here.
Special guests were Robin Murphy, Raytheon Professor at Texas A&M University and founder of the field of Rescue Robotics, Ross Mead, CEO of Semio and VP of AI LA, Wendy Ju, Interaction Design Professor at Cornell, Cory Kidd, CEO of Catalia Health – maker of medical social robots, Ken Goldberg, Director of CITRIS People and Robots Initiative and Jessica Armstrong, mechanical engineer at SuitX and local coordinator for Open Source Covid-19 Medical Supplies. Moderated by Andra Keay, Managing Director of Silicon Valley Robotics, with extra help from Erin Pan, Silicon Valley Robotics, and Beau Ambur from Kickstarter.