GelSight has a feel for the future
by Milan Kocic, Senior Director Sixth Sense, Hexagon’s Manufacturing Intelligence division
Engineering Reality 2023 volume 2 edition
Empowering Makers for an autonomous sustainable future
Sixth Sense, the open innovation platform from Hexagon’s Manufacturing Intelligence division, has identified a partnership with potential in its second cohort winner, GelSight – let’s meet the company
When MIT professor and neuroscientist Ted Adelson had his daughter, it changed his life far beyond the usual sleep deprivation. Professor Adelson, one of the world’s experts in human and machine vision, became obsessed with touch.
“His whole career was about how human vision works and how you can tie that to computer vision,” explains GelSight CEO Youssef Benmokhtar. “But when his daughter was born, he was actually fascinated by the fact that an infant relies on touch more than vision to explore the world in its first few weeks. It was an Isaac Newton moment. He decided to switch fields.”
Adelson posed the question – is there a way to build a sensor that is soft like a human finger and can capture touch signals with high sensitivity? “Now MIT is a place where things happen,” says Youssef. “Everyone said no, nothing like that exists today. So, he got to work in his garage and came up with the gel material that we use in our sensors.”
With the skin-like sensitive gel created, Professor Adelson needed to transfer the data from the sensor. Given his history as an expert in vision, he decided to put a camera behind the gel, so the camera could see what the gel feels. “That’s how simple it is,” says Youssef. “It’s like having a camera inside your finger. That’s the genesis of the company.”
GelSight was part of the second cohort of Sixth Sense, chosen as one of eight innovative scaling start-ups to take part in our intensive eight-week programme which would see them work on proof of technology, proof of value, and deliver a persuasive pitch at the end of the programme. Each of these start-ups was answering a challenge in the areas of visualisation and digital reality, or AI and automation.
Spun out of MIT, GelSight’s tactile intelligence technology now digitises touch with human-like sensitivity. Most commonly used as a handheld device mounted on one of Hexagon’s Absolute Arm portable measuring arms (but also compatible with mounting onto robotic arms), GelSight’s products have a soft, high-resolution, tactile sensor that conforms like human skin to the shape of an object on contact, precisely capturing the topography of the surface. “Wherever vision struggles or is not capable of giving you the understanding of the physical world, such as on a transparent surface or in bright or low light, GelSight provides you that understanding through touch,” says Youssef.
Co-founder Kimo Johnson, with whom we also worked closely during the Sixth Sense programme, took Professor Adelson’s sensor and created the algorithms behind it. “These allow you to create a 3D map of anything the sensor touches,” explains Youssef.
When the innovative system was first invented, Professor Adelson was working on understanding how it could give robots the same sense of touch as humans. Would this enable them to start doing complex object manipulations? As the product progressed, it became clear that one of the best use cases for the invention was in surface inspection.
“The fact that our device works in any environment, makes it a really good metrology device,” explains Youssef. “It’s accurate to the micron level – beyond millimetres and not quite nanometers.”
This minute accuracy has meant that GelSight’s biggest customer base to date has been in the aerospace industry – searching for scratches and dents in the fuselage, wings, and engine blades of aircraft. “Aerospace is a high precision industry,” says Youssef. “They really care about the microns because when an aircraft flies, there is so much force applied to every part of the plane that a tint crack could mean potential engine failure or an airframe breaking.”
GelSight is enabling a traditional industry to digitise a historically analogue, manual action. “A lot of these factories have been using what they call the ‘fingernail test,’” says Youssef. “They have just been scratching the surfaces with their nails and saying whether something feels out of spec or not. We replace that method so that it is no longer subjective.”
This sensitivity is opening up the possibility for aircraft manufacturers to define how big a scratch has to be before it is a fail or a pass – making everyone’s job easier and the world a little safer. “From there, we’ve been offering more and more things like measuring hole diameters or the radii of curvature,” he explains. “Because we have this very precise 3D map, we can add more capabilities overtime that we hear our customers need. Anything to simplify their workflow.”
With customers such as Rolls Royce (aerospace), French aircraft maker Safran and cargo company Kalitta Air – about half of GelSight’s revenue comes from aerospace. This was much higher but diversifying became important during the Covid-19 crisis so now automotive and forensics are double digit markets, and the team is exploring opportunities in oil and gas, renewable energies and other verticals.
“When cars are prepped for the paint shop, for example, they use transparent coatings first which of course is quite hard to check with the naked eye,” says Youssef.
GelSight’s role in aiding smart manufacturing is one thing. But the company isn’t afraid of a more leftfield project. The Norwegian ski team has publicly acknowledged GelSight’s important role in their success as a gold medallist. The company’s 3D measurement device helped the team to measure snow on the slopes to help skiers to understand how the surface of their skis would interact with snow.
GelSight is also currently working with Meta. “It’s part of the robotics play,” says Youssef, “providing that sense of touch. Meta has an AI group and we’ve worked with them to build some sensors for robotics.”
Figure 1. A GelSight sensor mounted on one of Hexagon’s Absolute Arm portable measuring arms, which situates the sensor’s output within a global coordinate referencing system.
Luckily, for Youssef and the team, this was a genuinely worthwhile investment. “Once you got to the 10-week finish line, regardless of the outcome, you were a better company. The coaches are very high quality, and they challenge you about the way you think about and present your company. We looked at ourselves from a different angle.”
Some of these outcomes were, to a degree, expected. But Youssef also realised there was another benefit to the scaling business. “It gave the team a huge boost in motivation,” he explains. “I found out after the Summit that everyone in the whole company stopped working and watched the live stream. People had more pride in the company, a clearer understanding of what we had been working on with Sixth Sense but also about our potential as a company. I had no idea it would have this impact and it’s amazing.”
Since winning, Youssef has had a number of different conversations with stakeholders within Hexagon and now hopes to be able to formalise what the partnership might be and what it could achieve. “I believe the potential is absolutely there and this is a huge opportunity. We see the market need when we talk to our customers every day and I think combined with the might of Hexagon, we could scale and reach more customers and satisfy more needs.”
For GelSight, there is a clear use case within metrology. But of course, its potential doesn’t end there. “There are trillions of tactile interactions in this world on a daily basis,” says Youssef. “It’s one of the most important senses that we have. GelSight is the only company that I know of that has been able to digitise that signal. So, for me, the potential for the company is that any tactile interaction now has the potential to be digitised. The applications are endless.”