Touchless control of heavy equipment using low-cost hand gesture recognition
Loading...
Authors
Khaleghi, Leyla
Artan, Unal
Etemad, Ali
Marshall, Joshua
Date
2022-03-01
Type
journal article
Language
en
Keyword
Alternative Title
Abstract
Human-machine interaction using remote hand gestures is becoming increasingly prevalent across various industries. However, their potential application to heavy construction equipment is often overlooked. This paper presents a robust and inexpensive hand gesture recognition system that was implemented and tested on a robotic 1-tonne wheel loader. The system uses an RGB camera paired with a laptop to process, in real time, hand gestures to control the loader. We first design 4 unique gestures for controlling the loader and then collect 26000 images to train and test a neural network for hand gesture recognition. Our system uses robust landmark detection using an off-the-shelf system prior to gesture recognition. We successfully controlled the loader to excavate in a rock pile by using the proposed hand gesture recognition system.
Description
Citation
L. Khaleghi, U. Artan, A. Etemad, and J. A. Marshall. Touchless control of heavy equipment using low-cost hand gesture recognition. In the Special Issue on An End-to-end Machine Learning Perspective on Industrial IoT of the IEEE Internet of Things Magazine, vol. 5, no. 1, March 2022.
Publisher
IEEE