Touchless control of heavy equipment using low-cost hand gesture recognition

Loading...
Thumbnail Image

Authors

Khaleghi, Leyla
Artan, Unal
Etemad, Ali
Marshall, Joshua

Date

2022-03-01

Type

journal article

Language

en

Keyword

Research Projects

Organizational Units

Journal Issue

Alternative Title

Abstract

Human-machine interaction using remote hand gestures is becoming increasingly prevalent across various industries. However, their potential application to heavy construction equipment is often overlooked. This paper presents a robust and inexpensive hand gesture recognition system that was implemented and tested on a robotic 1-tonne wheel loader. The system uses an RGB camera paired with a laptop to process, in real time, hand gestures to control the loader. We first design 4 unique gestures for controlling the loader and then collect 26000 images to train and test a neural network for hand gesture recognition. Our system uses robust landmark detection using an off-the-shelf system prior to gesture recognition. We successfully controlled the loader to excavate in a rock pile by using the proposed hand gesture recognition system.

Description

Citation

L. Khaleghi, U. Artan, A. Etemad, and J. A. Marshall. Touchless control of heavy equipment using low-cost hand gesture recognition. In the Special Issue on An End-to-end Machine Learning Perspective on Industrial IoT of the IEEE Internet of Things Magazine, vol. 5, no. 1, March 2022.

Publisher

IEEE

License

Journal

Volume

Issue

PubMed ID

ISSN

EISSN

Collections