keyboard_arrow_up
Enhancing user Safety with Real-Time Object Detection and Vibration Feedback: A Mobile Application integrated with Yolov5, Flutter, and nRF52 Hardware

Authors

Zhefu Lei, Lexington Christian Academy, USA

Abstract

Providing a means of navigation for hard of seeing individuals has always been a challenge. The most infamous is the white walking cane that people usually associate with the hard of seeing. However, while the cane has undoubtedly helped many navigate their surroundings, the limitations posed are overtly clear when it comes to navigating with a white cane. The solution posed in this paper, the EchoSense, is a vest composed of an onboard microprocessor, distance sensors, haptic motors, and a corresponding mobile application. The vest will alert the user of immediate obstructions to their path using the distance sensors and haptic motors, while the mobile app uses advanced real-time object detection to alert the user of specific objects in their path via vibrations from the mobile app as well as vibrations from the vest in the direction of the recognized object [1]. The onboard microprocessor is a bluefruit nRF52, and uses Adafruit distance sensors and haptic motor controllers. The mobile application was built using the Flutter framework, and is available on both iOS and Google Play stores [2]. During the development process, many challenges were encountered and overcame. The biggest challenge posed was the development of the mobile application, as compiling many different libraries in a very bleeding edge framework is bound to have some conflicting dependencies. To test to ensure the efficacy and efficiency of the EchoSense project, multiple rounds of experimenting were performed to find the response time of the real-time object detection and hardware [3]. All experimental results were satisfactory, and can be viewed in this paper. The EchoSense is a great choice for the hard of seeing to rely on for their daily navigation, as it is cost effective, efficient, and provides a great deal of information about the user’s surroundings without overwhelming their senses.

Keywords

Real-time object detection, Mobile application, Yolov5, Bluetooth connectivity, Vibration feedback

Full Text  Volume 15, Number 19