From motion to human activity recognition

Alhersh, Taha

PDF
Alhersh_2021_Approved_to_be_submitted_and_printed.pdf - Published
Download (8MB)

URL:	https://madoc.bib.uni-mannheim.de/59204
URN:	urn:nbn:de:bsz:180-madoc-592041
Document Type:	Doctoral dissertation
Year of publication:	2021
Place of publication:	Mannheim
University:	Universität Mannheim
Evaluator:	Stuckenschmidt, Heiner
Date of oral examination:	26 March 2021
Publication language:	English
Institution:	School of Business Informatics and Mathematics > Practical Computer Science II: Artificial Intelligence (Stuckenschmidt 2009-)
Subject:	004 Computer science, internet
Keywords (English):	computer vision , motion , optical flow , activity recognition , metrics
Abstract:	Advancements in wearable technology have the potential to transform the quality of life, business, and the global economy. Body sensors can be used in human activity recognition, which has direct impact on various application domains such as surveillance systems, healthcare systems, robotics, and other physical and metrological applications. Human activity recognition can be considered as a problem in both computer vision and pervasive computing. In this research, we started from a computer vision problem based on optical flow, and then introduced open issues in respect to existing techniques. We went on to study the relation between optical flow and human activity recognition, taking into consideration the effectiveness of using optical flow combined with other wearable sensors. As a result, feasible solutions have been presented to solve those problems, and then useful insights have been given for implementing corresponding techniques. A comprehensive set of experiments and discussions were performed during the research. Firstly, we suggested an unsupervised optical flow fine-tuning that overcomes the need for a ground truth for training on the one hand and enhanced motion boundaries on the other. Secondly, we provided theoretical justification for optical flow evaluation metrics. Moreover, we suggested novel optical flow performance metrics that have been evaluated alongside current metrics. Our empirical findings examined the performance of all metrics with regard to their sensitivity to change in motion between estimated optical flow and the ground truth. Finally, we investigated methods regarding feature extraction for Inertial Measurement Units (IMUs) and visual data captured from wearable sensors, for instance, statistical features, local visual descriptors, and features extracted from deep learning. The features generated were tested for human activity recognition using Support Vector Machines and Recurrent Neural Network as the main recognition methods.