Human Activity Recognition From Sensorised Patient's Data in HealthcareA Streaming Deep Learning-Based Approach

  1. Sandro Hurtado 1
  2. José García-Nieto 1
  3. Anton Popov 2
  4. Ismael Navas-Delgado 1
  1. 1 Universidad de Málaga
    info
    Universidad de Málaga

    Málaga, España

    ROR https://ror.org/036b2ww28

    Geographic location of the organization Universidad de Málaga
  2. 2 Igor Sikorsky Kyiv Polytechnic Institute
Journal:
IJIMAI

ISSN: 1989-1660

Year of publication: 2023

Issue Title: Special Issue on AI-driven Algorithms and Applications in the Dynamic and Evolving Environments

Volume: 8

Issue: 1

Pages: 23-37

Type: Article

DOI: 10.9781/IJIMAI.2022.05.004 DIALNET GOOGLE SCHOLAR lock_openDialnet editor

More publications in: IJIMAI

Sustainable development goals

Abstract

Physical inactivity is one of the main risk factors for mortality, and its relationship with the main chronic diseases has experienced intensive medical research. A well-known method for assessing people’s activity is the use of accelerometers implanted in wearables and mobile phones. However, a series of main critical issues arise in the healthcare context related to the limited amount of available labelled data to build a classification model. Moreover, the discrimination ability of activities is often challenging to capture since the variety of movement patterns in a particular group of patients (e.g. obesity or geriatric patients) is limited over time. Consequently, the proposed work presents a novel approach for Human Activity Recognition (HAR) in healthcare to avoid this problem. This proposal is based on semi-supervised classification with Encoder-Decoder Convolutional Neural Networks (CNNs) using a combination strategy of public labelled and private unlabelled raw sensor data. In this sense, the model will be able to take advantage of the large amount of unlabelled data available by extracting relevant characteristics in these data, which will increase the knowledge in the innermost layers. Hence, the trained model can generalize well when used in real-world use cases. Additionally, real-time patient monitoring is provided by Apache Spark streaming processing with sliding windows. For testing purposes, a real-world case study is conducted with a group of overweight patients in the healthcare system of Andalusia (Spain), classifying close to 30 TBs of accelerometer sensor-based data. The proposed HAR streaming deep-learning approach properly classifies movement patterns in real-time conditions, crucial for long-term daily patient monitoring.

Bibliographic References

  • K. González, J. Fuentes, J. L. Márquez, “Physical inactivity, sedentary behavior and chronic diseases,” Korean journal of family medicine, vol. 38, no. 3, p. 111, 2017.
  • W. L. Haskell, S. N. Blair, J. O. Hill, “Physical activity: health outcomes and importance for public health policy,” Preventive medicine, vol. 49, no. 4, pp. 280–282, 2009.
  • J. Wang, Y. Chen, S. Hao, X. Peng, L. Hu, “Deep learning for sensor-based activity recognition: A survey,” Pattern Recognition Letters, vol. 119, pp. 3–11, 2019.
  • P. Bet, P. C. Castro, M. A. Ponti, “Fall detection and fall risk assessment in older person using wearable sensors: a systematic review,” International journal of medical informatics, 2019.
  • A. Bourke, J. O’brien, G. Lyons, “Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm,” Gait & posture, vol. 26, no. 2, pp. 194–199, 2007.
  • F. Bagala, C. Becker, A. Cappello, L. Chiari, K. Aminian, J. M. Hausdorff, W. Zijlstra, J. Klenk, “Evaluation of accelerometer-based fall detection algorithms on real- world falls,” PloS one, vol. 7, no. 5, 2012.
  • F. M. Palechor, A. De la Hoz Manotas, P. A. Colpas, J. S. Ojeda, R. M. Ortega, M. P. Melo, “Cardiovascular disease analysis using supervised and unsupervised data mining techniques.,” JSW, vol. 12, no. 2, pp. 81– 90, 2017.
  • D. Arifoglu, A. Bouchachia, “Activity recognition and abnormal behaviour detection with recurrent neural networks,” Procedia Computer Science, vol. 110, pp. 86– 93, 2017.
  • G. Kalouris, E. I. Zacharaki, V. Megalooikonomou, “Improving cnn-based activity recognition by data augmentation and transfer learning,” in 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), vol. 1, 2019, pp. 1387–1394, IEEE.
  • A. Papagiannaki, E. I. Zacharaki, K. Deltouzos, R. Orselli, A. Freminet, S. Cela, E. Aristodemou, M. Polycarpou, M. Kotsani, A. Benetos, et al., “Meeting challenges of activity recognition for ageing population in real life settings,” in 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom), 2018, pp. 1–6, IEEE.
  • C. A. Ronao, S.-B. Cho, “Human activity recognition with smartphone sensors using deep learning neural networks,” Expert systems with applications, vol. 59, pp. 235–244, 2016.
  • Y. Saez, A. Baldominos, P. Isasi, “A comparison study of classifier algorithms for cross-person physical activity recognition,” Sensors, vol. 17, no. 1, p. 66, 2017.
  • T. Lv, X. Wang, L. Jin, Y. Xiao, M. Song, “Margin- based deep learning networks for human activity recognition,” Sensors, vol. 20, no. 7, p. 1871, 2020.
  • F. Cruciani, A. Vafeiadis, C. Nugent, I. Cleland, P. McCullagh, K. Votis, D. Giakoumis, D. Tzovaras, L. Chen, R. Hamzaoui, “Feature learning for human activity recognition using convolutional neural networks,” CCF Transactions on Pervasive Computing and Interaction, vol. 2, no. 1, pp. 18–32, 2020.
  • J. R. Kwapisz, G. M. Weiss, S. A. Moore, “Activity recognition using cell phone accelerometers,” ACM SIGKDD Explorations Newsletter, vol. 12, no. 2, pp. 74– 82, 2011.
  • A. Reiss, D. Stricker, “Introducing a new benchmarked dataset for activity monitoring,” in 2012 16th International Symposium on Wearable Computers, 2012, pp. 108–109, IEEE.
  • R. Chereshnev, A. Kertész-Farkas, “Hugadb: Human gait database for activity recognition from wearable inertial sensor networks,” in International Conference on Analysis of Images, Social Networks and Texts, 2017, pp. 131–141, Springer.
  • M. Zhang, A. A. Sawchuk, “Usc-had: a daily activity dataset for ubiquitous activity recognition using wearable sensors,” in Proceedings of the 2012 ACM Conference on Ubiquitous Computing, 2012, pp. 1036– 1043.
  • D. Balabka, “Semi-supervised learning for human activity recognition using adversarial autoencoders,” in Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC ’19 Adjunct, New York, NY, USA, 2019, p. 685–688, Association for Computing Machinery.
  • O. D. Lara, M. A. Labrador, “A survey on human activity recognition using wearable sensors,” IEEE communications surveys & tutorials, vol. 15, no. 3, pp. 1192–1209, 2012.
  • A. Subasi, K. Khateeb, T. Brahimi, A. Sarirete, “Human activity recognition using machine learning methods in a smart healthcare environment,” in Innovation in Health Informatics, M. D. Lytras, A. Sarirete Eds., Next Gen Tech Driven Personalized MedSmart Healthcare, Academic Press, 2020, pp. 123 – 144, doi: https://doi.org/10.1016/B978-0-12-819043-2.00005-8.
  • D. Anguita, A. Ghio, L. Oneto, X. Parra, J. L. Reyes- Ortiz, “Human activity recognition on smartphones using a multiclass hardwarefriendly support vector machine,” in International workshop on ambient assisted living, 2012, pp. 216–223, Springer.
  • L. Bao, S. S. Intille, “Activity recognition from user- annotated acceleration data,” in International conference on pervasive computing, 2004, pp. 1–17, Springer.
  • W. Wu, S. Dasgupta, E. E. Ramirez, C. Peterson, G. J. Norman, “Classification accuracies of physical activities using smartphone motion sensors,” Journal of medical Internet research, vol. 14, no. 5, p. e130, 2012.
  • Y. Chen, Y. Xue, “A deep learning approach to human activity recognition based on single accelerometer,” in 2015 IEEE International Conference on Systems, Man, and Cybernetics, 2015, pp. 1488–1492, IEEE.
  • N. Y. Hammerla, S. Halloran, T. Plötz, “Deep, convolutional, and recurrent models for human activity recognition using wearables,” arXiv preprint arXiv:1604.08880, 2016.
  • M. Zeng, T. Yu, X. Wang, L. T. Nguyen, O. J. Mengshoel, I. Lane, “Semi-supervised convolutional neural networks for human activity recognition,” in 2017 IEEE International Conference on Big Data (Big Data), 2017, pp. 522–529.
  • A. D. Antar, M. Ahmed, M. A. R. Ahad, “Challenges in sensor-based human activity recognition and a comparative analysis of benchmark datasets: a review,” in 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), 2019, pp. 134–139, IEEE.
  • S. Slim, A. Atia, M. Elfattah, M. Mostafa, “Survey on human activity recognition based on acceleration data,” International Journal of Advanced Computer Science and Applications, vol. 10, pp. 84–98, 2019.
  • Z. Hussain, M. Sheng, W. E. Zhang, “Different approaches for human activity recognition: A survey,” arXiv preprint arXiv:1906.05074, 2019.
  • A. Gupta, K. Gupta, K. Gupta, K. Gupta, “A survey on human activity recognition and classification,” in 2020 International Conference on Communication and Signal Processing (ICCSP), 2020, pp. 0915–0919, IEEE.
  • S. Reddy, M. Mun, J. Burke, D. Estrin, M. Hansen, M. Srivastava, “Using mobile phones to determine transportation modes,” ACM Transactions on Sensor Networks (TOSN), vol. 6, no. 2, pp. 1–27, 2010.
  • F. Foerster, M. Smeja, J. Fahrenberg, “Detection of posture and motion by accelerometry: a validation study in ambulatory monitoring,” Computers in human behavior, vol. 15, no. 5, pp. 571–583, 1999.
  • J. R. Kwapisz, G. M. Weiss, S. A. Moore, “Activity recognition using cell phone accelerometers,” ACM SIGKDD Explorations Newsletter, vol. 12, p. 74–82, Mar. 2011, doi: 10.1145/1964897.1964918.
  • A. Reiss, M. Weber, D. Stricker, “Exploring and extending the boundaries of physical activity recognition,” in 2011 IEEE International Conference on Systems, Man, and Cybernetics, 2011, pp. 46–50.
  • M. Zhang, A. A. Sawchuk, “Usc-had: A daily activity dataset for ubiquitous activity recognition using wearable sensors,” in ACM International Conference on Ubiquitous Computing (Ubicomp) Workshop on Situation, Activity and Goal Awareness (SAGAware), Pittsburgh, Pennsylvania, USA, September 2012.
  • R. Chereshnev, A. Kertész-Farkas, “Hugadb: Human gait database for activity recognition from wearable inertial sensor networks,” in Analysis of Images, Social Networks and Texts, Cham, 2018, pp. 131–141, Springer International Publishing.
  • H. He, E. A. Garcia, “Learning from imbalanced data,” IEEE Transactions on knowledge and data engineering, vol. 21, no. 9, pp. 1263–1284, 2009.
  • N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, “Smote: synthetic minority over- sampling technique,” Journal of artificial intelligence research, vol. 16, pp. 321–357, 2002.
  • K. T. Nguyen, F. Portet, C. Garbay, “Dealing with imbalanced data sets for human activity recognition using mobile phone sensors,” 2018.
  • S. Ertekin, J. Huang, L. Bottou, L. Giles, “Learning on the border: active learning in imbalanced data classification,” in Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, 2007, pp. 127–136.
  • D. A. Cieslak, T. R. Hoens, N. V. Chawla, W. P. Kegelmeyer, “Hellinger distance decision trees are robust and skew-insensitive,” Data Mining and Knowledge Discovery, vol. 24, no. 1, pp. 136–158, 2012.
  • L. G. Fahad, S. F. Tahir, M. Rajarajan, “Activity recognition in smart homes using clustering based classification,” in 2014 22nd International Conference on Pattern Recognition, 2014, pp. 1348–1353, IEEE.
  • A. Bulling, U. Blanke, B. Schiele, “A tutorial on human activity recognition using body-worn inertial sensors,” vol. 46, no. 3, 2014, doi: 10.1145/2499621.
  • Z. Zhang, “Improved adam optimizer for deep neural networks,” in 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), 2018, pp. 1–2, IEEE.
  • S. Ioffe, C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.
  • J. Snoek, H. Larochelle, R. P. Adams, “Practical bayesian optimization of machine learning algorithms,” in Advances in neural information processing systems, 2012, pp. 2951–2959.
Data source: Dialnet