AFERSM-Net: Joint Network for Gesture Recognition and Location Classification
Keywords:
Gesture recognition, Location classification, Noise reduction, Feature extraction, Deep learningAbstract
With the widespread deployment of wireless communication systems and smart devices, gesture recognition and indoor location classification technologies based on WiFi wireless devices are increasingly used. Its technical principle is to identify human activities and locations by extracting gesture and location features from WiFi channel state information (CSI). However, the signal is susceptible to interference from the environment during CSI data acquisition to produce multipath effect noise, and the amplitude change with the change of location often affects the extraction and recognition of gesture features. To address these problems, Auxiliary Feature Extraction based Residual Shrinkage Multi-tasking Network (AFERSM-Net) is proposed for gesture recognition and position classification of one-dimensional multivariate time series. AFERSM-Net is a hybrid architecture that combines CNN for spatial feature extraction and LSTM networks for capturing temporal dependencies. Firstly, a reasonable threshold is set adaptively by the shrinkage module to dynamically identify and eliminate the transformed environmental noise. Secondly, the feature extraction module is used to focus on and extract location-independent gesture features to reduce the influence of location-independent features. Finally, the gesture features extracted by the feature extraction module are fused with the shared features of the residual shrinkage multi-tasking network as an aid. Its module fusion is mainly used to improve the accuracy of gesture recognition and solve the problem of insufficient model generalization ability. We evaluated this network on a dual-labeled gesture and location dataset, an the gesture recognition accuracy was 97.84% and the location classification accuracy was 98.92%, which outperformed other advanced network frameworks.
Downloads
References
Y. Liu, W. Zhou, M. Xi, S. Shen, and H. Li, “Multi-modal context propagation for person re-identification with wireless positioning,”
IEEE Transactions on Multimedia, vol. 24, pp. 3060–3073, 2021.doi:10.1109/TMM.2021.3092579.
Y. Cao, Z. Zhou, C. Zhu, P. Duan, and J. Li, “A lightweight deep learning algorithm for wifi-based identity recognition,” IEEE Internet of Things Journal, vol. 8, pp. 17449–17459, 2021.doi:10.1109/JIOT.2021.3078782.
H. Yang, Z. Xia, J. Shin, J. Hua, Y. Mao, and S. Zhong, “A comprehensive study of trajectory forgery and detection in location-based services,”IEEE Transactions on Mobile Computing, vol. 23, pp. 3228–3242, 2024. doi:10.1109/TMC.2023.3273411.
S. Zamboni, Z. T. Kefato, S. Girdzijauskas, N. Christoffer, and L. D.Col, “Pedestrian trajectory prediction with convolutional neural networks,” Pattern Recognition, vol. 121, pp. 108252–108263, 2021.doi:10.1016/j.patcog.2021.108252.
S. D. Regani, B. Wang, and K. Liu, “Wifi-based device-free gesture recognition through-the-wall,” in ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021. doi:10.1109/ICASSP39728.2021.9414894.
Q. Y, P. S, and Z. Li, “Cross-domain extendable gesture recognition system using wifi signals,” Electronics Letters, vol. 59, 2023.
doi:/10.1049/ell2.12931.
C. S. Álvarez Merino, H. Q. Luo-Chen, E. J. Khatib, and R. Barco, “Wifi ftm, uwb and cellular-based radio fusion for indoor positioning,” Sensors, vol. 21, p. 7020, 2021. doi:10.3390/s21217020.
H. Gu, J. Yang, G. Gui, and H. Gacanin., “Triplet matchnet based indoor position method using csi fingerprint similarity comparison,”
IEEE Transactions on Vehicular Technology, vol. 72, pp. 16905–16910, 2023. doi:10.1109/TVT.2023.3289631.
J. Liu, B. Jia, L. Guo, B. Huang, L. Wang, and T. Baker, “Ctsloc: An indoor localization method based on cnn by using time-series rssi,”
Cluster Computing, pp. 1–12, 2021. doi:10.1007/s10586-021-03458-2.
X. Song, X. Fan, C. Xiang, Q. Ye, and G. Fang, “A novel convolutional neural network-based indoor localization framework with
wifi fingerprinting,” IEEE Access, vol. 7, pp. 110698–110709, 2019. doi:10.1109/access.2019.2933921.
J. Li, T. Jiang, J. Yu, X. Ding, Y. Zhong, and Y. Liu, “An wifi-based human activity recognition system under multi-source interference,” in
Springer, vol. 878, pp. 937–944, 2022. doi:10.1007/978-981-19-0390-8_118.
P. F. Moshiri, H. Navidan, R. Shahbazian, S. A. Ghorashi, and D. Windridge, “Using gan to enhance the accuracy of indoor human activity recognition,” China Communications, 2020. doi:arxiv-2004.11228.
Z. Tang, Q. Liu, M. Wu, W. Chen, and J. Huang, “Wifi csi gesture recognition based on parallel lstm-fcn deep space-time neural network,” China Communications, vol. 18, no. 3, pp. 205–215, 2021. doi:10.23919/jcc.2021.03.016.
H. Wang, S. Wang, X. Wang, T. Liu, and Y. Wang, “Rdts noise reduction: A fast method study based on signal waveform type,” Optical Fiber Technology, vol. 65, no. 1, p. 102594, 2021.doi:10.1016/j.yofte.2021.102594.
A. Leshem and U. Erez, “The interference channel revisited: Aligning interference by adjusting antenna separation,” IEEE Transactions on Signal Processing, vol. 69, pp. 1874–1884, 2021. doi:10.1109/tsp.2021.3063442.
F. Wang, J. Feng, Y. Zhao, X. Zhang, and J. Han, “Joint activity recognition and indoor localization with wifi fingerprints,” IEEE Access, vol. 34, pp. 158–170, 2019. doi:10.1109/access.2019.2923743.
Y. Chen, C. Wang, K. Xiong, and Z. Huang, “Synchronized perturbation elimination and doa estimation via signal selection mechanism and parallel deep capsule networks in multipath environment,”Chinese Journal of Aeronautics, vol. 34, no. 4, pp. 158–170, 2021. doi:10.1016/j.cja.2021.01.016.
J. Yang, Y. Liu, Z. Liu, Y. Wu, T. Li, and Y. Yang, “A framework for human activity recognition based on wifi csi signal enhancement,”
International Journal of Antennas and Propagation, vol. 2021, 2021. doi:10.1155/2021/6654752.
M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep residual shrinkage networks for fault diagnosis,” IEEE Transactions
on Industrial Informatics, vol. 16, no. 7, pp. 4681–4690, 2020. doi:10.1109/tii.2019.2943898.
Z. Hao, J. Niu, X. Dang, and Z. Qiao, “Wipg: Contactless action recognition using ambient wi-fi signals,” Sensors, vol. 22, no. 1, p. 402,
doi:10.3390/s22010402.
Y. Ma, S. Arshad, S. Muniraju, E. Torkildson, and G. Zhou, “Locationand person-independent activity recognition with wifi, deep neural networks, and reinforcement learning,” ACM Transactions on Internet of Things, vol. 2, pp. 1–25, 2021. doi:10.1145/3424739.
Y.-S. Chen, Y.-C. Chang, and C.-Y. Li, “A semi-supervised transfer learning with dynamic associate domain adaptation for human activity recognition using wifi signals,” Sensors, vol. 21, no. 24, p. 8475, 2021. doi:10.3390/s21248475.
A. Alitaleshi, H. Jazayeriy, and J. Kazemitabar, “Affinity propagation clustering-aided two-label hierarchical extreme learning machine for wifi fingerprinting-based indoor positioning,” Journal of Ambient Intelligence and Humanized Computing, vol. 13, no. 6, pp. 3303–3317, 2022. doi:10.1007/s12652-022-03777-1.
R. Lu, W. Chen, C. Zhang, L. Li, Y. Peng, and Z. Zheng, “Characteristics of the bds-3 multipath effect and mitigation methods using precise point positioning,” GPS Solutions, vol. 26, no. 2, pp. 1–17, 2022. doi:10.1007/s10291-022-01227-1.
S. Xu and X. Yang, “A long term memory recognition framework on multi-complexity motion gestures,” in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), 2018. doi:10.1109/ICDAR.2017.41.
F. Karim, S. Majumdar, H. Darabi, and S. Harford, “Multivariate lstmfcns for time series classification,” Neural Networks, vol. 116, pp. 237–245, 2018. doi:10.1016/j.neunet.2019.04.014.
X. Zhang, C. Tang, Y. An, and K. Yin, “Wifi-based multi-task sensing,” Mobile and Ubiquitous Systems: Computing, Networking and Services, vol. 419, pp. 169–189, 2022. doi:10.1007/978-3-030-94822-1_10
