Egocentric Gesture Cues for Collaborative Avatar Understanding in Virtual Laboratory Sessions and Team Interaction Cues

Authors

  • Florence Aubert aculty of Engineering and Information Technology, University of Technology Sydney, Sydney, Australia
  • Ivy Fleming aculty of Engineering and Information Technology, University of Technology Sydney, Sydney, Australia

Keywords:

Virtual Laboratories, Egocentric Gestures, Avatar Understanding, Collaborative Virtual Environments

Abstract

The proliferation of immersive technologies has significantly advanced the capabilities of collaborative virtual environments, particularly in the domain of virtual laboratories. However, a critical bottleneck remains in the accurate translation and mutual understanding of non-verbal communication, specifically hand gestures, between collaborators. This paper investigates the framework of collaborative avatar understanding driven by egocentric gestures in virtual scientific laboratories. By capturing first-person gestural inputs and translating them into meaningful third-person avatar kinematics, we establish a paradigm that enhances behavioral fidelity and mutual comprehension. We analyze the theoretical foundations of egocentric gesture recognition, mapping these inputs to complex avatar interactions within shared digital workspaces. Through a comprehensive methodological approach involving simulated laboratory tasks, we evaluate the impact of high-fidelity gesture transmission on task efficiency, cognitive load, and perceived social presence. Our findings suggest that integrating precise egocentric gesture tracking into avatar behavior significantly reduces ambiguity in collaborative problem-solving scenarios, fostering a more intuitive and seamless cooperative experience. The research provides a robust foundation for future developments in telepresence, immersive education, and remote scientific collaboration, addressing existing limitations in communicative bandwidth within virtual spaces.

References

1. Ma, Y., & Qu, D. (2025, August). GEL-FMO: Gauge-Equivariant Liquid Fourier-Markov Operators for Uncertainty-Certified Multimodal Reasoning. In 2025 5th International Conference on Advanced Algorithms and Neural Networks (AANN) (pp. 604-607). IEEE.

2. Wang, M., Li, J., Li, Z., Luo, C., Chen, B., Xia, S. T., & Wang, Z. (2023, October). Unsupervised anomaly detection with local-sensitive VQVAE and global-sensitive transformers. In 2023 IEEE International Conference on Image Processing (ICIP) (pp. 1080-1084). IEEE.

3. Lv, Q., Kong, W., Li, H., Zeng, J., Qiu, Z., Qu, D., ... & Pang, J. (2025). F1: A vision-language-action model bridging understanding and generation to actions. arXiv preprint arXiv:2509.06951.

4. Z. Qi et al., "Self-Regulated Learning for Egocentric Video Activity Anticipation" in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 45, no. 06, pp. 6715-6730, June 2023, doi: 10.1109/TPAMI.2021.3059923.

5. Tu, P., Xie, X., Ai, G., Li, Y., Huang, Y., & Zheng, Y. (2023). FemtoDet: An object detection baseline for energy versus performance tradeoffs. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 13318-13327).

6. Li, Z., Chen, Z., Qu, F., Wang, M., Zhao, Y., Zhang, K., & Peng, Y. (2024). Point resampling and ray transformation aid to editable NeRF models. arXiv preprint arXiv:2405.07306.

7. Ma, W., Li, Y., Liu, C., Zhang, H., Li, J., Chen, K., & Gao, W. (2026). GeoCraft: A Diffusion Model-Based 3D Reconstruction Method Driven by Image and Point Cloud Fusion. Information Fusion, 104149.

8. Guo, H., Lu, C., Bao, F., Pang, T., Yan, S., Du, C., & Li, C. (2023). Gaussian mixture solvers for diffusion models. Advances in Neural Information Processing Systems, 36, 25598-25626.

9. Sui, M., Jiang, L., Lyu, T., Wang, H., Zhou, L., Chen, P., & Alhosain, A. (2024). Application of deep learning models based on efficientdet and openpose in user-oriented motion rehabilitation robot control. Journal of Intelligence Technology and Innovation, 2(3), 47-77.

10. Chu, Y. B., Li, J., Jia, P., Cui, J., Zhang, R., Kang, X., Lv, M., & Zhang, S. (2021). Irf1- and Egr1-activated transcription plays a key role in macrophage polarization: A multiomics sequencing study with partial validation. International immunopharmacology, 99, 108072. https://doi.org/10.1016/j.intimp.2021.108072

11. Song, S., Tang, Y., & Qin, R. (2025). Synthetic Data Matters: Re-training with Geo-typical Synthetic Labels for Building Detection. IEEE Transactions on Geoscience and Remote Sensing.

12. Zhang, Jiaquan, et al. "Lightweight llm agent memory with small language models." arXiv preprint arXiv:2604.07798 (2026).

13. Qu, W., Shao, Y., Meng, L., Huang, X., & Xiao, L. (2024). A conditional denoising diffusion probabilistic model for point cloud upsampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 20786-20795).

14. Qi, Z., Yuan, Y., Ruan, X., Wang, S., Zhang, W., & Huang, Q. (2024). Collaborative debias strategy for temporal sentence grounding in video. IEEE Transactions on Circuits and Systems for Video Technology, 34(11), 10972-10986.

15. Qu, D., & Ma, Y. (2025). Magnet-bn: markov-guided Bayesian neural networks for calibrated long-horizon sequence forecasting and community tracking. Mathematics, 13(17), 2740.

16. Guo, Hanzhong, et al. "Leveraging verifier-based reinforcement learning in image editing." arXiv preprint arXiv:2604.27505 (2026).

17. Mi, L., Yuan, T., Wang, W., Dai, H., Sun, L., Zheng, J., ... & Fu, X. (2024). Accelerated neural enhancement for video analytics with video quality adaptation. IEEE/ACM Transactions on Networking, 32(4), 3045-3060.

18. Peng, Q., Zheng, C., & Chen, C. (2024). A dual-augmentor framework for domain generalization in 3d human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2240-2249).

19. Qu, W., Wang, J., Gong, Y., Huang, X., & Xiao, L. (2025). An end-to-end robust point cloud semantic segmentation network with single-step conditional diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 27325-27335).

20. Tu, P., Huang, Y., Zheng, F., He, Z., Cao, L., & Shao, L. (2022, June). Guidedmix-net: Semi-supervised semantic segmentation by using labeled images as reference. In Proceedings of the AAAI conference on artificial intelligence (Vol. 36, No. 2, pp. 2379-2387).

21. Huang, H., Zhang, J., Zhang, J., Xu, J., & Wu, Q. (2020). Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification. IEEE Transactions on Multimedia, 23, 1666-1680.

22. Yang, Y., Ma, X., Li, C., Zheng, Z., Zhang, Q., Huang, G., ... & Zhao, Q. (2021). Believe what you see: Implicit constraint approach for offline multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 34, 10299-10312.

23. Zhang, J., Zhang, J., Song, Z., Shi, Z., Zhao, C., Shi, Y., ... & Wang, J. (2024). Hoi-m^ 3: Capture multiple humans and objects interaction within contextual environment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 516-526).

24. Kim, E. H., Wang, Z., Zong, H., Jiang, Z., Fu, Z., & Pedrycz, W. (2023). Design of tobacco leaves classifier through fuzzy clustering-based neural networks with multiple histogram analyses of images. IEEE Transactions on Industrial Informatics, 20(3), 4698-4709.

25. Zhao, Y., Chen, H., Liu, C., Li, Z., Herrmann, C., Hur, J., ... & Xu, M. (2025). Toward Material-Agnostic System Identification from Videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 5944-5956).

26. Xu, W., Huang, H., Gong, Y., Yu, L., Wu, Q., & Zhang, J. (2025). Hierarchical multi-prototype discrimination: Boosting support-query matching for few-shot segmentation. IEEE Transactions on Multimedia.

27. Wang, J., Wells III, W. M., Golland, P., & Zhang, M. (2018, September). Efficient laplace approximation for bayesian registration uncertainty quantification. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 880-888). Cham: Springer International Publishing.

28. Ding, Y., Li, Z., Huang, D., Li, Z., & Zhang, K. (2022, October). Enhancing multi-view stereo with contrastive matching and weighted focal loss. In 2022 IEEE International Conference on Image Processing (ICIP) (pp. 821-825). IEEE.

29. Zhao, Y., Li, Z., Guo, X., & Lu, Y. (2022). Alignment-guided temporal attention for video action recognition. Advances in Neural Information Processing Systems, 35, 13627-13639.

30. Cong, P., Xu, Y., Ren, Y., Zhang, J., Xu, L., Wang, J., ... & Ma, Y. (2023, June). Weakly supervised 3d multi-person pose estimation for large-scale scenes based on monocular camera and single lidar. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 37, No. 1, pp. 461-469).

31. Li, Z., Bai, X., Lu, J., Shen, P., Lam, E. Y., & Peng, Y. (2026). EventTracer: Fast Path Tracing-based Event Stream Rendering. IEEE Transactions on Visualization and Computer Graphics.

32. Zhang, B., Wang, X., Kao, H., Zhang, G., Gao, C., Wang, Y., ... & Chen, K. (2026). Consensus Matrix: A Role-Specialized Multi-Agent Framework for Structured Collaborative Decision-Making in Agentic Visual Media Workflows. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4613-4622).

33. Ziyue Wang, Sheng Jin, Zhongrong Zuo, Jiawei Wu, Han Qiu, Qi She, Hao Zhang, and Xudong Jiang. Video-ktr: Re?inforcing video reasoning via key token attribution. arXiv preprint arXiv:2601.19686, 2026.

34. Qu, D., Ma, Y., & Zhang, S. (2025, November). OAMF: Optics-Accelerated Multimodal Learning with Markov Temporal Priors and Fourier Regularization. In 2025 4th International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) (pp. 600-605). IEEE.

35. Li, Q., Xu, T., Luo, T., Zhong, Y., Li, Y., Zhou, Y., & Hou, C. (2026). Theory-driven label-specific representation for incomplete multi-view multi-label learning. Advances in Neural Information Processing Systems, 38, 121996-122037.

36. Ding, Y., Li, Z., Huang, D., Zhang, K., Li, Z., & Feng, W. (2022). Adaptive range guided multi-view depth estimation with normal ranking loss. In Proceedings of the Asian Conference on Computer Vision (pp. 1892-1908).

37. Li, Q., Liu, Z., Xu, T., Luo, T., & Hou, C. (2026). Adaptive Disentangled Representation Learning for Incomplete Multi-View Multi-Label Classification. arXiv preprint arXiv:2601.05785.

38. Liu, Y., & Kwon, H. (2025). Efficient Depth Estimation for Unstable Stereo Camera Systems on AR Glasses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6252-6261).

39. Ma, Y., & Qu, D. (2025, November). GELNO-FD: gauge-equivariant Fourier liquid neural operators for interpretable Markovian Bayesian dynamics. In Fifth International Conference on Advanced Algorithms and Signal Image Processing (AASIP 2025) (Vol. 13967, pp. 177-183). SPIE.

Downloads

Published

2026-07-12

Issue

Vol. 2 No. 1 (2026): Journal of Social Sciences, Humanities and Innovation

Section

Articles

License

Copyright (c) 2026 Florence Aubert, Ivy Fleming

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

All articles published in the Journal of Social Sciences, Humanities and Innovation (JSSHI) are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).

This license permits unrestricted use, distribution, reproduction, and adaptation in any medium or format, provided that the original author(s) and source are properly credited. Users are allowed to copy, redistribute, remix, transform, and build upon the material for any purpose, including commercial use, as long as appropriate attribution is given and any changes made are indicated.

Authors retain copyright of their work while granting the journal the right of first publication. The journal supports open access principles and encourages the wide dissemination of scholarly research to promote knowledge sharing and academic collaboration.

For more information about the license, visit: https://creativecommons.org/licenses/by/4.0/