Editable Emotional Talking-Head Synthesis Through Geometry-Aware Diffusion Over Dynamic Gaussian Representations
Keywords:
Talking-Head Synthesis, Diffusion Models, Gaussian Splatting, Emotion Editing, Facial AnimationAbstract
The synthesis of highly realistic and emotionally controllable talking-head videos remains a formidable challenge in computer vision and computer graphics. Existing frameworks predominantly rely on generative adversarial networks or implicit neural representations, which often suffer from either a lack of three-dimensional geometric consistency or prohibitive computational costs during rendering. Furthermore, infusing precise emotional states into synchronized lip movements without disrupting the underlying anatomical constraints poses an ongoing theoretical and practical dilemma. To address these multifaceted challenges, this paper presents a novel framework for editable emotional talking-head synthesis that leverages geometry-aware diffusion models operating over dynamic three-dimensional Gaussian representations. By representing the human head through an ensemble of explicit, trackable, and dynamically updatable Gaussian splats, the proposed architecture achieves unparalleled rendering efficiency and multi-view consistency. The core innovation lies in the geometry-aware diffusion mechanism, which learns to model the complex temporal dynamics of facial deformations and emotional micro-expressions in the parameter space of the Gaussian attributes. This allows for fine-grained editing of emotional states in a continuous latent space while preserving strict lip-sync fidelity. Extensive experimental evaluations demonstrate that the proposed method significantly outperforms state-of-the-art techniques across standard quantitative metrics, including visual quality, lip-sync accuracy, and emotional expressiveness. The explicit nature of the Gaussian representation also facilitates real-time inference and user-directed spatial editing.
References
1. Li, Y., Wang, S., & Tan, G. (2025). ID-NeRF: Indirect diffusion-guided neural radiance fields for generalizable view synthesis. Expert Systems with Applications, 266, 126068.
2. Cakmak, S., Zhou, E., & Gao, S. (2021, December). Contextual ranking and selection with Gaussian processes. In 2021 Winter Simulation Conference (WSC) (pp. 1-12). IEEE.
3. He, H., Sun, Z., Fan, J., & Mok, P. Y. (2025). TiPGAN: High-quality tileable textures synthesis with intrinsic priors for cloth digitization applications. Computer-Aided Design, 183, 103866.
4. Fang, S., Shen, I., Wang, Y., Tsai, Y. H., Yang, Y., Zhou, S., ... & Yang, M. H. (2025). Meshllm: Empowering large language models to progressively understand and generate 3d mesh. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 14061-14072).
5. Yang, Z., Hu, D., Guo, Q., Zuo, L., & Ji, W. (2023). Visual E 2 C: AI-driven visual end-edge-cloud architecture for 6G in low-carbon smart cities. IEEE Wireless Communications, 30(3), 204-210.
6. Zhang, J., Shi, Y., Ma, Y., Xu, L., Yu, J., & Wang, J. (2023, June). Ikol: Inverse kinematics optimization layer for 3d human pose and shape estimation via gauss-newton differentiation. In Proceedings of the AAAI conference on artificial intelligence (Vol. 37, No. 3, pp. 3454-3462).
7. Wang, Y., Yu, H., Tang, J., Gao, Y., Wang, X., Yue, Y., & Yang, Y. (2026). FilterGS: Traversal-Free Parallel Filtering and Adaptive Shrinking for Large-Scale LoD 3D Gaussian Splatting. arXiv preprint arXiv:2603.23891.
8. Z. Wang, R. Yuan, Z. Geng, H. Li, X. Qu, X. Li, S. Chen,
H. Fu, R. B. Dannenberg, and K. Zhang, ‘‘Singing timbre popularity
assessment based on multimodal large foundation model,’’ in Proc.
33rd ACM Int. Conf. Multimedia, Oct. 2025, pp. 12227–12236, doi:
10.1145/3746027.3758148.
9. Fan, J., Gao, B., Jin, H., & Jiang, L. (2022). UCC: Uncertainty guided cross-head co-training for semi-supervised semantic segmentation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 9937–9946). IEEE. https://doi.org/10.1109/CVPR52688.2022.00971
10. Meng, X., & Lin, J. (2026). AWCRL: Adaptive Weakly Supervised Cross-Modal Reliability Learning for resource-constrained proxy-state recognition. Pattern Recognition, 114998.
11. Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020). NeRF: Representing scenes as neural radiance fields for view synthesis. In Proceedings of ECCV.
12. Fan, Y., Tang, Z., Miyazaki, T., Huang, Y., & Omachi, S. (2026). A real-driven image synthesis framework for adapting pre-trained scene text detectors. Image and Vision Computing, 106027.
13. Kerbl, B., Kopanas, G., Leimkühler, T., & Drettakis, G. (2023). 3D Gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), Article 139.
14. Jiaxing, Z., & Hao, T. (2025). Sam2 for image and video segmentation: A comprehensive survey.
15. Li, Z., Bai, X., Zhang, T., Shen, P., Xu, W., & Peng, Y. (2025, October). Enhanced Velocity Field Modeling for Gaussian Video Reconstruction. In 2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 304-314). IEEE.
16. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of CVPR.
17. Qu, W., Shao, Y., Meng, L., Huang, X., & Xiao, L. (2024, June). A conditional denoising diffusion probabilistic model for point cloud upsampling. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 20786-20795). IEEE.
18. Jing, G., Gao, P., Lee, Y., Hu, Y., & Zhang, H. (2025). 3D-aided pedestrian representation learning for video-based person re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 35 (12), 12830-12845.
19. Yang, Y., & Zhang, S. (2025). Pixel-wise calibration for a multi-focus microscopic 3D imaging system.Optics and Lasers in Engineering,194, 109127.
20. Qu, W., Wang, J., Gong, Y., Huang, X., & Xiao, L. (2025, June). An end-to-end robust point cloud semantic segmentation network with single-step conditional diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 27325-27335). IEEE.
21. Sun, M., Wu, L., Deng, Y. K., Li, X., & Xue, F. Reconstructing Ground-Penetrating Radargrams into 3D Point Cloud Format for Building Inspection: A Web AR Approach Evaluated by Technology Acceptance Model.
22. Hao, Y., Xu, M., Ye, C., Qin, J., Lu, S., Qin, Y., & Han, X. (2026). LoFA: Learning to Predict Personalized Prior for Fast Adaptation of Visual Generative Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 21986-21996).
23. Wang, J., Malawade, A. V., Zhou, J., Yu, S. Y., & Al Faruque, M. A. (2024, January). Rs2g: Data-driven scene-graph extraction and embedding for robust autonomous perception and scenario understanding. In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 7478-7487). IEEE.
24. Ye, Z., Ma, S., Yang, J., Yang, X., Yang, Y., Gong, Z., ... & Wang, H. (2026, April). Object fidelity diffusion for remote sensing image generation. In International Conference on Learning Representations (Vol. 2026, pp. 3258-3278).
25. Liu, Y., Cai, Q., Wang, C., Yang, J., Fan, H., Dong, J., & Chen, S. (2024). Geometry-enhanced attentive multi-view stereo for challenging matching scenarios. IEEE Transactions on Circuits and Systems for Video Technology, 34(8), 7401-7416.
26. Wang, Y., Li, F. W., & Wang, J. (2026, May). Motion-Guided Semantic Alignment with Negative Prompts for Zero-Shot Video Action Recognition. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 10712-10716). IEEE.
27. Chang Liu, Tianjiao Jing, Chengcheng Ma, XuanQi Zhou, Zhengxuan Lian, Qin Jin, Hongliang Yuan, and Shi-Sheng Huang. 2025. EmoDiffTalk:Emotion-aware
Diffusion for Editable 3D Gaussian Talking Head. CoRR abs/2512.05991 (2025).
arXiv:2512.05991 doi:10.48550/ARXIV.2512.05991
28. Lyu, S., Mao, S., & Ma, Y. (2025). Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising. arXiv preprint arXiv:2511.03272.
29. Yang, Z., Ji, W., Guo, Q., & Wang, Z. (2023, October). Javp: Joint-aware video processing with edge-cloud collaboration for dnn inference. In Proceedings of the 31st ACM International Conference on Multimedia (pp. 9152-9160).
30. Cong, P., Xu, Y., Ren, Y., Zhang, J., Xu, L., Wang, J., ... & Ma, Y. (2023, June). Weakly supervised 3d multi-person pose estimation for large-scale scenes based on monocular camera and single lidar. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 37, No. 1, pp. 461-469).
31. Li, Y. K., Yu, Y. Z., Liu, Y. L., & Gou, C. (2022, October). MS-GCN: Multi-stream graph convolution network for driver head pose estimation. In 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC) (pp. 3819-3824). IEEE.
32. Zhang, W., Huang, M., Zhou, Y., Zhang, J., Yu, J., Wang, J., & Xu, L. (2024). Both2hands: Inferring 3d hands from both text prompts and body dynamics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2393-2404).
33. Li, Z., Chen, Z., Qu, F., Wang, M., Zhao, Y., Zhang, K., & Peng, Y. (2024). Point resampling and ray transformation aid to editable NeRF models. arXiv preprint arXiv:2405.07306.
34. Li, Y., Tan, G., & Gou, C. (2024). Cascaded iterative transformer for jointly predicting facial landmark, occlusion probability and head pose. International Journal of Computer Vision, 132(4), 1242-1257.
35. Li, H., Wang, Y., Huang, T., Huang, H., Wang, H., & Chu, X. (2025, October). Ld-rps: Zero-shot unified image restoration via latent diffusion recurrent posterior sampling. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 13684-13694). IEEE.
36. Yang, D., Ai, H., Liu, J., & He, B. (2022). Absolute scale estimation for underwater monocular visual odometry based on 2-D imaging sonar. Measurement, 190, 110665.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Ye-Jin Park, Seo-Hyun Song, Tae-Hyun Bae
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in the Journal of Social Sciences, Humanities and Innovation (JSSHI) are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
This license permits unrestricted use, distribution, reproduction, and adaptation in any medium or format, provided that the original author(s) and source are properly credited. Users are allowed to copy, redistribute, remix, transform, and build upon the material for any purpose, including commercial use, as long as appropriate attribution is given and any changes made are indicated.
Authors retain copyright of their work while granting the journal the right of first publication. The journal supports open access principles and encourages the wide dissemination of scholarly research to promote knowledge sharing and academic collaboration.
For more information about the license, visit: https://creativecommons.org/licenses/by/4.0/
