I am Jiangning Zhang (张江宁), currently working as a Principal Researcher of two teams (Industry Perception and AIGC) at YouTu Lab, Tencent, Shanghai (技术大咖计划). I receive Ph.D. degree in College of Control Science and Engineering, Zhejiang University, Hangzhou, China, under the supervision of Prof. Yong Liu. My research interests include:
🌱 Neural Architecture Design: transformer-based architecture, light-wight vision model.
🌱 Multi-modal GAN-/VAE-/Diffusion-based AIGC researches: image/video generation, multi-modal human-centric editing and generation, and 2D/3D virtual digital human related researches (3D face/body/hand/cloth reconstruction, multi-modal digital human animation, motion generation, etc.).
🌱 3D Scene Segmentation via Foundation Models, Visual Anomaly Classification and Segmentation.

💬 Feel free to drop me emails (186368@zju.edu.cn) if you have interests on above topics, and remote cooperations are welcomed.
💬 You can contact me if you are applying for a Research Intern or a B.S./Ph.D. student in computer vision / robotic perception, and I co-supervise students with Prof. Yong Liu at Zhejiang University.
💖 I love 📷photography, 🍲cooking, and 🌏traveling, enjoy together!!!!!!
📝 Publications

🔥 News

📝 Publications

2024  2023  2022  2021  2020  Challenges 

* joint first author | # corresponding author
Summary: IJCV (2), TPAMI (2), TIP (1)
Summary: CVPR (13), ICCV (5), ECCV (10), NeurIPS (4)

2024

1. MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection [Paper] [Project] [Code]
   Haoyang He, Yuhu Bai, Jiangning Zhang, Qingdong He, Hongxu Chen, Zhenye Gan, Chengjie Wang, Xiangtai Li, Guanzhong Tian, and Lei Xie
   Conference on Neural Information Processing Systems (NeurIPS). 2024.
2. MotionBooth: Motion-Aware Customized Text-to-Video Generation [Paper] [Project] [Code]
   Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen
   Conference on Neural Information Processing Systems (NeurIPS). 2024.
3. Dual-path Frequency Discriminators for Few-shot Anomaly Detection [Paper] [Code]
   Yuhu Bai*, Jiangning Zhang*, Zhaofeng Chen, Yuhang Dong, Yunkang Cao, and Guanzhong Tian
   Knowledge-Based Systems (KBS). 2024.
4. MDT-A2G: Exploring Masked Diffusion Transformers for Co-Speech Gesture Generation [Paper] [Project]
   Xiaofeng Mao, Zhengkai Jiang, Qilin Wang, Chencan Fu, Jiangning Zhang, Jiafu Wu, Yabiao Wang, Chengjie Wang, Wei Li, and Mingmin Chi
   ACM Multimedia (ACM MM). 2024.
5. MambaGesture: Enhancing Co-Speech Gesture Generation with Mamba and Disentangled Multi-Modality Fusion [Paper] [Project]
   Chencan Fu, Yabiao Wang, Jiangning Zhang, Zhengkai Jiang, Xiaofeng Mao, Jiafu Wu, Weijian Cao, Chengjie Wang, Yanhao Ge, and Yong Liu
   ACM Multimedia (ACM MM). 2024.
6. MotionMaster: Training-free Camera Motion Transfer For Video Generation [Paper] [Code] [Project]
   Teng Hu*, Jiangning Zhang*, Ran Yi, Yating Wang, Hongrui Huang, Jieyu Weng, Yabiao Wang, and Lizhuang Ma
   ACM Multimedia (ACM MM). 2024.
7. DiffuMatting: Synthesizing Arbitrary Objects with Matting-level Annotation [Paper]
   Xiaobin Hu, Xu Peng, Donghao Luo, Xiaozhong Ji, Jinlong Peng, Zhengkai Jiang, Jiangning Zhang, Taisong Jin, Chengjie Wang, and Rongrong Ji
   European Conference on Computer Vision (ECCV). 2024.
8. Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control [Paper] [Code] [Project]
   Yue Han, Junwei Zhu, Keke He, Xu Chen, Yanhao Ge, Wei Li, Xiangtai Li, Jiangning Zhang#, Chengjie Wang, and Yong Liu
   European Conference on Computer Vision (ECCV). 2024.
9. TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation [Paper] [Code] [Project]
   Yufei Liu, Junwei Zhu, Junshu Tang, Shijie Zhang, Jiangning Zhang, Weijian Cao, Chengjie Wang, Yunsheng Wu, and Dongjin Huang
   European Conference on Computer Vision (ECCV). 2024.
10. AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection [Paper]
    Yunkang Cao, Jiangning Zhang, Luca Frittoli, Yuqi Cheng, Weiming Shen, and Giacomo Boracchi
    European Conference on Computer Vision (ECCV). 2024.
11. FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesis [Paper]
    Ke Fan, Junshu Tang, Weijian Cao, Ran Yi, Moran Li, Jingyu Gong, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Lizhuang Ma
    European Conference on Computer Vision (ECCV). 2024.
12. Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection [Paper]
    Yuanpeng Tu, Boshen Zhang, Liang Liu, Yuxi Li, Xuhai Chen, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Cai Rong Zhao
    European Conference on Computer Vision (ECCV). 2024.
13. Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentation [Paper] [Code]
    Yue Han*, Jiangning Zhang*, Zhucun Xue, Chao Xu, Xintian Shen, Yabiao Wang, Chengjie Wang, Yong Liu, and Xiangtai Li
    Transactions on Pattern Analysis and Machine Intelligence (T-PAMI). 2024.
14. GPT-4V-AD: Exploring Grounding Potential of VQA-oriented GPT-4V for Zero-shot Anomaly Detection [Paper] [Code]
    Jiangning Zhang, Haoyang He, Xuhai Chen, Zhucun Xue, Yabiao Wang, Chengjie Wang, Lei Xie, and Yong Liu
    International Joint Conference on Artificial Intelligence (IJCAI) ADFM. 2024.
15. CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection [Paper]
    Xuhai Chen, Jiangning Zhang, Guanzhong Tian, Haoyang He, Wuhao Zhang, Yabiao Wang, Chengjie Wang, and Yong Liu
    International Joint Conference on Artificial Intelligence (IJCAI) ADFM. 2024.
16. UniM-OV3D: Uni-Modality Open-Vocabulary 3D Scene Understanding with Fine-Grained Feature Representation [Paper] [Code]
    Qingdong He, Jinlong Peng, Zhengkai Jiang, Kai Wu, Xiaozhong Ji, Jiangning Zhang#, Yabiao Wang, Chengjie Wang, Mingang Chen, and Yunsheng Wu
    International Joint Conference on Artificial Intelligence (IJCAI). 2024.
17. Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection [Paper] [Code] [Project]
    Chengjie Wang, Wenbing Zhu, Bin-Bin Gao, Zhenye Gan, Jiangning Zhang, Zhihao Gu, Shuguang Qian, Mingang Chen, and Lizhuang Ma
    Computer Vision and Pattern Recognition (CVPR). 2024.
18. Towards Language-Driven Video Inpainting Via Multimodal Large Language Models [Paper] [Code] [Project]
    Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yunhai Tong, Ziwei Liu, and Chen Change Loy
    Computer Vision and Pattern Recognition (CVPR). 2024.
19. SuperSVG: Superpixel-Based Scalable Vector Graphics Synthesis [Paper] [Code]
    Teng Hu, Ran Yi, Baihong Qian, Jiangning Zhang, Paul Rosin, and Yukun Lai
    Computer Vision and Pattern Recognition (CVPR). 2024.
20. PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization [Paper] [Project]
    Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, and Rongrong Ji
    Computer Vision and Pattern Recognition (CVPR). 2024.
21. EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm [Paper] [Code]
    Jiangning Zhang, Xiangtai Li, Yabiao Wang, Chengjie Wang, Yibo Yang, Yong Liu, and Dacheng Tao
    International Journal of Computer Vision (IJCV). 2024.
22. Towards open vocabulary learning: A survey [Paper] [Code]
    Jianzong Wu, Xiangtai Li, Shilin Xu, Haobo Yuan, Henghui Ding, Yibo Yang, Xia Li, Jiangning Zhang, Yunhai Tong, Xudong Jiang, Bernard Ghanem, and Dacheng Tao
    Transactions on Pattern Analysis and Machine Intelligence (T-PAMI). 2024.

2023

1. Hear to Segment: Unmixing the Audio to Guide the Semantic Segmentation [Paper]
   Yuhang Ling, Yuxi Li, Zhenye Gan, Jiangning Zhang, Mingmin Chi, and Yabiao Wang
   International Conference on Acoustics, Speech, and Signal Processing (ICASSP). 2024.
2. DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [Paper] [Code] [Project]
   Haoyang He*, Jiangning Zhang*, Hongxu Chen, Xuhai Chen, Zhishan Li, Xu Chen, Yabiao Wang, Chengjie Wang, and Lei Xie
   Association for the Advancement of Artificial Intelligence (AAAI). 2024.
3. Rethinking Reverse Distillation for Multi-Modal Anomaly Detection [Paper]
   Zhihao Gu, Jiangning Zhang, Liang Liu, Xu Chen, Jinlong Peng, Zhenye Gan, Guannan Jiang, Annan Shu, Yabiao Wang, and Lizhuang Ma
   Association for the Advancement of Artificial Intelligence (AAAI). 2024.
4. Self-supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes [Paper]
   Yuanpeng Tu, Yuxi Li, Boshen Zhang, Liang Liu, Jiangning Zhang, Yabiao Wang, and Cairong Zhao
   Association for the Advancement of Artificial Intelligence (AAAI). 2024.
5. AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model [Paper] [Code] [Project]
   Teng Hu*, Jiangning Zhang*, Ran Yi, Yuzhen Du, Xu Chen, Liang Liu, Yabiao Wang, and Chengjie Wang
   Association for the Advancement of Artificial Intelligence (AAAI). 2024.
6. PVG: Progressive Vision Graph for Vision Recognition [Paper]
   Jiafu Wu, Jian Li, Jiangning Zhang, Boshen Zhang, Mingmin Chi, Yabiao Wang, and Chengjie Wang
   ACM Multimedia (ACM MM). 2023.
7. Toward High Quality Facial Representation Learning [Paper] [Code]
   Yue Wang, Jinlong Peng, Jiangning Zhang, Ran Yi, Liang Liu, Yabiao Wang, and Chengjie Wang
   ACM Multimedia (ACM MM). 2023.
8. Rethinking Mobile Block for Efficient Attention-based Models [Paper] [Code]
   Jiangning Zhang, Xiangtai Li, Jian Li, Liang Liu, Zhucun Xue, Boshen Zhang, Zhengkai Jiang, Tianxin Huang, Yabiao Wang, and Chengjie Wang
   International Conference on Computer Vision (ICCV). 2023.
9. Remembering Normality: Memory-guided Knowledge Distillation for Unsupervised Anomaly Detection [Paper]
   Zhihao Gu, Liang Liu, Xu Chen, Ran Yi, Jiangning Zhang, Yabiao Wang, Chengjie Wang, Annan Shu, Guannan Jiang, and Lizhuang Ma
   International Conference on Computer Vision (ICCV). 2023.
10. Learning Global-aware Kernel for Image Harmonization [Paper]
    Xintian Shen*, Jiangning Zhang*, Jun Chen, Shipeng Bai, Yue Han, Yabiao Wang, Chengjie Wang, and Yong Liu
    International Conference on Computer Vision (ICCV). 2023.
11. Phasic Content Fusing Diffusion Model with Directional Distribution Consistency for Few-Shot Model Adaption [Paper]
    Teng Hu*, Jiangning Zhang*, Liang Liu, Ran Yi, Siqi Kou, Haokun Zhu, Xu Chen, Yabiao Wang, Chengjie Wang, and Lizhuang Ma
    International Conference on Computer Vision (ICCV). 2023.
12. SFNet: Faster and Accurate Semantic Segmentation Via Semantic Flow [Paper] [Code]
    Xiangtai Li, Jiangning Zhang, Yibo Yang, Guangliang Cheng, Kuiyuan Yang, Yunhai Tong, and Dacheng Tao
    International Journal of Computer Vision (IJCV). 2023.
13. Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detection [Paper] [Code]
    Yufei Liang*, Jiangning Zhang*, Shiwei Zhao, Runze Wu, Yong Liu, and Shuwen Pan
    Transactions on Image Processing (TIP). 2023.
14. Multimodal Industrial Anomaly Detection via Hybrid Fusion [Paper] [Code]
    Yue Wang, Jinlong Peng, Jiangning Zhang, Ran Yi, Yabiao Wang, and Chengjie Wang
    Computer Vision and Pattern Recognition (CVPR). 2023.
15. Learning To Measure the Point Cloud Reconstruction Loss in a Representation Space [Paper]
    Tianxin Huang, Zhonggan Ding, Jiangning Zhang, Ying Tai, Zhenyu Zhang, Mingang Chen, Chengjie Wang, and Yong Liu
    Computer Vision and Pattern Recognition (CVPR). 2023.
16. MixTeacher: Mining Promising Labels With Mixed Scale Teacher for Semi-Supervised Object Detection [Paper] [Code]
    Liang Liu, Boshen Zhang, Jiangning Zhang, Wuhao Zhang, Zhenye Gan, Guanzhong Tian, Wenbing Zhu, Yabiao Wang, and Chengjie Wang
    Computer Vision and Pattern Recognition (CVPR). 2023.
17. Better "CMOS" Produces Clearer Images: Learning Space-Variant Blur Estimation for Blind Image Super-Resolution [Paper]
    Xuhai Chen*, Jiangning Zhang*, Chao Xu, Yabiao Wang, Chengjie Wang, and Yong Liu
    Computer Vision and Pattern Recognition (CVPR). 2023.
18. High-Fidelity Generalized Emotional Talking Face Generation With Multi-Modal Emotion Space Learning [Paper]
    Chao Xu, Junwei Zhu, Jiangning Zhang, Yue Han, Wenqing Chu, Ying Tai, Chengjie Wang, Zhifeng Xie, and Yong Liu
    Computer Vision and Pattern Recognition (CVPR). 2023.
19. Learning With Noisy Labels via Self-Supervised Adversarial Noisy Masking [Paper] [Code]
    Yuanpeng Tu, Boshen Zhang, Yuxi Li, Liang Liu, Jian Li, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Cairong Zhao
    Computer Vision and Pattern Recognition (CVPR). 2023.

2022

1. Calibrated Teacher for Sparsely Annotated Object Detection [Paper] [Code]
   Haohan Wang, Liang Liu, Boshen Zhang, Jiangning Zhang, Wuhao Zhang, Zhenye Gan, Yabiao Wang, Chengjie Wang, and Haoqian Wang
   Association for the Advancement of Artificial Intelligence (AAAI). 2023.
2. Fast Point Cloud Sampling Network [Paper]
   Tianxin Huang, Jun Chen, Jiangning Zhang, Yong Liu, and Jie Liang
   Pattern Recognition Letters (PRL). 2023.
3. 3QNet: 3D Point Cloud Geometry Quantization Compression Network [Paper]
   Tianxin Huang, Jiangning Zhang, Jun Chen, Zhonggan Ding, Ying Tai, Zhenyu Zhang, Chengjie Wang, and Yong Liu
   ACM Transactions on Graphics (ACM TOG). 2022.
4. Adaptive Recurrent Forward Network for Dense Point Cloud Completion [Paper]
   Tianxin Huang, Hao Zou, Jinhao Cui, Jiangning Zhang, Xuemeng Yang, Lin Li, and Yong Liu
   Transactions on Multimedia (TMM). 2022.
5. Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping [Paper] [Code]
   Chao Xu*, Jiangning Zhang*, Yue Han, Guanzhong Tian, Xianfang Zeng, Ying Tai, Yabiao Wang, Chengjie Wang, and Yong Liu
   European Conference on Computer Vision (ECCV). 2022.
6. Resolution-free Point Cloud Sampling Network with Data Distillation [Paper] [Code]
   Tianxin Huang*, Jiangning Zhang*, Jun Chen, Yuang Liu, and Yong Liu
   European Conference on Computer Vision (ECCV). 2022.
7. Learning to Train a Point Cloud Reconstruction Network without Matching [Paper] [Code]
   Tianxin Huang, Xuemeng Yang, Jiangning Zhang, Jinhao Cui, Hao Zou, Jun Chen, Xiangrui Zhao, and Yong Liu
   European Conference on Computer Vision (ECCV). 2022.
8. Multilevel Spatial-Temporal Feature Aggregation for Video Object Detection [Paper]
   Chao Xu, Jiangning Zhang, Mengmeng Wang, Guanzhong Tian, and Yong Liu
   Transactions on Circuits and Systems for Video Technology (TCSVT). 2022.
9. Learning Hierarchical and Efficient Person Re-Identification for Robotic Navigation [Paper]
   Jiangning Zhang, Chao Xu, Xiangrui Zhao, Liang Liu, Yong Liu, Jinqiang Yao, and Zaisheng Pan
   International Journal of Intelligent Robotics and Applications (IJIRA, Best Paper Reward Nomination). 2022.
10. Iterative Few-shot Semantic Segmentation from Image Label Text [Paper] [Code]
    Haohan Wang, Liang Liu, Wuhao Zhang, Jiangning Zhang, Zhenye Gan, Yabiao Wang, Chengjie Wang, and Haoqian Wang
    International Joint Conference on Artificial Intelligence (IJCAI). 2022.
11. Region-Aware Face Swapping [Paper] [Code]
    Chao Xu*, Jiangning Zhang*, Miao Hua, Qian He, Zili Yi, and Yong Liu
    Computer Vision and Pattern Recognition (CVPR). 2022.

2021

1. SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution [Paper]
   Jiangning Zhang, Chao Xu, Jian Li, Yue Han, Yabiao Wang, Ying Tai, and Yong Liu
   Association for the Advancement of Artificial Intelligence (AAAI). 2022.
2. Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model [Paper] [Code]
   Jiangning Zhang, Chao Xu, Jian Li, Wenzhou Chen, Yabiao Wang, Ying Tai, Shuo Chen, Chengjie Wang, Feiyue Huang, and Yong Liu
   Conference on Neural Information Processing Systems (NeurIPS). 2021.
3. Real-Time Audio-Guided Multi-Face Reenactment [Paper] [Code]
   Jiangning Zhang, Xianfang Zeng, Chao Xu, and Yong Liu
   Signal Processing Letters (SPL). 2021.
4. Adding Before Pruning: Sparse Filter Fusion for Deep Convolutional Neural Networks via Auxiliary Attention [Paper] [Code]
   Guanzhong Tian, Yiran Sun, Yuang Liu, Xianfang Zeng, Mengmeng Wang, Yong Liu, Jiangning Zhang, and Jun Chen
   Transactions on Neural Networks and Learning Systems (TNNLS). 2021.
5. RFNet: Recurrent Forward Network for Dense Point Cloud Completion [Paper] [Code]
   Tianxin Huang, Hao Zou, Jinhao Cui, Xuemeng Yang, Mengmeng Wang, Xiangrui Zhao, Jiangning Zhang, Yi Yuan, Yifan Xu, and Yong Liu
   International Conference on Computer Vision (ICCV). 2021.

2020

1. DTVNet: Dynamic Time-Lapse Video Generation via Single Still Image [Paper] [Code]
   Jiangning Zhang*, Chao Xu*, Liang Liu, Mengmeng Wang, Xia Wu, Yong Liu, and Yunliang Jiang
   European Conference on Computer Vision (ECCV). 2020.
2. FReeNet: Multi-Identity Face Reenactment [Paper] [Code]
   Jiangning Zhang, Xianfang Zeng, Mengmeng Wang, Yusu Pan, Liang Liu, Yong Liu, Yu Ding, and Changjie Fan
   Computer Vision and Pattern Recognition (CVPR). 2020.
3. Learning by Analogy: Reliable Supervision From Transformations for Unsupervised Optical Flow Estimation [Paper] [Code]
   Liang Liu, Jiangning Zhang, Ruifei He, Yong Liu, Yabiao Wang, Ying Tai, Donghao Luo, Chengjie Wang, Jilin Li, and Feiyue Huang
   Computer Vision and Pattern Recognition (CVPR). 2020.
4. APB2FACE: Audio-Guided Face Reenactment with Auxiliary Pose and Blink Signals [Paper] [Code]
   Jiangning Zhang, Liang Liu, Zhucun Xue, and Yong Liu
   International Conference on Acoustics, Speech, and Signal Processing (ICASSP). 2020.
5. Realistic Face Reenactment via Self-Supervised Disentangling of Identity and Pose [Paper]
   Xianfang Zeng, Yusu Pan, Mengmeng Wang, Jiangning Zhang, and Yong Liu
   Association for the Advancement of Artificial Intelligence (AAAI). 2020.
6. Learning-Based Hand Motion Capture and Understanding in Assembly Process [Paper]
   Liang Liu, Yong Liu, and Jiangning Zhang
   Transactions on Industrial Electronics (TIE). 2018.

Challenges

1. 1st place for [Zero-shot Track] and 4th place for [Few-shot Track] in [Visual Anomaly and Novelty Detection (VAND) 2023 Challenge] by CVPR 2023
   A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD [Paper] [Code]
   Xuhai Chen*, Yue Han*, Jiangning Zhang*
   Computer Vision and Pattern Recognition (CVPR) VAND. 2023.

📖 Educations

• 2020.04 - 2022.10, M.D. 👉️ Ph.D. in College of Control Science and Engineering, Zhejiang University, Hangzhou, China.
• 2017.09 - 2020.04, Pursuing M.D. in College of Control Science and Engineering, Zhejiang University, Hangzhou, China.
• 2013.09 - 2017.06, Obtained B.S. in Electronic Information School, Wuhan University, Wuhan, China.

💻 Internships

• 2020.12 - 2022.10, YouTu Lab, Tencent, Shanghai, mentored by Researcher Yabiao Wang and Dr. Ying Tai.
• 2020.06 - 2020.10, 2012 Lab, Huawei, Hangzhou.
• 2019.07 - 2019.09, Guangzi Technique Center, Tencent, Shenzhen.
• 2019.04 - 2019.06, Fuxi AI Lab, NetEase, Hangzhou.