Jiangning Zhang (张江宁) works as a Principal Researcher of two teams (Industry Perception and AIGC) at YouTu Lab, Tencent, Shanghai. I receive Ph.D. degree in College of Control Science and Engineering, Zhejiang University, Hangzhou, China, under the supervision of Prof. Yong Liu. My major is Computer Vision and my research interests include:
🌱 GAN-/Diffusion-based AIGC researches with LLM, e.g., multi-modal image/video generation, 2D/3D virtual digital human related researches (3D face/body/hand reconstruction, multi-modal digital human drive, motion generation, etc.), text-to-image generation, multi-modal human-centric editing and generation, etc.
🌱 Neural Architecture Design (NAD), e.g., transformer-based architecture, light-wight vision model, etc.
🌱 Anomaly Classification and Segmentation.

💬 Feel free to drop me emails (186368@zju.edu.cn) if you have interests on above topics, and remote cooperations are welcomed.
💬 You can contact me if you are applying for a Research Intern or a B.S./Ph.D. student in computer vision / robotic perception, and I co-supervise students with Prof. Yong Liu at Zhejiang University.
💖 I love 📷photography, 🍲cooking, and 🌏traveling, enjoy together!!!!!!
📝 Publications

🔥 News

📝 Publications

2024  2023  2022  2021  2020  arXiv  Challenges 

* joint first author | # joint corresponding author

2024

1. UniM-OV3D: Uni-Modality Open-Vocabulary 3D Scene Understanding with Fine-Grained Feature Representation [Paper] [Code]
   Qingdong He, Jinlong Peng, Zhengkai Jiang, Kai Wu, Xiaozhong Ji, Jiangning Zhang#, Yabiao Wang, Chengjie Wang, Mingang Chen, and Yunsheng Wu
   International Joint Conference on Artificial Intelligence (IJCAI). 2024.
2. Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection [Paper] [Code] [Project]
   Chengjie Wang, Wenbing Zhu, Bin-Bin Gao, Zhenye Gan, Jiangning Zhang, Zhihao Gu, Shuguang Qian, Mingang Chen, and Lizhuang Ma
   Computer Vision and Pattern Recognition (CVPR). 2024.
3. Towards Language-Driven Video Inpainting Via Multimodal Large Language Models [Paper] [Code] [Project]
   Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yunhai Tong, Ziwei Liu, and Chen Change Loy
   Computer Vision and Pattern Recognition (CVPR). 2024.
4. SuperSVG: Superpixel-Based Scalable Vector Graphics Synthesis [Paper]
   Teng Hu, Ran Yi, Baihong Qian, Jiangning Zhang, Paul Rosin, and Yukun Lai
   Computer Vision and Pattern Recognition (CVPR). 2024.
5. PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization [Paper] [Project]
   Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, and Rongrong Ji
   Computer Vision and Pattern Recognition (CVPR). 2024.
6. EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm [Paper] [Code]
   Jiangning Zhang, Xiangtai Li, Yabiao Wang, Chengjie Wang, Yibo Yang, Yong Liu, and Dacheng Tao
   International Journal of Computer Vision (IJCV). 2024.
7. Towards open vocabulary learning: A survey [Paper] [Code]
   Jianzong Wu, Xiangtai Li, Shilin Xu, Haobo Yuan, Henghui Ding, Yibo Yang, Xia Li, Jiangning Zhang, Yunhai Tong, Xudong Jiang, Bernard Ghanem, and Dacheng Tao
   Transactions on Pattern Analysis and Machine Intelligence (T-PAMI). 2024.

2023

1. Hear to Segment: Unmixing the Audio to Guide the Semantic Segmentation [Paper]
   Yuhang Ling, Yuxi Li, Zhenye Gan, Jiangning Zhang, Mingmin Chi, and Yabiao Wang
   International Conference on Acoustics, Speech, and Signal Processing (ICASSP). 2024.
2. DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [Paper] [Code] [Project]
   Haoyang He*, Jiangning Zhang*, Hongxu Chen, Xuhai Chen, Zhishan Li, Xu Chen, Yabiao Wang, Chengjie Wang, and Lei Xie
   Association for the Advancement of Artificial Intelligence (AAAI). 2024.
3. Rethinking Reverse Distillation for Multi-Modal Anomaly Detection [Paper]
   Zhihao Gu, Jiangning Zhang, Liang Liu, Xu Chen, Jinlong Peng, Zhenye Gan, Guannan Jiang, Annan Shu, Yabiao Wang, and Lizhuang Ma
   Association for the Advancement of Artificial Intelligence (AAAI). 2024.
4. Self-supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes [Paper]
   Yuanpeng Tu, Yuxi Li, Boshen Zhang, Liang Liu, Jiangning Zhang, Yabiao Wang, and Cairong Zhao
   Association for the Advancement of Artificial Intelligence (AAAI). 2024.
5. AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model [Paper] [Code] [Project]
   Teng Hu*, Jiangning Zhang*, Ran Yi, Yuzhen Du, Xu Chen, Liang Liu, Yabiao Wang, and Chengjie Wang
   Association for the Advancement of Artificial Intelligence (AAAI). 2024.
6. PVG: Progressive Vision Graph for Vision Recognition [Paper]
   Jiafu Wu, Jian Li, Jiangning Zhang, Boshen Zhang, Mingmin Chi, Yabiao Wang, and Chengjie Wang
   ACM Multimedia (ACM MM). 2023.
7. Toward High Quality Facial Representation Learning [Paper] [Code]
   Yue Wang, Jinlong Peng, Jiangning Zhang, Ran Yi, Liang Liu, Yabiao Wang, and Chengjie Wang
   ACM Multimedia (ACM MM). 2023.
8. Rethinking Mobile Block for Efficient Attention-based Models [Paper] [Code]
   Jiangning Zhang, Xiangtai Li, Jian Li, Liang Liu, Zhucun Xue, Boshen Zhang, Zhengkai Jiang, Tianxin Huang, Yabiao Wang, and Chengjie Wang
   International Conference on Computer Vision (ICCV). 2023.
9. Remembering Normality: Memory-guided Knowledge Distillation for Unsupervised Anomaly Detection [Paper]
   Zhihao Gu, Liang Liu, Xu Chen, Ran Yi, Jiangning Zhang, Yabiao Wang, Chengjie Wang, Annan Shu, Guannan Jiang, and Lizhuang Ma
   International Conference on Computer Vision (ICCV). 2023.
10. Learning Global-aware Kernel for Image Harmonization [Paper]
    Xintian Shen*, Jiangning Zhang*, Jun Chen, Shipeng Bai, Yue Han, Yabiao Wang, Chengjie Wang, and Yong Liu
    International Conference on Computer Vision (ICCV). 2023.
11. Phasic Content Fusing Diffusion Model with Directional Distribution Consistency for Few-Shot Model Adaption [Paper]
    Teng Hu*, Jiangning Zhang*, Liang Liu, Ran Yi, Siqi Kou, Haokun Zhu, Xu Chen, Yabiao Wang, Chengjie Wang, and Lizhuang Ma
    International Conference on Computer Vision (ICCV). 2023.
12. SFNet: Faster and Accurate Semantic Segmentation Via Semantic Flow [Paper] [Code]
    Xiangtai Li, Jiangning Zhang, Yibo Yang, Guangliang Cheng, Kuiyuan Yang, Yunhai Tong, and Dacheng Tao
    International Journal of Computer Vision (IJCV). 2023.
13. Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detection [Paper] [Code]
    Yufei Liang*, Jiangning Zhang*, Shiwei Zhao, Runze Wu, Yong Liu, and Shuwen Pan
    Transactions on Image Processing (TIP). 2023.
14. Multimodal Industrial Anomaly Detection via Hybrid Fusion [Paper] [Code]
    Yue Wang, Jinlong Peng, Jiangning Zhang, Ran Yi, Yabiao Wang, and Chengjie Wang
    Computer Vision and Pattern Recognition (CVPR). 2023.
15. Learning To Measure the Point Cloud Reconstruction Loss in a Representation Space [Paper]
    Tianxin Huang, Zhonggan Ding, Jiangning Zhang, Ying Tai, Zhenyu Zhang, Mingang Chen, Chengjie Wang, and Yong Liu
    Computer Vision and Pattern Recognition (CVPR). 2023.
16. MixTeacher: Mining Promising Labels With Mixed Scale Teacher for Semi-Supervised Object Detection [Paper] [Code]
    Liang Liu, Boshen Zhang, Jiangning Zhang, Wuhao Zhang, Zhenye Gan, Guanzhong Tian, Wenbing Zhu, Yabiao Wang, and Chengjie Wang
    Computer Vision and Pattern Recognition (CVPR). 2023.
17. Better "CMOS" Produces Clearer Images: Learning Space-Variant Blur Estimation for Blind Image Super-Resolution [Paper]
    Xuhai Chen*, Jiangning Zhang*, Chao Xu, Yabiao Wang, Chengjie Wang, and Yong Liu
    Computer Vision and Pattern Recognition (CVPR). 2023.
18. High-Fidelity Generalized Emotional Talking Face Generation With Multi-Modal Emotion Space Learning [Paper]
    Chao Xu, Junwei Zhu, Jiangning Zhang, Yue Han, Wenqing Chu, Ying Tai, Chengjie Wang, Zhifeng Xie, and Yong Liu
    Computer Vision and Pattern Recognition (CVPR). 2023.
19. Learning With Noisy Labels via Self-Supervised Adversarial Noisy Masking [Paper] [Code]
    Yuanpeng Tu, Boshen Zhang, Yuxi Li, Liang Liu, Jian Li, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Cairong Zhao
    Computer Vision and Pattern Recognition (CVPR). 2023.

2022

1. Calibrated Teacher for Sparsely Annotated Object Detection [Paper] [Code]
   Haohan Wang, Liang Liu, Boshen Zhang, Jiangning Zhang, Wuhao Zhang, Zhenye Gan, Yabiao Wang, Chengjie Wang, and Haoqian Wang
   Association for the Advancement of Artificial Intelligence (AAAI). 2023.
2. Fast Point Cloud Sampling Network [Paper]
   Tianxin Huang, Jun Chen, Jiangning Zhang, Yong Liu, and Jie Liang
   Pattern Recognition Letters (PRL). 2023.
3. 3QNet: 3D Point Cloud Geometry Quantization Compression Network [Paper]
   Tianxin Huang, Jiangning Zhang, Jun Chen, Zhonggan Ding, Ying Tai, Zhenyu Zhang, Chengjie Wang, and Yong Liu
   ACM Transactions on Graphics (ACM TOG). 2022.
4. Adaptive Recurrent Forward Network for Dense Point Cloud Completion [Paper]
   Tianxin Huang, Hao Zou, Jinhao Cui, Jiangning Zhang, Xuemeng Yang, Lin Li, and Yong Liu
   Transactions on Multimedia (TMM). 2022.
5. Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping [Paper] [Code]
   Chao Xu*, Jiangning Zhang*, Yue Han, Guanzhong Tian, Xianfang Zeng, Ying Tai, Yabiao Wang, Chengjie Wang, and Yong Liu
   European Conference on Computer Vision (ECCV). 2022.
6. Resolution-free Point Cloud Sampling Network with Data Distillation [Paper] [Code]
   Tianxin Huang*, Jiangning Zhang*, Jun Chen, Yuang Liu, and Yong Liu
   European Conference on Computer Vision (ECCV). 2022.
7. Learning to Train a Point Cloud Reconstruction Network without Matching [Paper] [Code]
   Tianxin Huang, Xuemeng Yang, Jiangning Zhang, Jinhao Cui, Hao Zou, Jun Chen, Xiangrui Zhao, and Yong Liu
   European Conference on Computer Vision (ECCV). 2022.
8. Multilevel Spatial-Temporal Feature Aggregation for Video Object Detection [Paper]
   Chao Xu, Jiangning Zhang, Mengmeng Wang, Guanzhong Tian, and Yong Liu
   Transactions on Circuits and Systems for Video Technology (TCSVT). 2022.
9. Learning Hierarchical and Efficient Person Re-Identification for Robotic Navigation [Paper]
   Jiangning Zhang, Chao Xu, Xiangrui Zhao, Liang Liu, Yong Liu, Jinqiang Yao, and Zaisheng Pan
   International Journal of Intelligent Robotics and Applications (IJIRA, Best Paper Reward Nomination). 2022.
10. Iterative Few-shot Semantic Segmentation from Image Label Text [Paper] [Code]
    Haohan Wang, Liang Liu, Wuhao Zhang, Jiangning Zhang, Zhenye Gan, Yabiao Wang, Chengjie Wang, and Haoqian Wang
    International Joint Conference on Artificial Intelligence (IJCAI). 2022.
11. Region-Aware Face Swapping [Paper] [Code]
    Chao Xu*, Jiangning Zhang*, Miao Hua, Qian He, Zili Yi, and Yong Liu
    Computer Vision and Pattern Recognition (CVPR). 2022.

2021

1. SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution [Paper]
   Jiangning Zhang, Chao Xu, Jian Li, Yue Han, Yabiao Wang, Ying Tai, and Yong Liu
   Association for the Advancement of Artificial Intelligence (AAAI). 2022.
2. Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model [Paper] [Code]
   Jiangning Zhang, Chao Xu, Jian Li, Wenzhou Chen, Yabiao Wang, Ying Tai, Shuo Chen, Chengjie Wang, Feiyue Huang, and Yong Liu
   Conference on Neural Information Processing Systems (NeurIPS). 2021.
3. Real-Time Audio-Guided Multi-Face Reenactment [Paper] [Code]
   Jiangning Zhang, Xianfang Zeng, Chao Xu, and Yong Liu
   Signal Processing Letters (SPL). 2021.
4. Adding Before Pruning: Sparse Filter Fusion for Deep Convolutional Neural Networks via Auxiliary Attention [Paper] [Code]
   Guanzhong Tian, Yiran Sun, Yuang Liu, Xianfang Zeng, Mengmeng Wang, Yong Liu, Jiangning Zhang, and Jun Chen
   Transactions on Neural Networks and Learning Systems (TNNLS). 2021.
5. RFNet: Recurrent Forward Network for Dense Point Cloud Completion [Paper] [Code]
   Tianxin Huang, Hao Zou, Jinhao Cui, Xuemeng Yang, Mengmeng Wang, Xiangrui Zhao, Jiangning Zhang, Yi Yuan, Yifan Xu, and Yong Liu
   International Conference on Computer Vision (ICCV). 2021.

2020

1. DTVNet: Dynamic Time-Lapse Video Generation via Single Still Image [Paper] [Code]
   Jiangning Zhang*, Chao Xu*, Liang Liu, Mengmeng Wang, Xia Wu, Yong Liu, and Yunliang Jiang
   European Conference on Computer Vision (ECCV). 2020.
2. FReeNet: Multi-Identity Face Reenactment [Paper] [Code]
   Jiangning Zhang, Xianfang Zeng, Mengmeng Wang, Yusu Pan, Liang Liu, Yong Liu, Yu Ding, and Changjie Fan
   Computer Vision and Pattern Recognition (CVPR). 2020.
3. Learning by Analogy: Reliable Supervision From Transformations for Unsupervised Optical Flow Estimation [Paper] [Code]
   Liang Liu, Jiangning Zhang, Ruifei He, Yong Liu, Yabiao Wang, Ying Tai, Donghao Luo, Chengjie Wang, Jilin Li, and Feiyue Huang
   Computer Vision and Pattern Recognition (CVPR). 2020.
4. APB2FACE: Audio-Guided Face Reenactment with Auxiliary Pose and Blink Signals [Paper] [Code]
   Jiangning Zhang, Liang Liu, Zhucun Xue, and Yong Liu
   International Conference on Acoustics, Speech, and Signal Processing (ICASSP). 2020.
5. Realistic Face Reenactment via Self-Supervised Disentangling of Identity and Pose [Paper]
   Xianfang Zeng, Yusu Pan, Mengmeng Wang, Jiangning Zhang, and Yong Liu
   Association for the Advancement of Artificial Intelligence (AAAI). 2020.
6. Learning-Based Hand Motion Capture and Understanding in Assembly Process [Paper]
   Liang Liu, Yong Liu, and Jiangning Zhang
   Transactions on Industrial Electronics (TIE). 2018.

arXiv

1. PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization [Paper]
   Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, Rongrong Ji
   arXiv. 2023, 2312.06354.
2. Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection [Paper] [Code] [Project]
   Jiangning Zhang, Xuhai Chen, Yabiao Wang, Chengjie Wang, Yong Liu, Xiangtai Li, Ming-Hsuan Yang, Dacheng Tao
   arXiv. 2023, 2312.07495.
3. Exploring Grounding Potential of VQA-oriented GPT-4V for Zero-shot Anomaly Detection [Paper] [Code]
   Jiangning Zhang, Xuhai Chen, Zhucun Xue, Yabiao Wang, Chengjie Wang, and Yong Liu
   arXiv. 2023, 2311.02612.
4. CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection [Paper]
   Xuhai Chen*, Jiangning Zhang*, Guanzhong Tian, Haoyang He, Wuhao Zhang, Yabiao Wang, Chengjie Wang, Yunsheng Wu, and Yong Liu
   arXiv. 2023, 2311.00453.
5. Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator [Paper]
   Chao Xu, Shaoting Zhu, Junwei Zhu, Tianxin Huang, Jiangning Zhang, Ying Tai, and Yong Liu
   arXiv. 2023, 2305.02594.
6. Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentation [Paper] [Code]
   Yue Han*, Jiangning Zhang*, Zhucun Xue, Chao Xu, Xintian Shen, Yabiao Wang, Chengjie Wang, Yong Liu, and Xiangtai Li
   arXiv. 2022, 2301.01156.
7. SelFSR: Self-Conditioned Face Super-Resolution in the Wild via Flow Field Degradation Network [Paper]
   Xianfang Zeng*, Jiangning Zhang*, Liang Liu, Guangzhong Tian, and Yong Liu
   arXiv. 2021, 2112.10683.

Challenges

1. 1st place for [Zero-shot Track] and 4th place for [Few-shot Track] in [Visual Anomaly and Novelty Detection (VAND) 2023 Challenge] by CVPR 2023
   A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD [Paper] [Code]
   Xuhai Chen*, Yue Han*, Jiangning Zhang*
   Computer Vision and Pattern Recognition (CVPR) VAND. 2023.

📖 Educations

• 2020.04 - 2022.10, M.D. 👉️ Ph.D. in College of Control Science and Engineering, Zhejiang University, Hangzhou, China.
• 2017.09 - 2020.04, Pursuing M.D. in College of Control Science and Engineering, Zhejiang University, Hangzhou, China.
• 2013.09 - 2017.06, Obtained B.S. in Electronic Information School, Wuhan University, Wuhan, China.

💻 Internships

• 2020.12 - 2022.10, YouTu Lab, Tencent, Shanghai, mentored by Researcher Yabiao Wang and Dr. Ying Tai.
• 2020.06 - 2020.10, 2012 Lab, Huawei, Hangzhou.
• 2019.07 - 2019.09, Guangzi Technique Center, Tencent, Shenzhen.
• 2019.04 - 2019.06, Fuxi AI Lab, NetEase, Hangzhou.