Canva CORE CN
Staff Research Scientist, working on multi-layer image generation, design editing, etc.
About
I am a Staff Research Scientist at Canva, where I work on multi-layer image generation and design editing. Previously, I was a research manager at Du Xiaoman Financial, leading multimodal content generation initiatives spanning text-to-image, image-to-video, and text-to-speech, and before that a senior researcher at JD AI Research. I received my Ph.D. through the joint program of Microsoft Research Asia and Harbin Institute of Technology. My research interests include multimodal learning, image and video generation, and fine-grained visual recognition.
Experience
-
2025.09 – Present
-
2024.05 – 2025.09
Du Xiaoman Financial Multimedia Research Team for In2X
Research Manager, leading multimodal content generation initiatives, including text-to-image (T2I), image-to-video (I2V), text-to-speech (TTS), any-to-any multimodal LLMs, and more.
-
2018.02 – 2023.06
JD AI Research CV Lab
Senior Researcher, working on SnapShop (visual search), VQA, fine-grained recognition, relationship modeling in images, 3D imaging, etc.
-
2013.06 – 2018.02
Microsoft Research Asia Web Search and Mining Group
Research intern, working on deep learning for image representation and computer vision.
-
2012.01 – 2012.07
Microsoft Research Asia Web Search and Data Mining Group
Research intern, working on document retrieval results re-ranking.
Education
-
2014.08 – 2018.09
Ph.D., Joint Education Program of Microsoft Research Asia and Harbin Institute of Technology
Supervisors: Wei-Ying Ma and Tiejun Zhao.
Thesis: Research and Applications of Image-Text Multimodal Correlation Learning.
-
2012.08 – 2014.07
M.S. in Computer Science and Technology, Harbin Institute of Technology
School of Computer Science and Technology. Supervisor: Sheng Li.
-
2008.08 – 2012.07
B.S. in Computer Science and Technology, Harbin Institute of Technology
School of Computer Science and Technology.
Selected Honors
- Top 3% Paper Award at ICASSP 2023
- First Prize, Science and Technology Progress Award, China Society of Image and Graphics, 2022
- Third Place in Open World Image Classification Challenge at CVPR 2021
- First Place in AliProducts Challenge: Large-scale Product Recognition at CVPR 2020
- Second Place in iMet: Fine-grained Attributes Recognition Challenge at CVPR 2020
- First Place in iMaterialist Challenge on Product Recognition at CVPR 2019
- First Place in Fieldguide Challenge: Moths and Butterflies at CVPR 2019
- Second Place in iFood Challenge at FGVC workshop, CVPR 2019
- First Place in the track without using extra data (second among all teams), MSR Image Recognition Challenge at IEEE ICME 2016
- ACM Multimedia 2015 Student Travel Grant
- First Place in MSR-Bing Image Retrieval Challenge at ACM Multimedia 2014
Publications
A full list is available on Google Scholar.
Preprints
-
V2Flow: Unifying Visual Tokenization and Large Language Model Vocabularies for Autoregressive Image Generation
-
STAR: Scale-wise Text-to-image Generation via Auto-Regressive Representations
-
Deep Equilibrium Multimodal Fusion
-
Products-10K: A Large-scale Product Recognition Dataset
Conference & Journal Papers
2026
-
Masked Region Transformer for Layered Image Generation and Editing at Scale
-
Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation
-
PreferThinker: Reasoning-based Personalized Image Preference Assessment
2025
-
Enhancing Reward Models for High-quality Image Generation: Beyond Text-Image Alignment
-
Interactive Conversational Head Generation
-
Uniform Attention Maps: Boosting Image Fidelity in Reconstruction and Editing
2024
-
StyleInject: Parameter Efficient Tuning of Text-to-Image Diffusion Models
-
Teaching Masked Autoencoder With Strong Augmentations
-
Dynamic Prompt Optimizing for Text-to-Image Generation
-
CAMEL: CAusal Motion Enhancement Tailored for Lifting Text-driven Video Editing
2023
-
Learning and Evaluating Human Preferences for Conversational Head Generation
-
Augmentation Pathways Network for Visual Recognition
-
Visualizing and Understanding Patch Interactions in Vision Transformer
-
Visual-Aware Text-to-Speech
-
Boosting Generic Visual-Linguistic Representation with Dynamic Contexts
2022
-
Responsive Listening Head Generation: A Benchmark Dataset and Baseline
-
Directional Self-supervised Learning for Heavy Image Augmentations
2021
-
Exploiting Relationship for Complex-scene Image Generation
2020
-
Look-into-Object: Self-supervised Structure Modeling for Object Recognition
2019
-
VrR-VG: Refocusing Visually-Relevant Relationships
-
Destruction and Construction Learning for Fine-grained Image Recognition
2018
-
Deep Attention Neural Tensor Network for Visual Question Answering
-
Automatic Data Augmentation from Massive Web Images for Deep Visual Recognition
2017
-
Convolutional Neural Networks for Posed and Spontaneous Expression Recognition
2016
-
Improve Dog Recognition by Mining More Information from Both Click-through Logs and Pre-trained Models
2015
-
Automatic Image Dataset Construction from Click-through Logs Using Deep Neural Network
-
Learning Cross Space Mapping via DNN using Large Scale Click-through Logs
2014
-
RC-NET: A General Framework for Incorporating Knowledge into Word Representations
-
Bag-of-Words Based Deep Neural Network for Image Retrieval
-
DNN Flow: DNN Feature Pyramid Based Image Matching
-
Visualizing and Comparing Convolutional Neural Networks
-
Learning High-level Image Representation for Image Retrieval via Multi-Task DNN using Clickthrough Data
2013
-
Cross-lingual Projections between Languages from Different Families
-
Learning Domain Differences Automatically for Dependency Parsing Adaptation
* denotes interns mentored by me.
Selected Open-source Projects
DCL for Fine-grained Image Recognition
The PyTorch implementation of our research paper “Destruction and Construction Learning for Fine-grained Image Recognition” (CVPR 2019). The first-place solutions for the CVPR 2020 AliProducts Challenge: Large-scale Product Recognition [1], the CVPR 2019 iMaterialist Challenge on Product Recognition [2], and the CVPR 2019 Fieldguide Challenge: Moths and Butterflies [3].
Conversational Head Generator
Official solution for the first Conversational Head Generation Challenge, including vivid talking-head video generation, responsive listening-head video generation and implementations of 11 quantitative evaluation metrics.
Davinci-XLab AIGC Research Projects
Repositories of AIGC research projects, including STAR-T2I (scale-wise text-to-image model), V2Flow (vector-quantized image tokenizer based on LLM vocabularies), etc.
Professional Activities
Workshop & Challenge Organizer
- The second Conversational Head Generation Challenge, held in conjunction with ACM Multimedia 2023 in Ottawa, Canada leaderboard
- The first Conversational Head Generation Challenge, held in conjunction with ACM Multimedia 2022 in Lisbon, Portugal leaderboard
- Products-10K: Large Scale Product Recognition Challenge, held in conjunction with ICPR 2020 in Milan, Italy leaderboard
- Workshop 《视觉与语言协同学习及应用》 at VALSE 2021, Hangzhou, China
Journal Reviewer
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- IEEE Transactions on Image Processing
- IEEE Transactions on Multimedia
- IEEE Transactions on Neural Networks and Learning Systems
- IEEE Transactions on Circuits and Systems for Video Technology
- ACM Transactions on Intelligent Systems and Technology
- ACM Transactions on Multimedia Computing, Communications, and Applications
Conference Reviewer / Program Committee Member
- Reviewer (PC Member): CVPR (2019–); ICCV (2019–); ECCV (2020); NeurIPS (2022); ACM Multimedia (2021–); AAAI (2019–)
- SPC: IJCAI 2021
- Area Chair / Meta-Reviewer: CICAI 2022; ACM Multimedia (2022–), Grand Challenges Track; ICASSP (2023–)
Other
- Executive Area Chairs Committee, VALSE (2020–)