I am currently a Staff Research Scientist and Manager at Google Cloud AI Research, working on machine learning and its real-world applications across various tasks and modalities.
Previously I was at Apple where I published the first research paper from the Technology Development Group at CVPR and launched several key features in ARKit. I was also in the AI Research Group at Magic Leap with Andrew Rabinovich. I did my PhD in deep learning, advised by Professors Zhuowen Tu and Pamela Cosman at UC San Diego, and advised by Simon Osindero during the summer.
We are building a world-class team to explore the intersection of large AI models and high-value enterprise AI challenges. Contact me for more details.
Publications
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, Tomas PfisterACL 2023 / Paper (Findings) / Google AI Blog
QueryForm: A Simple Zero-shot Form Entity Query Framework
Zifeng Wang, Zizhao Zhang, Jacob Devlin, Chen-Yu Lee, Guolong Su, Hao Zhang, Jennifer Dy, Vincent Perot, Tomas PfisterACL 2023 / Paper (Findings)
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Hao Zhang, Timothy Dozat, Vincent Perot, Guolong Su, Xiang Zhang, Kihyuk Sohn, Nikolai Glushnev, Renshen Wang, Joshua Ainslie, Shangbang Long, Siyang Qin, Yasuhisa Fujii, Nan Hua, Tomas PfisterACL 2023 / Paper
Multimodal Prompting with Missing Modalities for Visual Recognition
Yi-Lun Lee, Yi-Hsuan Tsai, Wei-Chen Chiu, Chen-Yu LeeCVPR 2023 / Paper / Code
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval
Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, Tomas PfisterCVPR 2023 / Paper / Code / Google AI Blog
Prefix Conditioning Unifies Language and Label Supervision
Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, Tomas PfisterCVPR 2023 / Paper / Google AI Blog
Neural Spline Search for Quantile Probabilistic Modeling
Ruoxi Sun, Chun-Liang Li, Sercan O. Arik, Michael W. Dusenberry, Chen-Yu Lee, Tomas PfisterAAAI 2023 / Paper
Unifying Distribution Alignment as a Loss for Imbalanced Semi-Supervised Learning
Justin Lazarow, Kihyuk Sohn, Chen-Yu Lee, Chun-Liang Li, Zizhao Zhang, Tomas PfisterWACV 2023 / Paper
Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types
Kihyuk Sohn, Jinsung Yoon, Chun-Liang Li, Chen-Yu Lee, Tomas PfisterWACV 2023 / Paper
Self-supervise, Refine, Repeat: Improving Unsupervised Anomaly Detection
Jinsung Yoon, Kihyuk Sohn, Chun-Liang Li, Sercan O Arik, Chen-Yu Lee, Tomas PfisterTMLR 2022 / Paper
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas PfisterECCV 2022 / Paper / Code
FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Timothy Dozat, Vincent Perot, Guolong Su, Nan Hua, Joshua Ainslie, Renshen Wang, Yasuhisa Fujii, Tomas PfisterACL 2022 / Paper / Google AI Blog
Learning to Prompt for Continual Learning
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas PfisterCVPR 2022 / Paper / Code / Google AI Blog
Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts
Kunpeng Li, Zizhao Zhang, Guanhang Wu, Xuehan Xiong, Chen-Yu Lee, Zhichao Lu, Yun Fu, Tomas PfisterAAAI 2022 / Paper / Google AI Blog
ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Chu Wang, Renshen Wang, Yasuhisa Fujii, Siyang Qin, Ashok Popat, Tomas PfisterACL 2021 / Paper (Oral Presentation)
Learning to Branch for Multi-Task Learning
Pengsheng Guo, Chen-Yu Lee, Daniel UlbrichtICML 2020 / Paper
Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation
Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, Daniel UlbrichtCVPR 2019 / Paper / Code / ML Journal
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, Andrew RabinovichICML 2018 / Paper
RoomNet: End-to-End Room Layout Estimation
Chen-Yu Lee, Vijay Badrinarayanan, Tomasz Malisiewicz, Andrew RabinovichICCV 2017 / Paper
Generalizing Pooling Functions in CNNs: Mixed, Gated, and Tree
Chen-Yu Lee, Patrick Gallagher, Zhuowen TuTPAMI 2017 / Paper
Recursive Recurrent Nets with Attention Modeling for OCR in the Wild
Chen-Yu Lee and Simon OsinderoCVPR 2016 / Paper
Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree
Chen-Yu Lee, Patrick Gallagher, Zhuowen TuAISTATS 2016 / Paper / Code
Deeply-Supervised Nets
Chen-Yu Lee*, Saining Xie*, Patrick Gallagher, Zhengyou Zhang, Zhuowen Tu (*equal contribution)AISTATS 2015 / Paper / Code
NIPS 2014 Workshop / Paper (Oral Presentation)
Region-based Discriminative Feature Pooling for Scene Text Recognition
Chen-Yu Lee, Anurag Bhardwaj, Wei Di, Vignesh Jagadeesh, Robinson PiramuthuCVPR 2014 / Paper
Tracking Epithelial Cell Junctions in C. elegans Embryogenesis with Active Contours Guided by SIFT Flow
Sukryool Kang, Chen-Yu Lee, Monira Goncalves, Andrew Chisholm, Pamela CosmanTBME 2014 / Paper
Automated Cell Junction Tracking with Modified Active Contours Guided by SIFT Flow
Chen-Yu Lee, Sukryool Kang, Andrew Chisholm, Pamela CosmanISBI 2014 / Paper / Demo
Smoke Detection Using Spatial and Temporal Analyses
Chen-Yu Lee, Chin-Teng Lin, Chao-Ting Hong, Miin-Tsair SuIJICIC 2012 / Paper
Spatio‐temporal Analysis in Smoke Detection
Chen-Yu Lee, Chin‐Teng Lin, Chao‐Ting HongICSIPA 2009 / Paper / Project Page
Pre-Prints & Tech Reports
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models
Cheng-Yu Hsieh, Si-An Chen, Chun-Liang Li, Yasuhisa Fujii, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, Tomas PfisterarXiv 2023 / Paper
A Simple Semi-Supervised Learning Framework for Object Detection
Kihyuk Sohn*, Zizhao Zhang*, Chun-Liang Li, Han Zhang, Chen-Yu Lee, Tomas Pfister (*equal contribution)arXiv 2020 / Paper / Code
Training Deeper Convolutional Networks with Deep Supervision
Liwei Wang, Chen-Yu Lee, Zhuowen Tu, Svetlana LazebnikarXiv 2015 / Paper / Code / Leaderboard of MIT Place205
Patents
Meta-Learning for Multi-Task Learning for Neural Networks
Andrew Rabinovich, Vijay Badrinarayanan, Srivignesh Rajendran, Chen-Yu LeeUS Patent 11048978 (2021/6/29)
Room Layout Estimation Methods and Techniques
Chen-Yu Lee, Vijay Badrinarayanan, Tomasz Malisiewicz, Andrew RabinovichUS Patent 10657376 (2020/5/19)
System and Method for Scene Text Recognition
Anurag Bhardwaj, Chen-Yu Lee, Robinson Piramuthu, Vignesh Jagadeesh, Wei DiUS Patent 9245191 (2016/1/26)
Academic Services
Reviewer: CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, BMVC, PAMI, IJCV, JMLR, TIP, TNNLS
Outstanding Reviewer: CVPR 2019, BMVC 2019
Program Committee Member: Computer Vision for AR/VR Workshop at CVPR 2020