I am currently a Senior Staff Research Scientist and Manager at Google Cloud AI Research, driving innovation in machine learning and its real-world applications across diverse tasks and modalities.
Previously, I was at Apple where I published the Technology Development Group's inaugural research paper at CVPR and launched several key features in ARKit (now Vision Pro). I was also in the AI Research Group at Magic Leap with Andrew Rabinovich. I completed my PhD in deep learning, advised by Professors Zhuowen Tu and Pamela Cosman at UC San Diego, and mentored by Simon Osindero during the summer.
We are assembling a world-class team to explore the intersection of large AI models and high-value enterprise AI challenges. Contact me to learn more.
Publications
TableRAG: Million-Token Tabular Reasoning with Large Language Models
Si-An Chen, Lesly Miculicich, Julian Martin Eisenschlos, Zifeng Wang, Zilong Wang, Yanfei Chen, Yasuhisa Fujii, Hsuan-Tien Lin, Chen-Yu Lee, Tomas PfisterNeurIPS 2024 / Paper / Code
Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval
Yanfei Chen, Jinsung Yoon, Devendra Singh Sachan, Qingze Wang, Vincent Cohen-Addad, Mohammadhossein Bateni, Chen-Yu Lee, Tomas PfisterEMNLP 2024 / Paper (Findings)
CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation
I-Hung Hsu, Zifeng Wang, Long T. Le, Lesly Miculicich, Nanyun Peng, Chen-Yu Lee, Tomas PfisterACL 2024 / Paper (Findings)
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long Le, Abhishek Kumar, James R. Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, Tomas PfisterACL 2024 / Paper (Findings)
LMDX: Language Model-based Document Information Extraction and Localization
Vincent Perot, Kai Kang, Florian Luisier, Guolong Su, Xiaoyu Sun, Ramya Sree Boppana, Zilong Wang, Zifeng Wang, Jiaqi Mu, Hao Zhang, Chen-Yu Lee, Nan HuaACL 2024 / Paper (Findings)
CodecLM: Aligning Language Models with Tailored Synthetic Data
Zifeng Wang, Chun-Liang Li, Vincent Perot, Long T. Le, Jin Miao, Zizhao Zhang, Chen-Yu Lee, Tomas PfisterNAACL 2024 / Paper (Findings) / Google AI Blog
Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding
Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, Tomas PfisterICLR 2024 / Paper / Code / Google AI Blog
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, Tomas PfisterACL 2023 / Paper (Findings) / Code / Google AI Blog (Google ACL 2023 Spotlight)
QueryForm: A Simple Zero-shot Form Entity Query Framework
Zifeng Wang, Zizhao Zhang, Jacob Devlin, Chen-Yu Lee, Guolong Su, Hao Zhang, Jennifer Dy, Vincent Perot, Tomas PfisterACL 2023 / Paper (Findings)
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Hao Zhang, Timothy Dozat, Vincent Perot, Guolong Su, Xiang Zhang, Kihyuk Sohn, Nikolai Glushnev, Renshen Wang, Joshua Ainslie, Shangbang Long, Siyang Qin, Yasuhisa Fujii, Nan Hua, Tomas PfisterACL 2023 / Paper
VRDU: A Benchmark for Visually-rich Document Understanding
Zilong Wang, Yichao Zhou, Wei Wei, Chen-Yu Lee, Sandeep TataKDD 2023 / Paper / Code / Dataset
Multimodal Prompting with Missing Modalities for Visual Recognition
Yi-Lun Lee, Yi-Hsuan Tsai, Wei-Chen Chiu, Chen-Yu LeeCVPR 2023 / Paper / Code
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval
Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, Tomas PfisterCVPR 2023 / Paper / Code / Google AI Blog
Prefix Conditioning Unifies Language and Label Supervision
Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, Tomas PfisterCVPR 2023 / Paper / Google AI Blog
Neural Spline Search for Quantile Probabilistic Modeling
Ruoxi Sun, Chun-Liang Li, Sercan O. Arik, Michael W. Dusenberry, Chen-Yu Lee, Tomas PfisterAAAI 2023 / Paper
Unifying Distribution Alignment as a Loss for Imbalanced Semi-Supervised Learning
Justin Lazarow, Kihyuk Sohn, Chen-Yu Lee, Chun-Liang Li, Zizhao Zhang, Tomas PfisterWACV 2023 / Paper
Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types
Kihyuk Sohn, Jinsung Yoon, Chun-Liang Li, Chen-Yu Lee, Tomas PfisterWACV 2023 / Paper
Self-supervise, Refine, Repeat: Improving Unsupervised Anomaly Detection
Jinsung Yoon, Kihyuk Sohn, Chun-Liang Li, Sercan O Arik, Chen-Yu Lee, Tomas PfisterTMLR 2022 / Paper
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas PfisterECCV 2022 / Paper / Code
FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Timothy Dozat, Vincent Perot, Guolong Su, Nan Hua, Joshua Ainslie, Renshen Wang, Yasuhisa Fujii, Tomas PfisterACL 2022 / Paper / Google AI Blog
Learning to Prompt for Continual Learning
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas PfisterCVPR 2022 / Paper / Code / Google AI Blog
Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts
Kunpeng Li, Zizhao Zhang, Guanhang Wu, Xuehan Xiong, Chen-Yu Lee, Zhichao Lu, Yun Fu, Tomas PfisterAAAI 2022 / Paper / Google AI Blog
ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Chu Wang, Renshen Wang, Yasuhisa Fujii, Siyang Qin, Ashok Popat, Tomas PfisterACL 2021 / Paper (Oral Presentation)
Learning to Branch for Multi-Task Learning
Pengsheng Guo, Chen-Yu Lee, Daniel UlbrichtICML 2020 / Paper
Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation
Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, Daniel UlbrichtCVPR 2019 / Paper / Code / ML Journal
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, Andrew RabinovichICML 2018 / Paper
RoomNet: End-to-End Room Layout Estimation
Chen-Yu Lee, Vijay Badrinarayanan, Tomasz Malisiewicz, Andrew RabinovichICCV 2017 / Paper
Generalizing Pooling Functions in CNNs: Mixed, Gated, and Tree
Chen-Yu Lee, Patrick Gallagher, Zhuowen TuTPAMI 2017 / Paper
Recursive Recurrent Nets with Attention Modeling for OCR in the Wild
Chen-Yu Lee and Simon OsinderoCVPR 2016 / Paper
Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree
Chen-Yu Lee, Patrick Gallagher, Zhuowen TuAISTATS 2016 / Paper / Code
Deeply-Supervised Nets
Chen-Yu Lee*, Saining Xie*, Patrick Gallagher, Zhengyou Zhang, Zhuowen Tu (*equal contribution)AISTATS 2015 / Paper / Code
NIPS 2014 Workshop / Paper (Oral Presentation)
Region-based Discriminative Feature Pooling for Scene Text Recognition
Chen-Yu Lee, Anurag Bhardwaj, Wei Di, Vignesh Jagadeesh, Robinson PiramuthuCVPR 2014 / Paper
Tracking Epithelial Cell Junctions in C. elegans Embryogenesis with Active Contours Guided by SIFT Flow
Sukryool Kang, Chen-Yu Lee, Monira Goncalves, Andrew Chisholm, Pamela CosmanTBME 2014 / Paper
Automated Cell Junction Tracking with Modified Active Contours Guided by SIFT Flow
Chen-Yu Lee, Sukryool Kang, Andrew Chisholm, Pamela CosmanISBI 2014 / Paper / Demo
Smoke Detection Using Spatial and Temporal Analyses
Chen-Yu Lee, Chin-Teng Lin, Chao-Ting Hong, Miin-Tsair SuIJICIC 2012 / Paper
Spatio‐temporal Analysis in Smoke Detection
Chen-Yu Lee, Chin‐Teng Lin, Chao‐Ting HongICSIPA 2009 / Paper / Project Page
Pre-Prints & Tech Reports
Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling
Wenda Xu, Rujun Han, Zifeng Wang, Long T. Le, Dhruv Madeka, Lei Li, William Yang Wang, Rishabh Agarwal, Chen-Yu Lee, Tomas PfisterarXiv 2024 / Paper
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence
Shangbin Feng, Zifeng Wang, Yike Wang, Sayna Ebrahimi, Hamid Palangi, Lesly Miculicich, Achin Kulshrestha, Nathalie Rauschmayr, Yejin Choi, Yulia Tsvetkov, Chen-Yu Lee, Tomas PfisterarXiv 2024 / Paper
Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting
Zilong Wang, Zifeng Wang, Long Le, Huaixiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, Chen-Yu Lee, Tomas PfisterarXiv 2024 / Paper / Google AI Blog
Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction
Kuniaki Saito, Kihyuk Sohn, Chen-Yu Lee, Yoshitaka UshikuarXiv 2024 / Paper
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models
Cheng-Yu Hsieh, Si-An Chen, Chun-Liang Li, Yasuhisa Fujii, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, Tomas PfisterarXiv 2023 / Paper
A Simple Semi-Supervised Learning Framework for Object Detection
Kihyuk Sohn*, Zizhao Zhang*, Chun-Liang Li, Han Zhang, Chen-Yu Lee, Tomas Pfister (*equal contribution)arXiv 2020 / Paper / Code
Training Deeper Convolutional Networks with Deep Supervision
Liwei Wang, Chen-Yu Lee, Zhuowen Tu, Svetlana LazebnikarXiv 2015 / Paper / Code / Leaderboard of MIT Place205
Academic Services
Reviewer: NeurIPS, ICML, ICLR, ACL, NAACL, EMNLP, COLM, CVPR, ICCV, ECCV, BMVC, PAMI, IJCV, JMLR
Outstanding Reviewer: CVPR 2019, BMVC 2019
Program Committee Member: Computer Vision for AR/VR Workshop at CVPR 2020