Chen-Yu Lee

Chen-Yu Lee










I am currently a Staff Research Scientist and Manager at Google Cloud AI Research, working on machine learning and its real-world applications to computer vision and natural language processing.

Previously I was at Apple where I published the first research paper from the Technology Development Group at CVPR and launched several key features in ARKit. I was also in the AI Research Group at Magic Leap with Andrew Rabinovich. I did my PhD in deep learning, advised by Professors Zhuowen Tu and Pamela Cosman at UC San Diego, and advised by Simon Osindero during the summer.

We are building a world-class team to explore the intersection of vision and language research. Contact me for more details.

Publications

Unifying Distribution Alignment as a Loss for Imbalanced Semi-Supervised Learning

Justin Lazarow, Kihyuk Sohn, Chen-Yu Lee, Chun-Liang Li, Zizhao Zhang, Tomas Pfister
WACV 2023 / Paper

Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types

Kihyuk Sohn, Jinsung Yoon, Chun-Liang Li, Chen-Yu Lee, Tomas Pfister
WACV 2023 / Paper

Self-supervise, Refine, Repeat: Improving Unsupervised Anomaly Detection

Jinsung Yoon, Kihyuk Sohn, Chun-Liang Li, Sercan O Arik, Chen-Yu Lee, Tomas Pfister
TMLR 2022 / Paper

DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning

Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas Pfister
ECCV 2022 / Paper / Code

FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction

Chen-Yu Lee, Chun-Liang Li, Timothy Dozat, Vincent Perot, Guolong Su, Nan Hua, Joshua Ainslie, Renshen Wang, Yasuhisa Fujii, Tomas Pfister
ACL 2022 / Paper / Google AI Blog

Learning to Prompt for Continual Learning

Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas Pfister
CVPR 2022 / Paper / Code / Google AI Blog

Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts

Kunpeng Li, Zizhao Zhang, Guanhang Wu, Xuehan Xiong, Chen-Yu Lee, Zhichao Lu, Yun Fu, Tomas Pfister
AAAI 2022 / Paper / Google AI Blog

ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction

Chen-Yu Lee, Chun-Liang Li, Chu Wang, Renshen Wang, Yasuhisa Fujii, Siyang Qin, Ashok Popat, Tomas Pfister
ACL 2021 / Paper (Oral Presentation)

Learning to Branch for Multi-Task Learning

Pengsheng Guo, Chen-Yu Lee, Daniel Ulbricht
ICML 2020 / Paper

Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation

Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, Daniel Ulbricht
CVPR 2019 / Paper / Code /  ML Journal

GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks

Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, Andrew Rabinovich
ICML 2018 / Paper

RoomNet: End-to-End Room Layout Estimation

Chen-Yu Lee, Vijay Badrinarayanan, Tomasz Malisiewicz, Andrew Rabinovich
ICCV 2017 / Paper

Generalizing Pooling Functions in CNNs: Mixed, Gated, and Tree

Chen-Yu Lee, Patrick Gallagher, Zhuowen Tu
TPAMI 2017 / Paper

Recursive Recurrent Nets with Attention Modeling for OCR in the Wild

Chen-Yu Lee and Simon Osindero
CVPR 2016 / Paper

Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree

Chen-Yu Lee, Patrick Gallagher, Zhuowen Tu
AISTATS 2016 / Paper / Code

Deeply-Supervised Nets

Chen-Yu Lee*, Saining Xie*, Patrick Gallagher, Zhengyou Zhang, Zhuowen Tu (*equal contribution)
AISTATS 2015 / Paper / Code
NIPS 2014 Workshop / Paper (Oral Presentation)

Region-based Discriminative Feature Pooling for Scene Text Recognition

Chen-Yu Lee, Anurag Bhardwaj, Wei Di, Vignesh Jagadeesh, Robinson Piramuthu
CVPR 2014 / Paper

Tracking Epithelial Cell Junctions in C. elegans Embryogenesis with Active Contours Guided by SIFT Flow

Sukryool Kang, Chen-Yu Lee, Monira Goncalves, Andrew Chisholm, Pamela Cosman
TBME 2014 / Paper

Automated Cell Junction Tracking with Modified Active Contours Guided by SIFT Flow

Chen-Yu Lee, Sukryool Kang, Andrew Chisholm, Pamela Cosman
ISBI 2014 / Paper / Demo

Smoke Detection Using Spatial and Temporal Analyses

Chen-Yu Lee, Chin-Teng Lin, Chao-Ting Hong, Miin-Tsair Su
IJICIC 2012 / Paper

Spatio‐temporal Analysis in Smoke Detection

Chen-Yu Lee, Chin‐Teng Lin, Chao‐Ting Hong
ICSIPA 2009 / Paper / Project Page

Tech Report

A Simple Semi-Supervised Learning Framework for Object Detection

Kihyuk Sohn*, Zizhao Zhang*, Chun-Liang Li, Han Zhang, Chen-Yu Lee, Tomas Pfister (*equal contribution)
arXiv 2020 / Paper / Code

Training Deeper Convolutional Networks with Deep Supervision

Liwei Wang, Chen-Yu Lee, Zhuowen Tu, Svetlana Lazebnik
arXiv 2015 / Paper / Code / Leaderboard of MIT Place205

Patents

Meta-Learning for Multi-Task Learning for Neural Networks

Andrew Rabinovich, Vijay Badrinarayanan, Srivignesh Rajendran, Chen-Yu Lee
US Patent 11048978 (2021/6/29)

Room Layout Estimation Methods and Techniques

Chen-Yu Lee, Vijay Badrinarayanan, Tomasz Malisiewicz, Andrew Rabinovich
US Patent 10657376 (2020/5/19)

System and Method for Scene Text Recognition

Anurag Bhardwaj, Chen-Yu Lee, Robinson Piramuthu, Vignesh Jagadeesh, Wei Di
US Patent 9245191 (2016/1/26)

Academic Services

Reviewer

CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, BMVC, PAMI, IJCV, JMLR, TIP, TNNLS

Outstanding Reviewer

CVPR 2019, BMVC 2019

Program Committee Member

Computer Vision for AR/VR Workshop at CVPR 2020