I am currently a Staff Research Scientist and Manager at Google Cloud AI Research, working on machine learning and its real-world applications to computer vision and natural language processing.
Previously I was at Apple where I published the first research paper from the Technology Development Group at CVPR and launched several key features in ARKit. I was also in the AI Research Group at Magic Leap with Andrew Rabinovich. I did my PhD in deep learning, advised by Professors Zhuowen Tu and Pamela Cosman at UC San Diego, and advised by Simon Osindero during the summer.
We are building a world-class team to explore the intersection of vision and language research. Contact me for more details.
Publications
FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Timothy Dozat, Vincent Perot, Guolong Su, Nan Hua, Joshua Ainslie, Renshen Wang, Yasuhisa Fujii, and Tomas PfisterACL 2022 / Paper / Google AI Blog
Learning to Prompt for Continual Learning
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas PfisterCVPR 2022 / Paper / Google AI Blog
Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts
Kunpeng Li, Zizhao Zhang, Guanhang Wu, Xuehan Xiong, Chen-Yu Lee, Zhichao Lu, Yun Fu, and Tomas PfisterAAAI 2022 / Paper / Google AI Blog
ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Chu Wang, Renshen Wang, Yasuhisa Fujii, Siyang Qin, Ashok Popat, and Tomas PfisterACL 2021 / Paper (Oral Presentation)
Learning to Branch for Multi-Task Learning
Pengsheng Guo, Chen-Yu Lee, and Daniel UlbrichtICML 2020 / Paper
Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation
Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, and Daniel UlbrichtCVPR 2019 / Paper / Code / ML Journal
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew RabinovichICML 2018 / Paper
RoomNet: End-to-End Room Layout Estimation
Chen-Yu Lee, Vijay Badrinarayanan, Tomasz Malisiewicz, and Andrew RabinovichICCV 2017 / Paper
Generalizing Pooling Functions in CNNs: Mixed, Gated, and Tree
Chen-Yu Lee, Patrick Gallagher, and Zhuowen TuTPAMI 2017 / Paper
Recursive Recurrent Nets with Attention Modeling for OCR in the Wild
Chen-Yu Lee and Simon OsinderoCVPR 2016 / Paper
Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree
Chen-Yu Lee, Patrick Gallagher, and Zhuowen TuAISTATS 2016 / Paper / Code
Deeply-Supervised Nets
Chen-Yu Lee*, Saining Xie*, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu (*equal contribution)AISTATS 2015 / Paper / Code
NIPS 2014 Workshop / Paper (Oral Presentation)
Region-based Discriminative Feature Pooling for Scene Text Recognition
Chen-Yu Lee, Anurag Bhardwaj, Wei Di, Vignesh Jagadeesh, and Robinson PiramuthuCVPR 2014 / Paper
Tracking Epithelial Cell Junctions in C. elegans Embryogenesis with Active Contours Guided by SIFT Flow
Sukryool Kang, Chen-Yu Lee, Monira Goncalves, Andrew Chisholm, and Pamela CosmanTBME 2014 / Paper
Automated Cell Junction Tracking with Modified Active Contours Guided by SIFT Flow
Chen-Yu Lee, Sukryool Kang, Andrew Chisholm, and Pamela CosmanISBI 2014 / Paper / Demo
Smoke Detection Using Spatial and Temporal Analyses
Chen-Yu Lee, Chin-Teng Lin, Chao-Ting Hong, and Miin-Tsair SuIJICIC 2012 / Paper
Spatio‐temporal Analysis in Smoke Detection
Chen-Yu Lee, Chin‐Teng Lin, and Chao‐Ting HongICSIPA 2009 / Paper / Project Page
Tech Report
Training Deeper Convolutional Networks with Deep Supervision
Liwei Wang, Chen-Yu Lee, Zhuowen Tu, and Svetlana LazebnikarXiv 2015 / Paper / Code / Leaderboard of MIT Place205
Patents
Meta-Learning for Multi-Task Learning for Neural Networks
Andrew Rabinovich, Vijay Badrinarayanan, Srivignesh Rajendran, and Chen-Yu LeeUS Patent 11048978 (2021/6/29)
Room Layout Estimation Methods and Techniques
Chen-Yu Lee, Vijay Badrinarayanan, Tomasz Malisiewicz, and Andrew RabinovichUS Patent 10657376 (2020/5/19)
System and Method for Scene Text Recognition
Anurag Bhardwaj, Chen-Yu Lee, Robinson Piramuthu, Vignesh Jagadeesh, and Wei DiUS Patent 9245191 (2016/1/26)