Wanrong Zhu

CS Ph.D. Candidate

University of California, Santa Barbara

Hi, I am a Ph.D. candidate in the Natural Language Processing Group at UCSB, advised by William Wang. Before joining UCSB, I received my B.S. degree in Computer Science from Peking University.

wanrongzhu [at] cs.ucsb.edu
Henley Hall, UCSB

Education

University of California, Santa Barbara
Ph.D. in Computer Science
Sep. 2019 - Present
Peking University
B.S. in Computer Science
Sep. 2015 - July 2019

Interests

Natural Language Processing
Language-and-Vision
Text Generation

Experience

Mosaic Team, AI2
Research Intern. Hosts: Jack Hessel and Youngjae Yu
June 2022 - Sep. 2022
Google Research
Research Intern. Hosts: Bo Pang and Ashish Thapliyal
June 2021 - Oct. 2021
AdsAI Team, Google
Research Intern. Host: Pradyumna Narayana
June 2020 - Oct. 2020
Language Technology Institution, Carnegie Mellon University
Research Assistant. Advisor: Zhiting Hu
July 2018 - Sep. 2018

Publications & Preprints

LayoutGPT: Compositional Visual Planning and Generation with Large Language Models

Preprint (arXiv 2305.15393)

Weixi Feng\*, Wanrong Zhu\*, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Xuehai He, Sugato Basu, Xin Eric Wang, William Yang Wang

Paper Code

LayoutGPT: Compositional Visual Planning and Generation with Large Language Models

Multimodal Procedural Planning via Dual Text-Image Prompting

Preprint (arXiv 2305.01795)

Yujie Lu, Pan Lu, Zhiyu Chen, Wanrong Zhu, Xin Eric Wang, William Yang Wang

Paper Code

Multimodal Procedural Planning via Dual Text-Image Prompting

Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved With Text

Preprint (arXiv 2304.06939)

Wanrong Zhu\*, Jack Hessel\*, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, Yejin Choi

Paper Github

Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved With Text

OpenFlamingo: An Open-Source Framework for Training Vision-Language Models with In-Context Learning

Stay-tuned for the technical report!

Anas Awadalla, Irena Gao, Joshua Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, Ludwig Schmidt

Blog Post Code Demo

OpenFlamingo: An Open-Source Framework for Training Vision-Language Models with In-Context Learning

Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation

Preprint (arXiv 2305.11317)

Wanrong Zhu, Xinyi Wang, Yujie Lu, Tsu-Jui Fu, Xin Eric Wang, Miguel Eckstein, William Yang Wang

Paper

Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation

Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning

Preprint (arXiv 2301.11916)

Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers, William Yang Wang

Paper Code

Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning

Visualize Before You Write: Imagination-Guided Open-Ended Text Generation

The 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023, Findings)

Wanrong Zhu, An Yan, Yujie Lu, Wenda Xu, Xin Eric Wang, Miguel Eckstein, William Yang Wang

Paper Code

Visualize Before You Write: Imagination-Guided Open-Ended Text Generation

CLIP also Understands Text: Prompting CLIP for Phrase Understanding

Preprint (arXiv 2210.05836)

An Yan, Jiacheng Li, Wanrong Zhu, Yujie Lu, William Yang Wang, Julian McAuley

Paper

CLIP also Understands Text: Prompting CLIP for Phrase Understanding

Neuro-Symbolic Causal Language Planning with Commonsense Prompting

The 11th International Conference on Learning Representations (ICLR 2023, Spotlight)

Yujie Lu, Weixi Feng, Wanrong Zhu, Wenda Xu, Xin Eric Wang, Miguel Eckstein, William Yang Wang

Paper Project Code

Neuro-Symbolic Causal Language Planning with Commonsense Prompting

End-to-end Dense Video Captioning as Sequence Generation

The 29th International Conference on Computational Linguistics (COLING 2022)

Wanrong Zhu, Bo Pang, Ashish V. Thapliyal, William Yang Wang, Radu Soricut

Paper Video Poster

End-to-end Dense Video Captioning as Sequence Generation

Imagination-Augmented Natural Language Understanding

The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022, Oral)

Yujie Lu, Wanrong Zhu, Xin Eric Wang, Miguel Eckstein, William Yang Wang

Paper Code

Imagination-Augmented Natural Language Understanding

ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation

The 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023, Findings)

Wanrong Zhu, Xin Eric Wang, An Yan, Miguel Eckstein, William Yang Wang

Paper

ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation

Diagnosing Vision-and-Language Navigation: What Really Matters

The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022, Oral)

Wanrong Zhu, Yuankai Qi, Pradyumna Narayana, Kazoo Sone, Sugato Basu, Xin Eric Wang, Qi Wu, Miguel Eckstein, William Yang Wang

Paper Code Video Slides

Diagnosing Vision-and-Language Navigation: What Really Matters

Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation

The 16th conference of the European Chapter of the Association for Computational Linguistics (EACL 2021)

Wanrong Zhu, Xin Eric Wang, Tsu-Jui Fu, An Yan, Pradyumna Narayana, Kazoo Sone, Sugato Basu, William Yang Wang

Paper Code Video & Slides Poster

Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation

Towards Understanding Sample Variance in Visually Grounded Language Generation: Evaluations and Observations

The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020, Short)

Wanrong Zhu, Xin Eric Wang, Pradyumna Narayana, Kazoo Sone, Sugato Basu, William Yang Wang

Paper Video & Slides

Towards Understanding Sample Variance in Visually Grounded Language Generation: Evaluations and Observations

Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation

The 57th Annual Meeting of the Association for Computational Linguistics:System Demonstrations (ACL 2019 System Demonstration)

Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Singh Sachan, Eric P. Xing

Paper Code Docs

Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation

Text Infilling

Preprint (arXiv 1901.00158)

Wanrong Zhu, Zhiting Hu, Eric P. Xing

Paper Code