avatar

Wanrong Zhu

Research Scientist
Adobe Research
wzhu [at] adobe.com


About Me

I am a research scientist at Adobe Research.

I got my PhD in Computer Science at UCSB, advised by William Wang. I am honored and humbled to be named a 2023 Rising Stars in Machine Learning by University of Maryland. Before that, I received my B.S. degree in Computer Science from Peking University.

My research interest lies in multimodal study, in particular vision-and-language study and text generation.

Education

University of California, Santa Barbara
Ph.D. in Computer Science
Sep. 2019 - June 2024

Peking University
B.S. in Computer Science
Sep. 2015 - July 2019

Research Experience

Research Intern @ Adobe Research
Hosts: Jennifer Healey and Ruiyi Zhang
June 2023 - Sep. 2023

Research Intern @ AI2 Mosaic
Hosts: Jack Hessel and Youngjae Yu
June 2022 - Sep. 2022

Research Intern @ Google Research
Research Intern. Hosts: Bo Pang and Ashish Thapliyal
June 2021 - Oct. 2021

Research Intern @ Google Ads
Host: Pradyumna Narayana
June 2020 - Oct. 2020

Research Assistant @ Language Technology Institution, Carnegie Mellon University
Advisor: Zhiting Hu
July 2018 - Sep. 2018

Publications

  1. List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs
    An Yan, Zhengyuan Yang, Junda Wu, Wanrong Zhu, Jianwei Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Julian McAuley, Jianfeng Gao, Lijuan Wang.
    The First Conference on Language Modeling (CoLM 2024)
    Paper

  2. VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View
    Raphael Schumann, Wanrong Zhu, Weixi Feng, Tsu-Jui Fu, Stefan Riezler, William Yang Wang.
    The Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI 2024)
    Paper | Project Page | Code | Demo

  3. OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models
    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, Ludwig Schmidt.
    Technical Report
    Paper | Code | Blog Post

  4. Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved With Text
    Wanrong Zhu*, Jack Hessel*, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, Yejin Choi.
    The Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track (NeurIPS D&B 2023)
    Paper | Dataset | Press

  5. VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use
    Yonatan Bitton*, Hritik Bansal*, Jack Hessel*, Rulin Shao, Wanrong Zhu, Anas Awadalla, Josh Gardner, Rohan Taori, Ludwig Schimdt.
    The Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track (NeurIPS D&B 2023)
    Paper | Project Page | Code | Dataset | Leaderboard

  6. LayoutGPT: Compositional Visual Planning and Generation with Large Language Models
    Weixi Feng*, Wanrong Zhu*, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Xuehai He, Sugato Basu, Xin Eric Wang, William Yang Wang.
    The Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)
    Paper | Project Page | Code

  7. Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning
    Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers, William Yang Wang.
    The Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)
    Paper | Code

  8. Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation
    Wanrong Zhu, Xinyi Wang, Yujie Lu, Tsu-Jui Fu, Xin Eric Wang, Miguel Eckstein, William Yang Wang.
    The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023, Short)
    Paper

  9. Visualize Before You Write: Imagination-Guided Open-Ended Text Generation
    Wanrong Zhu, An Yan, Yujie Lu, Wenda Xu, Xin Eric Wang, Miguel Eckstein, William Yang Wang.
    The 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023, Findings)
    Paper | Code

  10. ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation
    Wanrong Zhu, Xin Eric Wang, An Yan, Miguel Eckstein, William Yang Wang.
    The 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023, Findings)
    Paper

  11. Neuro-Symbolic Causal Language Planning with Commonsense Prompting
    Yujie Lu, Weixi Feng, Wanrong Zhu, Wenda Xu, Xin Eric Wang, Miguel Eckstein, William Yang Wang.
    The 11th International Conference on Learning Representations (ICLR 2023, Spotlight)
    Paper | Code

  12. End-to-end Dense Video Captioning as Sequence Generation
    Wanrong Zhu, Bo Pang, Ashish Thapliyal, William Yang Wang, Radu Soricut.
    The 29th International Conference on Computational Linguistics (COLING 2022)
    Paper

  13. Imagination-Augmented Natural Language Understanding
    Yujie Lu, Wanrong Zhu, Xin Eric Wang, Miguel Eckstein, William Yang Wang.
    The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022, Oral)
    Paper | Code

  14. Diagnosing Vision-and-Language Navigation: What Really Matters
    Wanrong Zhu, Yuankai Qi, Pradyumna Narayana, Kazoo Sone, Sugato Basu, Xin Eric Wang, Qi Wu, Miguel Eckstein, William Yang Wang.
    The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022, Oral)
    Paper | Code

  15. Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation
    Wanrong Zhu, Xin Eric Wang, Tsu-Jui Fu, An Yan, Pradyumna Narayana, Kazoo Sone, Sugato Basu, William Yang Wang.
    The 16th conference of the European Chapter of the Association for Computational Linguistics (EACL 2021)
    Paper | Code

  16. Towards Understanding Sample Variance in Visually Grounded Language Generation: Evaluations and Observations
    Wanrong Zhu, Xin Eric Wang, Pradyumna Narayana, Kazoo Sone, Sugato Basu, William Wang.
    The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020, Short)
    Paper

  17. Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
    Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Singh Sachan, Eric P. Xing.
    The 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL 2019)
    Paper | Code


Powered by Jekyll and Minimal Light theme.