Le Zhuo

About Me

I am a BSc student at Beihang Univerisity, majoring in Computer Science and Technology. I am currently working with Prof. Jian Tang at Mila - Québec AI Institute. During my undergraduate years at BUAA, I was supervised by Prof. Si Liu and I was fortunate to work with Prof. Shuicheng Yan and Dr. Jie Fu at Beijing Academy of Artificial Intelligence. I was also a research intern at Alibaba Inc., where I worked on dance generation with Dr. Aixi Zhang.

I am currently looking for a PhD position starting from Fall 2024! Please feel free to contact me if you are interested.

Research Interests

Generative Modeling: Broad families of generative models and their potential applications, e.g., diffusion models, flow matching, etc.
AI4Science: Exploring ways to make deep learning algorithms more robust to common challenges in drug discovery, e.g., protein docking, molecular generation, etc.
Multimodal Learning: Developing models that seamlessly integrate and bridge the gap between various modalities, such as text, image, and audio.

News

[Dec. 2023] I am honored to receive the 2023 SenseTime Scholarship, which is awarded to 30 Chinese undergraduate students in the field of AI!
[Sep. 2023] Our benchmark paper about music representation learning is accepted to NeurIPS 2023!
[Jul. 2023] Our paper about music driven dance generation is accepted to ACM MM 2023 as an oral presentation!
[Jul. 2023] Our paper about video background music generation is accepted to ICCV 2023!
[Jun. 2023] Our paper about automatic lyrics transcription is accepted to ISMIR 2023!

Publications

Preprint

ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training

Le Zhuo*, Zewen Chi*, Minghao Xu*, Heyan Huang, Jianan Zhao, Heqi Zheng, Conghui He, Xian-Ling Mao, Wentao Zhang

Under review.

PDF Code Project Page

Preprint

LLMs as Visual Explainers: Advancing Image Classification with Evolving Visual Descriptions

Songhao Han*, Le Zhuo*, Yue Liao, Si Liu

Under review.

PDF Code

Preprint

GraphText: Graph Reasoning in Text Space

Jianan Zhao, Le Zhuo, Yikang Shen, Meng Qu, Kai Liu, Michael Bronstein, Zhaocheng Zhu, Jian Tang

Under review.

PDF

ACM MM

DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation

Qiaosong Qi*, Le Zhuo*, Aixi Zhang, Yue Liao, Fei Fang, Si Liu, Shuicheng Yan

ACM International Conference on Multimedia (ACM MM), 2023.

PDF Oral Presentation

ICCV

Video Background Music Generation: Dataset, Method and Evaluation

Le Zhuo*, Zhaokai Wang*, Baisen Wang*, Yue Liao, Chenxi Bao, Stanley Peng, Songhao Han, Aixi Zhang, Fei Fang, Si Liu

IEEE/CVF International Conference on Computer Vision (ICCV), 2023.

PDF Code

NeurIPS

MARBLE: Music Audio Representation Benchmark for Universal Evaluation

Ruibin Yuan*, Yinghao Ma*, Yizhi Li*, Ge Zhang*, Xingran Chen, Hanzhi Yin, Le Zhuo, Yiqi Liu, Jiawen Huang, Zeyue Tian, Binyue Deng, Ningzhi Wang, Chenghua Lin, Emmanouil Benetos, Anton Ragni, Norbert Gyenge, Roger Dannenbert, Wenhu Chen, Gus Xia, Wei Xue, Si Liu, Shi Wang, Ruibo Liu, Yike Guo, Jie Fu

Neural Information Processing Systems (NeurIPS), 2023.

PDF Code Project Page

ISMIR

LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT

Le Zhuo, Ruibin Yuan, Jiahao Pan, Yizhi Li, Ge Zhang, Jiawen Huang, Si Liu, Roger Dannenberg, Jie Fu, Chenghua Lin, Emmanouil Benetos, Wenhu Chen, Wei Xue, Yike Guo

International Society for Music Information Retrieval (ISMIR), 2023.

PDF Code

About Me

Research Interests

News

Publications

Services

Conference Reviewers