Tsu-Jui (Ray) Fu

I am a 3rd-year Ph.D. student at UCSB, advised by William Wang. I work on language grounding and interacting with environments using language. My goal is to bridge the gap between vision and language via the AI system.


2019 - Now
Research Assistant @ UCSB NLP
Advisor: William Yang Wang

Summer 2022
Research Intern @ Meta AI
Advisor: Licheng Yu and Sean Bell

Summer 2021
Research Intern @ Microsoft Azure AI
Advisor: Linjie Li, Zhe Gan, and Lijuan Wang

Summer 2020
Research Intern @ Microsoft Research
Advisor: Yale Song and Daniel McDuff

Summer 2019
Research Intern @ Preferred Networks
Advisor: Yuta Tsuboi and Jason Naradowsky

2018 - 2019
Research Assistant @ Academia Sinica CKIP
Advisor: Wei-Yun Ma

VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, and Zicheng Liu
Paper / Code

Language-Driven Artistic Style Transfer
Tsu-Jui Fu, Xin Eric Wang, and William Yang Wang
Paper / Code

M3L: Language-based Video Editing via Multi-Modal Multi-Level Transformer
Tsu-Jui Fu, Xin Eric Wang, Scott Grafton, Miguel Eckstein, and William Yang Wang
Paper / Slide / Video / Dataset

DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents
Tsu-Jui Fu, William Yang Wang, Daniel McDuff, and Yale Song
Paper / Slide / Video / Dataset

H-FND: Hierarchical False-Negative Denoising for Distant Supervision Relation Extraction
Tsu-Jui Fu*, Jhih-Wei Chen*, Chen-Kang Lee, and Wei-Yun Ma
Findings of ACL'21 (Long)
Paper / Slide / Video / Code

Semi-Supervised Policy Initialization for Playing Games with Language Hints
Tsu-Jui Fu and William Yang Wang
NAACL'21 (Short)
Paper / Slide / Video / Code

L2C: Describing Visual Differences Needs Semantic Understanding of Individuals
An Yan, Xin Eric Wang, Tsu-Jui Fu, and William Yang Wang
EACL'21 (Short)

Multimodal Style Transfer Learning for Outdoor Vision-and-Language Navigation
Wanrong Zhu, Xin Eric Wang, Tsu-Jui Fu, An Yan, Pradyumna Narayana, Kazoo Sone, Sugato Basu, and William Yang Wang
EACL'21 (Long)
Paper / Code

SSCR: Iterative Language-Based Image Editing via Self-Supervised Counterfactual Reasoning
Tsu-Jui Fu, Xin Eric Wang, Scott Grafton, Miguel Eckstein, and William Yang Wang
EMNLP'20 (Oral)
Paper / Slide / Code

Counterfactual Vision-and-Language Navigation via Adversarial Path Sampler
Tsu-Jui Fu, Xin Eric Wang, Matthew Peterson, Scott Grafton, Miguel Eckstein, and William Yang Wang
ECCV'20 (Spotlight)
Paper / Slide / Video / Model

Why Attention? Analyzing and Remedying BiLSTM Deficiency in Modeling Cross-Context for NER
Peng-Hsuan Li, Tsu-Jui Fu, and Wei-Yun Ma
AAAI'20 (Oral)
Paper / Code

A Distributed Scheme for Accelerating Semantic Video Segmentation on An Embedded Cluster
Tsu-Jui Fu*, Hsuan-Kung Yang*, Kuan-Wei Ho, Po-Han Chiang and Chun-Yi Lee
ICCD'19 (Oral)
Paper / Video

Adversarial Active Exploration for Inverse Dynamics Model Learning
Zhang-Wei Hong, Tsu-Jui Fu, Tzu-Yun Shann, Yi-Hsiang Chang, and Chun-Yi Lee
CoRL'19 (Oral)

GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction
Tsu-Jui Fu, Peng-Hsuan Li, and Wei-Yun Ma
ACL'19 (Long)
Paper / Slide / Code

Attentive and Adversarial Learning for Video Summarization
Tsu-Jui Fu, Shao-Heng Tai, and Hwann-Tzong Chen
WACV'19 (Oral)
Paper / Video / Code

Region-Semantics Preserving Image Synthesis
Kang-Jun Liu, Tsu-Jui Fu, and Shan-Hung Wu
Paper / Video / Code

Diversity-Driven Exploration Strategy for Deep Reinforcement Learning
Zhang-Wei Hong, Tzu-Yun Shann, Shih-Yang Su, Yi-Hsiang Chang, Tsu-Jui Fu, and Chun-Yi Lee
Paper / Video

Speed Reading: Learning to Read ForBackward via Shuttle
Tsu-Jui Fu and Wei-Yun Ma
EMNLP'18 (Long)
Paper / Code

Dynamic Video Segmentation Network
Yu-Syuan Xu, Tsu-Jui Fu*, Hsuan-Kung Yang*, and Chun-Yi Lee
Paper / Video / Code

Template from Jon Barron