Andy T. Liu<br/>劉廷緯

Andy T. Liu
劉廷緯

Ph.D. graduate from NTU
Former Applied Scientist Intern at AWS

Table of Content

About Me

My name is Andy T. (Ting-Wei) Liu. I received my Ph.D. in Electrical Engineering & Computer Science (EECS) from the National Taiwan University (NTU) in January 2024, under the supervision of Prof. Hung-yi Lee. As part of the “Speech Processing and Machine Learning” lab at NTU, I also had the privilege of collaborating with Prof. Lin-shan Lee.

With over five years of dedicated experience, I specialize in Self-Supervised Learning (SSL), Speech Foundation Models, Automatic Speech Recognition (ASR), Large Language Models (LLM), and Multimodal Models. My research has garnered significant attention, accumulating over 1,800 citations and an h-index of 12 on Google Scholar.

One of my publications, “TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech,” was recognized by the IEEE Signal Processing Society as one of the top 25 downloaded articles from Sept. 2021 - Sept. 2022 for IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP) on IEEE Xplore®.

As a co-founder of the S3PRL Toolkit, which has earned over 2,100 stars on GitHub, I’ve been actively developing this resource since 2019. The S3PRL Toolkit provides an easy-to-use interface for the research community, offering access to self-supervised foundation models and downstream tasks.

I am passionate about advancing speech processing and machine learning technologies, and I remain committed to bridging the gap between cutting-edge research and practical applications in this dynamic field.

Back

Work Experience

  • Doctoral Research Assistant
    National Taiwan University (NTU)
    Taipei, Taiwan, Sep. 2018 - Jan. 2024.
    • When pursuing my PhD, I was advised by Prof. Hung-yi Lee and Prof. Lin-shan Lee from the “Speech Processing and Machine Learning” lab at NTU.
  • Applied Scientist Intern
    Amazon Web Services (AWS)
    New York, U.S.A, Jun. 2021 - Oct. 2021.
    • Worked with the Amazon Transcribe team under the supervision of Andrew Arnold and Wei Xiao.
    • Completed a research project that uses LLM (Large Language Models) to achieve low-resource NER (Named Entity Recognition).
  • Ph.D. Researcher
    ASUS Intelligent Cloud Services (AICS)
    Taipei, Taiwan, Sep. 2020 - Apr. 2024.
    • Under the AICS Ph.D. Student Program.
    • Worked with the AI research team in Singapore remotely under the supervision of Stefan Winkler (Nov. 2021 - Jan 2024).
    • Worked with the AI medical team under the supervision of Professor Chih-Jen Lin and Professor Victor Tsai.

Back

Extracurricular Experience

  • The S3PRL Toolkit: Self-Supervised Speech Pre-training and Representation Learning [ repo GitHub stars ]
  • SUPERB: Speech processing Universal PERformance Benchmark [ website GitHub stars ]
  • ZeroSpeech TTS-without-T Challenge [ repo ]
  • Tacotron English TTS [ repo ]
  • Tacotron Code-Switch TTS [ repo ]
  • Sequence GAN Chatbot [ repo ]

Back

Honors

  • ASUS Ph.D. Scholarship (華碩AI研發中心博士生學位計畫) Sep. 2020 - Jun. 2023

  • Merry Electro-Acoustic Thesis Award - 1st Place (美律電聲論文獎 - 金質獎) 2020

  • NTU Frontier Speech Technology Scholarship (國立臺灣大學前瞻語音科技獎學金) Oct. 2019 - Aug. 2020

  • FAOS Outstanding Students Conference Travel Grant (傑出人才優秀學生出國開會補助) 2019

  • NTU Electrical Engineering Innovation Award - 2nd place (國立臺灣大學電機系精專獎 - 貳獎) 2017

  • Macau Government Lotus Award (澳門政府蓮花獎) 2014

Back

Presentations

Back

Publications

*Sorted by recency

  • On the social bias of speech self-supervised models
    Yi-Cheng Lin, Tzu-Quan Lin, Hsi-Che Lin, Andy T. Liu, Hung-yi Lee
    Accepted by INTERSPEECH 2024, conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf ]

  • A Large-Scale Evaluation of Speech Foundation Models
    Shu-wen Yang, Andy T. Liu, Heng-Jui Chang, Zili Huang, Cheng-I Lai, Haibin Wu, Jiatong Shi, Xuankai Chang, Hsiang-Sheng Tsai, Wen-Chin Huang, Tzu-hsun Feng, Po-Han Chi, Yist Y. Lin, Yung-Sung Chuang, Tzu-Hsien Huang, Wei-Cheng Tseng, Kushal Lakhotia, Shang-Wen Li, Abdelrahman Mohamed, Shinji Watanabe, Hung-yi Lee
    Published in IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 32, 2024 (TASLP)
    [ ieee | arxiv ]

  • Parallel Synthesis for Autoregressive Speech Generation
    Po-chun Hsu, Da-rong Liu, Andy T. Liu, Hung-yi Lee
    Published in IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 31, 2023 (TASLP)
    [ ieee | arxiv ]

  • QaNER: Prompting question answering models for few-shot named entity recognition
    Andy T. Liu, Wei Xiao, Henghui Zhu, Dejiao Zhang, Shang-Wen Li, Andrew Arnold
    Work done while interning at Amazon AI. arXiv preprint, 2022, Cornell University
    [ arxiv | pdf ]

  • SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities
    Hsiang-Sheng Tsai, Heng-Jui Chang, Wen-Chin Huang, Zili Huang, Kushal Lakhotia, Shu-wen Yang, Shuyan Dong, Andy T. Liu, Cheng-I Jeff Lai, Jiatong Shi, Xuankai Chang, Phil Hall, Hsuan-Jui Chen, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
    Accepted by ACL 2022, Annual Meeting of the ACL, organized by the Association for Computational Linguistics (ACL)
    [ acl | arxiv | pdf | code ]

  • SUPERB: Speech processing Universal PERformance Benchmark
    Shu-wen Yang, Andy T Liu, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y Lin, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko-tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
    Accepted by INTERSPEECH 2021, conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf | code ]

  • Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning
    Haibin Wu, Xu Li, Andy T. Liu, Zhiyong Wu, Helen Meng, Hung-yi Lee
    Published in IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 30, 2021 (TASLP)
    [ ieee | arxiv | pdf | code ]

  • Don’t Speak Too Fast: The Impact of Data Bias on Self-Supervised Speech Models
    Yen Meng, Yi-Hui Chou, Andy T. Liu, Hung-yi Lee
    Lecture session in ICASSP 2022, conference organized by the IEEE Signal Processing Society (SPS)
    [ ieee | arxiv | pdf | code ]

  • TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
    Andy T. Liu, Shang-Wen Li, Hung-yi Lee
    Published in IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 29, 2021 (TASLP)
    [ ieee | arxiv | pdf | code ]

  • Adversarial Defense for Automatic Speaker Verification by Cascaded Self-Supervised Learning Models
    Haibin Wu, Xu Li, Andy T. Liu, Zhiyong Wu, Helen Meng, Hung-yi Lee
    Virtual session in ICASSP 2021, a conference organized by the IEEE Signal Processing Society (SPS)
    [ ieee | arxiv | pdf | code ]

  • Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning
    Haibin Wu, Andy T. Liu, Hung-yi Lee
    Virtual session in INTERSPEECH 2020, a conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf | code ]

  • Understanding Self-Attention of Self-Supervised Audio Transformers
    Shu-wen Yang, Andy T. Liu, Hung-yi Lee
    Virtual session in INTERSPEECH 2020, a conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf | demo ]

  • Towards Robust Neural Vocoding for Speech Generation: A Survey
    Po-chun Hsu, Chun-hsuan Wang, Andy T. Liu, Hung-yi Lee
    arXiv preprint, 2020, Cornell University
    [ arxiv | pdf | demo ]

  • Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders
    Andy T. Liu, Shu-wen Yang, Po-Han Chi, Po-chun Hsu, Hung-yi Lee
    Lecture session in ICASSP 2020, a conference organized by the IEEE Signal Processing Society (SPS)
    [ ieee | arxiv | pdf | code | slide | talk ]

  • Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion
    Andy T. Liu, Po-chun Hsu, Hung-yi Lee
    Oral session in INTERSPEECH 2019, a conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf | code | slide ]

Back

Professional Volunteer Experience

Back

Teaching Assistant Experience

Back

Interests

  • PADI Scuba Diving Instructor, May. 2023 - Present
  • Scuba Diving, Jul. 2020 - Present
  • Photographer, Amateur, Nov. 2021 - Present
  • Guitar, Amateur, Oct. 2016 - Jan. 2020
  • Road Biking, (NTU Cycling Club), Jun. 2014 - Aug. 2017
  • Pet geckos: Instagram

Back

Contact

liuandyt@gmail.com

Write me if you are looking for a collaboration!