Andy T. Liu<br/>劉廷緯

Andy T. Liu
劉廷緯

Ph.D. graduate from NTU
Former Applied Scientist Intern at AWS

Table of Content

About Me

My name is Andy T. (Ting-Wei) Liu. I received my Ph.D. degree in the College of Electrical Engineering & Computer Science (EECS) from the National Taiwan University (NTU), Taipei, Taiwan, in 2024, supervised by Professor Hung-Yi Lee. I’m from the “Speech Processing and Machine Learning” laboratory at NTU, also working with Professor Lin-shan Lee.

I offer more than five years of dedicated experience in machine learning, self-supervised learning, foundation models, and generative models for speech and NLP.

My scholarly impact is affirmed by over 1,500 citations and an h-index of 12 on Google Scholar. The IEEE Signal Processing Society has identified one of my publications, “TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech”, as one of the top 25 downloaded articles from Sept. 2021 - Sept. 2022 for IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP) on IEEE Xplore®.

My colleague Shu-wen (Leo) Yang and I founded the S3PRL Toolkit in 2019 – now celebrated with over 2,000 stars on GitHub – and have kept developing ever since. Our S3PRL Toolkit provides an easy-to-use interface to the research community, including self-supervised foundation models and downstream tasks.

Back

Work Experience

  • Applied Scientist Intern
    AWS (Amazon Web Services)
    New York, U.S.A, Jun. 2021 - Oct. 2021.
    • Worked with the Amazon Transcribe team under the supervision of Andrew Arnold and Wei Xiao.
    • Completed a research paper that uses prompting foundation models to achieve low-resource NER (Named Entity Recognition).
    • Built fully automated machine learning NLP pipelines for few-shot NLP using PyTorch and Huggingface.
    • Implemented several few-shot baselines and adopted them on multiple datasets.
  • Ph.D. Student Program
    AICS (ASUS Intelligent Cloud Services)
    Taipei, Taiwan, Nov. 2021 - Jun. 2023.
    • Worked with the AICS Singapore AI research team under the supervision of Stefan Winkler.
    • Worked on research projects developing AI solutions in the healthcare sector.
  • Ph.D. Student Program
    AICS (ASUS Intelligent Cloud Services)
    Taipei, Taiwan, Sep. 2020 - Jun. 2021.
    • Worked with the AICS medical team under the supervision of Professor Chih-Jen Lin and Professor Victor Tsai.
    • Worked on representation learning algorithms that encode patient information in the medical domain.

Back

Extracurricular Experience

  • The S3PRL Toolkit: Self-Supervised Speech Pre-training and Representation Learning [ repo GitHub stars ]
  • SUPERB: Speech processing Universal PERformance Benchmark [ website GitHub stars ]
  • ZeroSpeech TTS-without-T Challenge [ repo ]
  • Tacotron English TTS [ repo ]
  • Tacotron Code-Switch TTS [ repo ]
  • Sequence GAN Chatbot [ repo ]

Back

Honors

  • ASUS Ph.D. Scholarship (華碩AI研發中心博士生學位計畫) Sep. 2020 - Jun. 2023

  • Merry Electro-Acoustic Thesis Award - 1st Place (美律電聲論文獎 - 金質獎) 2020

  • NTU Frontier Speech Technology Scholarship (國立臺灣大學前瞻語音科技獎學金) Oct. 2019 - Aug. 2020

  • FAOS Outstanding Students Conference Travel Grant (傑出人才優秀學生出國開會補助) 2019

  • NTU Electrical Engineering Innovation Award - 2nd place (國立臺灣大學電機系精專獎 - 貳獎) 2017

  • Macau Government Lotus Award (澳門政府蓮花獎) 2014

Back

Presentations

Back

Publications

*Sorted by recency

  • Parallel Synthesis for Autoregressive Speech Generation
    Po-chun Hsu, Da-rong Liu, Andy T. Liu, Hung-yi Lee
    Published in IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 31, 2023 (TASLP)
    [ ieee | arxiv ]

  • QaNER: Prompting question answering models for few-shot named entity recognition
    Andy T. Liu, Wei Xiao, Henghui Zhu, Dejiao Zhang, Shang-Wen Li, Andrew Arnold
    Work done while interning at Amazon AI. arXiv preprint, 2022, Cornell University
    [ arxiv | pdf ]

  • SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities
    Hsiang-Sheng Tsai, Heng-Jui Chang, Wen-Chin Huang, Zili Huang, Kushal Lakhotia, Shu-wen Yang, Shuyan Dong, Andy T. Liu, Cheng-I Jeff Lai, Jiatong Shi, Xuankai Chang, Phil Hall, Hsuan-Jui Chen, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
    Accepted by ACL 2022, conference organized by the Association for Computational Linguistics Association (ISCA)
    [ acl | arxiv | pdf | code ]

  • SUPERB: Speech processing Universal PERformance Benchmark
    Shu-wen Yang, Andy T Liu, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y Lin, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko-tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
    Accepted by INTERSPEECH 2021, conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf | code ]

  • Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning
    Haibin Wu, Xu Li, Andy T. Liu, Zhiyong Wu, Helen Meng, Hung-yi Lee
    Published in IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 30, 2021 (TASLP)
    [ ieee | arxiv | pdf | code ]

  • Don’t Speak Too Fast: The Impact of Data Bias on Self-Supervised Speech Models
    Yen Meng, Yi-Hui Chou, Andy T. Liu, Hung-yi Lee
    Lecture session in ICASSP 2022, conference organized by the IEEE Signal Processing Society (SPS)
    [ ieee | arxiv | pdf | code ]

  • TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
    Andy T. Liu, Shang-Wen Li, Hung-yi Lee
    Published in IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 29, 2021 (TASLP)
    [ ieee | arxiv | pdf | code ]

  • Adversarial Defense for Automatic Speaker Verification by Cascaded Self-Supervised Learning Models
    Haibin Wu, Xu Li, Andy T. Liu, Zhiyong Wu, Helen Meng, Hung-yi Lee
    Virtual session in ICASSP 2021, a conference organized by the IEEE Signal Processing Society (SPS)
    [ ieee | arxiv | pdf | code ]

  • Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning
    Haibin Wu, Andy T. Liu, Hung-yi Lee
    Virtual session in INTERSPEECH 2020, a conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf | code ]

  • Understanding Self-Attention of Self-Supervised Audio Transformers
    Shu-wen Yang, Andy T. Liu, Hung-yi Lee
    Virtual session in INTERSPEECH 2020, a conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf | demo ]

  • Towards Robust Neural Vocoding for Speech Generation: A Survey
    Po-chun Hsu, Chun-hsuan Wang, Andy T. Liu, Hung-yi Lee
    arXiv preprint, 2020, Cornell University
    [ arxiv | pdf | demo ]

  • Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders
    Andy T. Liu, Shu-wen Yang, Po-Han Chi, Po-chun Hsu, Hung-yi Lee
    Lecture session in ICASSP 2020, a conference organized by the IEEE Signal Processing Society (SPS)
    [ ieee | arxiv | pdf | code | slide | talk ]

  • Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion
    Andy T. Liu, Po-chun Hsu, Hung-yi Lee
    Oral session in INTERSPEECH 2019, conference organized by the International Speech Communication Association (ISCA)
    [ isca | arxiv | pdf | code | slide ]

Back

Professional Volunteer Experience

Back

Teaching Assistant Experience

Back

Interests

  • PADI Scuba Diving Instructor, May. 2023 - Present
  • Scuba Diving, Jul. 2020 - Present
  • Photographer, Amateur, Nov. 2021 - Present
  • Guitar, Amateur, Oct. 2016 - Jan. 2020
  • Road Biking, (NTU Cycling Club), Jun. 2014 - Aug. 2017
  • Pet geckos: Instagram

Back

Contact

liuandyt@gmail.com

Write me if you are looking for a collaboration!