AI & Robotics Researcher

YuanFu Yang

Assistant Professor
Institute of Artificial Intelligence Innovation
National Yang Ming Chiao Tung University (NYCU), Hsinchu, Taiwan

Assistant Professor at NYCU Institute of Artificial Intelligence Innovation. Pioneering research in Computer Vision, Generative AI, and Robotic AI — bridging the gap between intelligent systems and real-world automation. Former Data Scientist & Team Lead at TSMC.

Portrait of YuanFu Yang
YuanFu Yang · NYCU
01 — Biography

About

Scholar, engineer, and builder of perceptive machines.

I am an Assistant Professor at the Institute of Artificial Intelligence Innovation, National Yang Ming Chiao Tung University (NYCU), Taiwan, where I lead research at the intersection of computer vision, generative AI, and robotic intelligence.

Prior to academia, I spent 17 years at TSMC (2006–2023) as a Data Scientist and Team Lead, where I drove AI adoption for semiconductor manufacturing — developing state-of-the-art deep learning systems for defect inspection, yield prediction, and robotic automation.

My mission is to build intelligent machines that perceive, reason, and act in complex real-world environments, with a focus on bridging simulation and reality.

PositionAssistant Professor, NYCU
EducationPh.D. in EECS, National Tsing Hua University
LocationHsinchu, Taiwan
02 — Research

Research Focus

Three interleaved threads — perception, generation, and embodiment.

01 / CV

Computer Vision

Developing 2D and 3D foundation models for robust perception. Our work spans object detection, instance segmentation, depth estimation, and dense 3D scene reconstruction — with applications in smart manufacturing and autonomous systems.

2D/3D Foundation Models Object Detection 3D Reconstruction Defect Inspection Small Object Tracking
02 / GEN

Generative AI

Harnessing diffusion models and generative architectures to create digital twins, synthesize training data, and model complex industrial processes. Bridging the gap between simulation and physical reality through learned generative representations.

Diffusion Models Generative Digital Twins Process Optimization Anomaly Detection Quantum-Classical AI
03 / ROB

Robotic AI

Building robots that see, understand, and act. Our research combines reinforcement learning, vision-language models, and sim-to-real transfer to create intelligent agents capable of complex manipulation and navigation in unstructured environments.

Reinforcement Learning Vision-Language Models Sim-to-Real Transfer Robot Manipulation Autonomous Agents
03 — Experience

Appointments

From silicon to students — an industry-to-academia arc.

  1. Assistant Professor
    Current
    Institute of AI Innovation, NYCU
    2024 – Present
    Leading the ROSSI Lab with research focus on Computer Vision, Generative AI, and Robotic AI. Principal Investigator for the Frontier Robotic AI Project (NSTC 2025–2027). Actively recruiting graduate and doctoral students.
  2. Chief Consultant
    Current
    MIRLE Automation
    2025 – Present
    Providing strategic AI consulting for industrial automation systems, integrating computer vision and robotic AI solutions into manufacturing workflows.
  3. AI Consultant
    2024 – 2025
    LCY GROUP
    2024 – 2025
    Applied AI expertise to accelerate industrial digitalization initiatives across chemical manufacturing processes and supply chain optimization.
  4. Data Scientist & Team Lead
    17 Years
    TSMC (Taiwan Semiconductor Manufacturing Co.)
    2006 – 2023
    • Led AI/ML teams developing deep learning systems for wafer defect inspection, classification, and yield prediction.
    • Deployed robotic automation systems improving factory throughput and quality control.
    • Progressed from engineer to Chief Engineer and Supervisor, mentoring cross-functional teams.
    • Won multiple internal innovation competitions for AI-driven manufacturing solutions.
04 — Teaching

Courses

Graduate courses taught at the Institute of Artificial Intelligence Innovation, NYCU.

Computer Vision

Graduate Course Spring 2025 Institute of AI Innovation, NYCU Hsinchu, Taiwan

Course Overview & Objectives

This course aims to provide students with a deep understanding of the fundamental concepts and techniques of computer vision, including image formation, feature extraction, 3D reconstruction, image segmentation, object recognition, deep learning, object detection, object tracking, and facial recognition. It also aims to equip students with the implementation and application of algorithms, models, and frameworks related to computer vision, enabling them to address computer vision problems across various domains such as autonomous driving, smart homes, medical image analysis, and more. Through this course, students will comprehend the limitations and challenges of current computer vision applications and explore future development directions. By undertaking this course, students will acquire the following abilities:

  • Fundamental knowledge and skills in computer vision and image processing.
  • Sensitivity and analytical skills toward emerging technologies and trends.
  • Capability to conduct independent research and development, possessing teamwork and project management abilities.
  • Innovative thinking and problem-solving skills, with the capacity to apply learned knowledge to promote technological innovation and societal progress.

Prerequisites

  • Proficiency in Python
    • All class assignments will be in Python. If you have a lot of programming experience but in a different language (e.g. C++/Matlab/Javascript) you will probably be fine.
  • College Calculus, Linear Algebra
    • You should be comfortable taking derivatives and understanding matrix vector operations and notation.
  • Basic Probability and Statistics
    • You should know basics of probabilities, gaussian distributions, mean, standard deviation, etc.

Grading

  • Assignments (40%) — Including programming assignments, literature review reports, etc.
  • Midterm Report (20%) — Students are required to select a computer vision-related paper from the past three years and write a research report. Additional points will be awarded if the report includes a demo and technical implementation.
  • Final Project (40%) — Students must choose a computer vision-related topic and produce both a research report and a practical implementation project. The grading criteria include a clear understanding of the problem, the innovation and practicality of the solution, and the completeness and effectiveness of the technical implementation.
  • Class Participation (10%) — −1 each absent.

Office Hours

  • Monday 11:00–12:00 am
  • Room: Engineering Building-6 (374)

Schedule

WkDateTopicSlidesHomeworkNotes
12/18Course Introduction Lec0 Lec1
22/25CV Introduction / Image Formation Lec2
33/4Intensity Transformation Lec3 HW1: Image Sensing Pipeline
43/11Edge Detection Lec4 Group Form Due
53/18Corner Detection Lec5 HW1 Due
63/25Line Detection Lec6 HW2: Harris Corner Detection
74/1Camera Calibration Lec7
84/8Midterm Report HW2 & Midterm Report Due
94/15Image Segmentation Lec8 HW3: Camera Calibration
104/22Special Lecture
114/29Object Detection Lec9 HW3 Due
125/6Deep Image Segmentation Lec10 HW4: Image Segmentation
135/13Image Classification Lec11
145/20VLM / LLM Lec12 HW4 Due
155/273D Vision Lec13
166/3Final Project Presentation Final Report Due

Resources

Textbooks

Generative AI

Graduate Course Fall 2024 Institute of AI Innovation, NYCU Hsinchu, Taiwan

Course Overview & Objectives

This course provides students with a comprehensive introduction to the fundamentals of Generative Artificial Intelligence (GenAI), with a particular focus on image synthesis. It is designed to cultivate both theoretical understanding and practical implementation skills, while encouraging students to critically examine the applications of GenAI across diverse domains. By engaging with state-of-the-art models and techniques, students will explore the current limitations and challenges of GenAI, as well as emerging research trends and potential future directions in image generation.

To strengthen the connection between theory and practice, the course features joint instruction with Dr. Chia-Min Cheng, Senior Expert from AI Technology Division of MediaTek, who will share cutting-edge industry insights. His lectures will bridge academic concepts with real-world product development, highlighting both technical challenges and commercial opportunities.

  • Acquire foundational knowledge and technical skills in generative AI and image synthesis.
  • Demonstrate sensitivity to emerging technologies and the ability to critically analyze new trends.
  • Conduct independent research and development projects, while also contributing effectively to team collaboration and project management.
  • Apply innovative thinking and problem-solving skills to advance technological innovation and create broader societal impact.

Prerequisites

  • Programming Proficiency in Python
    • All assignments will be implemented in Python. Students with extensive programming experience in other languages (e.g., C++, Matlab, or JavaScript) should still be able to adapt.
  • Mathematics Foundations
    • Calculus and Linear Algebra: Ability to take derivatives, and familiarity with matrix–vector operations and notation.
    • Probability and Statistics: Understanding of basic probability concepts, Gaussian distributions, mean, standard deviation, and related statistical measures.
  • Deep Learning / Machine Learning

Grading

  • Assignments (20%) — Including programming assignments, literature review reports, etc.
  • Midterm Report (30%) — Students need to select a GenAI-related paper from the past three years and write a research report. The report should include Background, Motivation, Proposed Method, Result, and Personal Reflection. Additional points will be awarded for including a demo and technical implementation in the report.
  • Final Project (40%) — Including project presentation and final project report. Students are required to select a GenAI-related topic, develop a proposal, and undertake an implementation project. The evaluation criteria encompass a thorough comprehension of the problem, innovation, practicality of the solution, completeness, and performance of the technical implementation. Additional credit will be awarded for incorporating a demo and technical implementation.
  • Class Participation (10%) — For each absence during roll call: −1 for the final grade.

Office Hours

  • Monday 11:00–12:00 am
  • Room: Engineering Building-6 (374)

Schedule

WkDateTopicSlidesHomeworkNotes
19/4Class Introduction and Overview Lec0
29/11Introduction of GenAI Lec1
39/18Overview of Vision Generative Models Lec2
49/25Autoencoder Lec3 Group Form Due
510/2Basic Principles and Concepts of GANs Lec4 HW1
610/9Applications and Developments of GANs Lec5
710/16Basic Principles and Concepts of DMs Lec6 HW1 Due
810/23Special Lecture
910/30Midterm Report
1011/6Applications and Developments of DMs Midterm Report Due
1111/133D Vision Lec7 HW2
1211/20GenAI Application in Industry — Mobile, Automotive, AR/VR Lec8
1311/273D Visual Effects Lec9HW2 Due
1412/4Mixed Reality Lec10
1512/11VLM & LMM Lec11
1612/18Final Project Presentation Final Report Due

Artificial Intelligence (IIAI30017)

Graduate Course Fall 2024 Institute of AI Innovation, NYCU Hsinchu, Taiwan

Course Overview & Objectives

This course, Artificial Intelligence (IIAI30017), offers students a comprehensive introduction to the fundamental concepts, techniques, and applications of artificial intelligence (AI). Through lectures, programming assignments, literature reviews, and project-based learning, students will gain both theoretical understanding and practical skills in AI.

  • Provide a strong foundation in core AI principles, including search, knowledge representation, reasoning, probability, and machine learning.
  • Develop proficiency in Python programming for AI applications, reinforcing concepts through practical assignments.
  • Enhance students' ability to critically read, evaluate, and present recent AI research papers.
  • Cultivate problem-solving skills through a midterm report and a final project, encouraging creativity, innovation, and technical rigor.
  • Equip students with hands-on experience in developing AI solutions that bridge theory and practice, with additional focus on demos and implementations.
  • Encourage teamwork, communication, and class participation, preparing students to engage in both academic and industrial AI contexts.

Prerequisites

  • Programming Proficiency in Python
    • All assignments will be implemented in Python. Students with extensive programming experience in other languages (e.g., C++, Matlab, or JavaScript) should still be able to adapt.
  • Mathematics Foundations
    • Calculus and Linear Algebra: Ability to take derivatives, and familiarity with matrix–vector operations and notation.
    • Probability and Statistics: Understanding of basic probability concepts, Gaussian distributions, mean, standard deviation, and related statistical measures.

Grading

  • Assignments (30%) — Including programming assignments, literature review reports, etc.
  • Midterm Report (20%) — Students need to select an AI-related paper from the past three years and write a research report. The report should include Background, Motivation, Proposed Method, Result, and Personal Reflection. Additional points will be awarded for including a demo and technical implementation in the report.
  • Final Project (40%) — Including project presentation and final project report. Students are required to select an AI-related topic, develop a proposal, and undertake an implementation project. The evaluation criteria encompass a thorough comprehension of the problem, innovation, practicality of the solution, completeness, and performance of the technical implementation. Additional credit will be awarded for incorporating a demo and technical implementation.
  • Class Participation (10%) — For each absence during roll call: −1 for the final grade.

Office Hours

  • Monday 11:00–12:00 am
  • Room: Engineering Building-6 (374)

Schedule

WkDateTopicSlidesHomeworkNotes
19/2Class Introduction and Overview Lec0
29/9Introduction of Artificial Intelligence / Python Tutorial Lec1
39/16Informed Search Lec2
49/23Evolutionary Algorithms 1 Lec3 HW1: Greedy Best-First Search Group Form Due
59/30Evolutionary Algorithms 2
610/7Local Search Methods Lec4 HW1 Due
710/14Swarm Intelligence 1 Lec5 HW2: GA
810/21Swarm Intelligence 2
910/28Midterm Report HW2 Due
1011/4Neural Networks Lec6 Midterm Report Due
1111/11Convolutional Architecture Lec7
1211/18Training Neural Networks 1 Lec8 HW3: Neural Networks
1311/25Training Neural Networks 2
1412/23D Vision Lec9HW3 Due
1512/9Final Project Presentation
1612/16Special Lecture Final Report Due

Textbook

05 — Publications

Publications

Peer-reviewed papers, journal articles, and competition entries — in reverse chronological order.

2026
  1. Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset
    Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Denver 2026 arXiv:2512.24160
  2. Generative Digital Twins: Vision-Language Simulation Models for Executable Industrial Systems
    Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Findings, Denver 2026 arXiv:2512.20387
  3. SceneFoundry: Generating Interactive Infinite 3D Worlds
    Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Findings, Denver 2026 arXiv:2601.05810
2025
  1. Photolithography Overlay Map Generation with Implicit Knowledge Distillation Diffusion Transformer
    Proc. IEEE/CVF International Conference on Computer Vision (ICCV), Honolulu 2025 PDF
  2. DLSF: Dual-Layer Synergistic Fusion for High-Fidelity Image Synthesis
    Int'l Conf. on Machine Vision Applications (MVA), Kyoto 2025 Oral IEEE Xplore
  3. Boosting Small Object Tracking via Collaborative Detection Transformer
    Int'l Conf. on Machine Vision Applications (MVA), Kyoto 2025 Oral IEEE Xplore
  4. MVA 2025 Small Multi-Object Tracking for Spotting Birds Challenge: Dataset, Methods, and Results
    Int'l Conf. on Machine Vision Applications (MVA), Kyoto 2025 Challenge Winner IEEE Xplore
2024
  1. Knowledge Distillation Cross Domain Diffusion Model: A Generative AI Approach for Defect Pattern Segmentation
    IEEE Transactions on Semiconductor Manufacturing (IEEE TSM) 2024 IEEE Xplore
2023
  1. Medium. Permeation: SARS-COV-2 Painting Creation by Generative Model
    Kyoto Asian Modern Art Exhibition (KAMC), Kyoto 2023 arXiv:2304.11354
2022
  1. Semiconductor Defect Detection by Hybrid Classical-Quantum Deep Learning
    Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans 2022 IEEE Xplore
  2. QRF: Implicit Neural Representations with Quantum Radiance Fields
    arXiv preprint 2022 arXiv:2211.03418
  3. Hybrid Quantum-Classical Machine Learning for Lithography Hotspot Detection
    Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs 2022 IEEE Xplore
2021
  1. Semiconductor Defect Pattern Classification by Self-Proliferation-and-Attention Neural Network
    IEEE Transactions on Semiconductor Manufacturing (IEEE TSM) 2021 IEEE Xplore
  2. A Novel Deep Learning Architecture for Global Defect Classification: Self-Proliferating Neural Network (SPNet)
    Advanced Semiconductor Manufacturing Conference (ASMC), Milpitas 2021 IEEE Xplore
2020
  1. Double Feature Extraction Method for Wafer Map Classification Based on Convolution Neural Network
    Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs 2020 Best Paper IEEE Xplore
2019
  1. A Deep Learning Model for Identification of Defect Patterns in Semiconductor Wafer Map
    Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs 2019 IEEE Xplore
View on Google Scholar
06 — Recognition

Awards & Honors

Selected distinctions and service roles.

MVA Small Multi-Object Tracking Challenge — Winner
Machine Vision Applications · 2025
Best Paper Award
SEMI Advanced Semiconductor Manufacturing Conference (ASMC) · 2020
CVPR TechArt Exhibition — Selected Piece
CVPR, Vancouver · 2023
TSMC Internal Innovation Competition — Multiple Awards
1st Place, AI & Automation Category · 2018–2022
Area Chair — Machine Vision Applications Conference
MVA 2025 · Present
Principal Investigator — Frontier Robotic AI Project
NSTC Grant · 2025–2027
07 — Contact

Get in Touch

Open to collaborations, student applications, and speaking invitations.

I am actively recruiting graduate and doctoral students passionate about AI, robotics, and computer vision. If you are interested in joining the ROSSI Lab or collaborating on research, feel free to reach out.