Hardware for Machine Learning

AI Hardware
Course Overview

Machine learning has emerged to be a key approach to solving complex cognition and learning problems. Deep neural networks, in particular, have become pervasive due to their successes across a variety of applications, including computer vision, speech recognition, natural language processing, etc. While machine learning algorithms deliver impressive accuracy on many deployment scenarios, the complexity of the algorithms also poses a unique computational challenge to state-of-the-art hardware design.

To this end, this course is designed to help students come up to speed on various aspects of hardware for machine learning, including basics of deep learning, deep learning frameworks, hardware accelerators, co-optimization of algorithms and hardware, training and inference, support for state-of-the-art deep learning networks. In particular, this course is structured around building hardware prototypes for machine learning systems using state-of-the-art platforms (e.g., FPGAs and ASICs). It's also a seminar-style course so students are expected to present, discuss, and interact with research papers. At the end of the semester, students will present their work based on a class research project.

Instructor
Sophia Shao Ph.D.

Professor Sophia Shao

ysshao at berkeley dot edu

Teaching Assistants
Alon Amid

Alon Amid

alonamid at berkeley dot edu

Hasan Genc

Hasan Genc

hngenc at berkeley dot edu

Guest Speakers
Yu-Hsin Chen

Yu-Hsin Chen

Research Scientist, Facebook AR/VR

Eric Chung

Eric Chung

Principal Research Manager, Microsoft

Eric Chung

Ron Diamant

Principal Engineer, AWS

Dave Fick

Dave Fick

CTO/Founder, Mythic

Amir Gholami

Amir Gholami

Research Fellow, UC Berkeley

Randy Huang

Randy Huang

Principal Engineer, AWS

Vijay Janapa Reddi

Vijay Janapa Reddi

Associate Professor, Harvard University/MLPerf Inference Co-Chair

Mikhail Smelyanskiy

Mikhail Smelyanskiy

Director, Facebook AI System

Cliff Young

Cliff Young

Software Engineer, Google

Brian Zimmer

Brian Zimmer

Senior Research Scientist, NVIDIA

Lectures and Office Hours
Lectures Mon, Wed 2:00 pm - 3:30 pm 540AB Cory Prof. Sophia Shao
Office Hours Mon 11:00 am - 12:00 pm 570 Cory Prof. Sophia Shao
Tue 10:00 am - 11:00 am 567 Soda Alon Amid
Thu 11:30 am - 12:30 pm 567 Soda Hasan Genc
Grading
Readings 25 %
Labs 25 %
Projects 50 %
Labs
  • Ask questions on our Piazza forum.
  • Labs will be posted as links in the outline below. Please submit completed lab via Gradescope. See Piazza for the entry code.
  • Labs will be challenging and graded for correctness.
Course Outline
Week Date Lecture Topic Readings Paper Review (due 1pm) Labs/Projects
1 1/22 Class Organization & Introduction [slides.pdf] [slides.pptx]
2 1/27 Introduction to DNNs [slides.pdf] [slides.pptx] AlexNet, NeurIPS'2012 [Submit your review here.] Lab 1 (due 2/7)
1/29 DNN Quantization [slides.pdf] [slides.pptx] Integer-Arithmetic-Only Inference, CVPR'2018 [Submit your review here.]
3 2/3 Kernel Computation [slides.pdf] [slides.pptx] cuDNN, arXiv'2014 [Submit your review here.]
2/5 Dataflows [slides.pdf] [slides.pptx] Eyeriss, ISCA'2016 [Submit your review here.]
4 2/10 Guest Lecture: Efficient Neural Network Training and Inference, Amir Gholami [slides.pdf] HAWQ, ICCV'2019 Paper review is not required for guest lectures. Lab 2 (due 2/21)
2/12 DNN Accelerators [slides.pdf] [slides.pptx] TPU, ISCA'2017 [Submit your review here.]
5 2/17 Presidents' Day. No Lecture.
2/19 Guest Lecture: Accelerating Inference for Deep Neural Network, Yu-Hsin Chen [slides.pdf] Eyeriss v2, IEEE JETCAS'2019 Paper review is not required for guest lectures.
6 2/24 Mapping [slides.pdf] [slides.pptx] PHiPAC'ICS1997 [Submit your review here.] Lab 3 (due 3/6)
2/26 Chipyard and FireSim Tutorial [slides.pdf] FireSim, ISCA'2018 [Submit your review here.]
7 3/2 Guest Lecture: Accelerating the Pace of AWS Inferentia Chip Development: From Concept to End Customers Use, Randy Huang, Ron Diamant [slides.pdf]
3/4 Data Orchestration [slides.pdf] [slides.pptx] Decoupled Access-Execute, SIGARCH'1982 [Submit your review here.]
8 3/9 Sparsity [slides.pdf] [slides.pptx] SCNN, ISCA'2017 [Submit your review here.] Projects released.
3/11 Guest Lecture: AI at Facebook Datacenter Scale, Misha Smelyanskiy [slides.pdf] DL Inference at Facebook, arXiv'2018 Paper review is not required for guest lectures.
9 3/16 HW-SW Co-Design [slides.pdf] [slides.pptx] NetAdapt, ECCV'2018 [Submit your review here.]
3/18 Other Operators & Near-Data Processing [slides.pdf] [slides.pptx] Architectural Implications of DLRM, HPCA'2019 [Submit your review here.] Project Proposal (due 3/22)
10 3/23 Spring recess. No Lecture.
3/25 Spring recess. No Lecture.
11 3/30 Guest Lecture: Eric Chung, Microsoft, Project Brainwave: An AI Supercomputer for Real-time Inferencing at Cloud Scale [slides.pdf] Brainwave, ISCA'2018 Paper review is not required for guest lectures.
4/1 Guest Lecture: Vijay Janapa Reddi, Harvard, The Vision Behind MLPerf [slides.pdf] MLPerf Inference, arXiv'2019 Paper review is not required for guest lectures.
12 4/6 Guest Lecture: Cliff Young, Google, Neural Networks Have Rebooted Computer Architecture: What Should We Reboot Next? [slides.pdf] ParaDnn, arXiv'2019 Paper review is not required for guest lectures.
4/8 Training [slides.pdf] [slides.pptx] vDNN, MICRO'2016 [Submit your review here.] Project checkpoint 1 (4/10)
13 4/13 Guest Lecture: Dave Fick, Mythic, Analog Compute-in-Memory at Mythic [slides.pdf] Mythic, Microprocessor Report'2018 Paper review is not required for guest lectures.
4/15 Advanced Technology [slides.pdf] [slides.pptx] In-memory Computing, IEEE SSC-M'2019 [Submit your review here.]
14 4/20 Guest Lecture: Brian Zimmer, NVIDIA, Problems Facing Analog and In-memory Computing [slides.pdf] Error in Analog/Mixed-Signal HW, DAC'2019 Paper review is not required for guest lectures.
4/22 Accelerator-Level Parallelism [slides.pdf] [slides.pptx] Gables, HPCA'2019 [Submit your review here.] Project checkpoint 2 (4/24)
15 4/27 End-to-end Deployment [slides.pdf] [slides.pptx] QoS for Accelerated ML, HPCA'2019 [Submit your review here.]
4/29 Conclusion [slides.pdf] [slides.pptx]
16 5/4 Poster session Project final report (due 5/8)
Resources
Textbooks/Courses
Honor Code
  • If you turn in someone else’s work as if it were your own, you are guilty of cheating. This includes problem sets, answers on exams, lab exercise checks, project design, and any required course turn-in material.
  • Also, if you knowingly aid in cheating, you are guilty.
  • We have software that compares your submitted work to others.
  • However, it is okay to discuss with others lab exercises and the project (obviously, okay to work with project partner). Okay to discuss homework with others. But everyone must turn in their own work.
  • Do not post your work on public repositories like github (private o.k.)
  • If we catch you cheating, you will get negative points on the assignment: It is better to not do the work than to cheat! 
If it is a midterm exam, final exam, or final project, you get an F in the class. All cases of cheating reported to the office of student conduct.