CSE 203B, Winter 2024 Convex Optimization

University of California, San Diego

Instructor (Office hours TBA in Piazza)

• CK Cheng, room CSE2130, email: ckcheng+203B@ucsd.edu, tel: 858 534-6184

Teaching Assistant (Office hours TBA in Piazza)

• Gupta, Aayush, email:aag011@ucsd.edu
• Koga, Tatsuki, email:tkoga@ucsd.edu
• Li, Albert, email:ajl015@ucsd.edu (Lead TA)
• Mohan, Dhruthick Gowda, email:dgmohan@ucsd.edu
• Mokashi, Nachiket Ajay, email:nmokashi@ucsd.edu
• Yue, Yang, email:yayue@ucsd.edu

Class Platform

• Canvas
• Gradescope
• Piazza
• UCSD Podcast of lectures and discussion sessions

Schedule

• Lectures: 2:00-3:20PM TTH, CTL(Catalyst) 125
• Discussion: 8:00-8:50AM F, SOLIS 107

References

• Convex Optimization, S. Boyd and L. Vandenberghe, Cambridge, 2004 (required textbook).
• Numerical Recipes: The Art of Scientific Computing, Third Edition, W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Cambridge University Press, 2007 (Recomended Reference)
• Matrix Computations, 4th Edition, G.H. Golub and C.F. Van Loan, Johns Hopkins, 2013 (Rec mmended Reference)
• Convex Optimization by R. Tibshirani, http://www.stat.cmu.edu/~ryantibs/convexopt/ (related resources)
• EE364a: Convex Optimization I, http://stanford.edu/class/ee364a/ (related resources)
• https://cseweb.ucsd.edu/classes/cse203B-a (CSE203B notes in Winter 2023)
• https://cseweb.ucsd.edu/~kuan/ (CK Cheng personal website)

Prerequisite

Linear algebra and basic knowledge of numerical methods, or intention of conducting projects related to scientific computation.

Content

We study the formulations and algorithms for solving convex optimization problems. The topics include convex sets, functions, optimality conditions, and duality concepts. If time permits, we will talk about gradient descent, conjugate gradient, interior-point methods, and applications. The objective of the course is to provide students with the background and techniques for scientific computing and system optimization.

Lectures

• Part I: Theory
• Lecture 1 Introduction, Class Logistics, Reading assignment: Chapter 1, Lecture slides pptx file, pdf file, and high level introduction (Reference: Chapter 5): pptx file, pdf file.
• Lecture 2 Convex Sets, Reading assignment: Chapter 2, Lecture slides pptx file, pdf file.
• Lecture 3 Convex Functions, Reading assignment: Chapter 3, Lecture slides pptx file, pdf file.
• Lecture 4 Formula, Reading assignment: Chapter 4, Lecture slides pptx file, pdf file.
• Lecture 5 Duality, Reading assignment: Chapter 5, Lecture slides pptx file, pdf file.
• Midterm review, week7-B, no slides.
• Part II: Algorithms
• Lecture 9 Unconstrained Minimization, Reading assignment: Chapter 9, Lecture slides pptx file, pdf file.
• Lecture 10 Equality Constrained Minmization, Reading assignment: Chapter 10. Lecture slides pptx file, pdf file.
• Lecture 11 Interior Point Methods, Reading assignment: Chapter 11. Lecture slides pptx file, pdf file.

Homework: gradescope submission

• Homework 1, Due 1/19/2024, pdf file, latex file. Solution: Solution pdf file.
• Homework 2, Due 1/26/2024, pdf file, latex file. Solution: pdf file.
• Homework 3, Due date shifted to 2/9/2024, pdf file, tex file. Solution: pdf file.
• Homework 4, Due 2/23/2024, pdf file, tex file. Solution: pdf file.

Discussions

• W1 Discussion pdf file
• W2 Discussion pdf file
• W3 Discussion pdf file
• W4 Discussion pdf file
• W5 Discussion pdf file
• W6 Discussion pdf file
• W7 Discussion pdf file

Exam

• Take-home exam starting at 10AM on Sunday 2/25/2024, and ending at 10AM on Tuesday 2/27/2024 (W7-8), pdf file, tex file. Solution: pdf file.

Project

• Project outlines due 2/9/2024, outline format pdf file, pptx file (W5)
• Report due 6:00PM Th 3/21/2024, outlines and rubrics, pdf file, pptx file (W11)
• Exemplar Report, pdf file, Robust Principal Component Pursuit for Low-Rank Matrix Recovery, by N. Wahi, S. Peddabomma, P. Nagasamudra, S. Madhuvarusu.
• Emerging Topic Report, pdf file, Training Inference-friendly Large Language Model, by Y. Zhou, L. Zheng, Y. Yang, and L. Yun.