CSE 203B, Winter 2023 Convex Optimization

University of California, San Diego

Instructor (Office hours TBA in Piazza)

• CK Cheng, room CSE2130, email: ckcheng+203B@ucsd.edu, tel: 858 534-6184

Teaching Assistant (Office hours TBA in Piazza)

• Chen, Danlu, email:dac013@ucsd.edu
• Giri, Vijay, email:vgiri@ucsd.edu
• Holtz, Chester, email:chholtz@ucsd.edu (Lead TA)
• Magee, Lucas, email:lmagee@ucsd.edu
• Singh, Abhishek, email:abs006@ucsd.edu
• Song, Meng, email:mes050@ucsd.edu

Class Platform

• Canvas
• Gradescope
• Piazza
• UCSD Podcast of lectures and discussion sessions

Schedule

• Lectures: 12:30-1:50PM TTH, SOLIS 107
• Discussion: 4:00-4:50PM F, WLH 2001

References

• Convex Optimization, S. Boyd and L. Vandenberghe, Cambridge, 2004 (required textbook).
• Numerical Recipes: The Art of Scientific Computing, Third Edition, W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Cambridge University Press, 2007 (Recomended Reference)
• Matrix Computations, 4th Edition, G.H. Golub and C.F. Van Loan, Johns Hopkins, 2013 (Recommended Reference)
• Convex Optimization by R. Tibshirani, http://www.stat.cmu.edu/~ryantibs/convexopt/ (related resources)
• EE364a: Convex Optimization I, http://stanford.edu/class/ee364a/ (related resources)
• https://cseweb.ucsd.edu/~kuan/ (CSE203B notes in previous quarters)

Prerequisite

Linear algebra and basic knowledge of numerical methods, or intention of conducting projects related to scientific computation.

Content

We study the formulations and algorithms solving convex optimization problems. The topics include convex sets, functions, optimality conditions, duality concepts. If time permits, we will talk about gradient descent, conjugate gradient, interior-point methods, and applications. The objective of the course is to provide students the background and techniques for scientific computing and system optimization.

Lectures

• Part I: Theory
• Lecture 1 Introduction, Class Logistics, Reading assignment: Chapter 1, Lecture slides pptx, pdf.
• Lecture 2 Convex Sets, Reading assignment: Chapter 2, Lecture slides pptx, pdf.
• Lecture 3 Convex Functions, Reading assignment: Chapter 3, Lecture slides pptx, pdf.
• Lecture 4 Formula, Reading assignment: Chapter 4, Lecture slides pptx, pdf, slides with notes Week-4A pdf, slides with notes Week-4B pdf, slides with notes end of Week 4/start of Week 5 pdf.
• Lecture 5 Duality, Reading assignment: Chapter 5, Lecture slides pptx, pdf, Notes week-5 pdf, Notes2 week-5 pdf,
• Midterm review, week7-B pdf.
• Part II: Algorithms
• Lecture 9 Unconstrained Minimization, Reading assignment: Chapter 9, Lecture slides pptx, pdf.
• Lecture 10 Equality Constrained Minmization, Reading assignment: Chapter 10. Lecture slides pptx, pdf.
• Lecture 11 Interior Point Methods, Reading assignment: Chapter 11. Lecture slides pptx, pdf.

Homework: gradescope submission

• Homework 1, Due 1/18/2023, pdf, tex. Solution: pdf.
• Homework 2, Due 1/25/2023, pdf, tex. Solution: pdf.
• Homework 3, Due date shifted to 2/8/2023, note that HW was revised 1/30/2023 pdf, tex. Solution: pdf.
• Homework 4, Due 2/22/2023, pdf, tex. Solution: pdf.

Discussions

• W1 Discussion (Slides)
• W2 Discussion (Slides)
• W3 Discussion (Slides)
• W4 Discussion (Slides)
• W5 Discussion (Slides)
• W6 Discussion (Slides)
• W7 Discussion (Midterm review slides)

Exam

• Take-home exam starting at 10AM on Sunday 2/26, and ending at 10AM on Tuesday 2/28 (W7-8), pdf, tex. Solution: pdf.

Project

• Project outlines due 2/1/2023, outline format pdf, pptx (W4)
• Report due 230PM Tu 3/21/2023, outlines and rubrics, pdf, pptx (W11)
• Two prototypes out of 53 reports
• 1. Towards Mixed-Integer Programming for Strategy Formulation: A Study in Convex Optimization Under Quasiconvex Constraints for Pokemon, by Hailey James, Avni Kothar, and Hayden McTavish, pdf file
• 2. A Scalable and Modular Framework for Training Provably Robust Large Transformer Models via Neural Network Duality, by Andre (Jianyou) Wang, Yongce Li, Weili Cao, and Yang Yue, pdf file