CS292F (Spring 2020) Convex Optimization

Syllabus [ link ]

Instructor: Prof. Yu-Xiang Wang

TA: Kaiqi Zhang

Lecture Section: Monday/Wednesday 1:00-2:40 pm Location: on Zoom (Password sent to you via email.)

Piazza: piazza.com/ucsb/winter2020/cs292f/home
Piazza is our main channel of communication. Questions should be posted here.

Gradescope: https://www.gradescope.com/courses/109563
This is where you submit your homeworks and reading summaries.

Office hours: Instructor: by appointment.
TA: Kaiqi Zhang 4-6 pm every Friday on Zoom: [ Zoom link] You will need a password from your email sent by Kaiqi.

Course evaluation: 80% Homework, 15% Reading Notes, 5% Participation. Bonus 5% for scribing. There are more bonus points in each homework.

Scribing: Please volunteer here, use this latex template

Course Schedule / Scribed Notes

=
Week Date Topic Reading Assignment Notes Scribe Lecture Video
1 30-Mar Intro + Convex Set and Convex Function BV Ch.1, Ch.2, Ch.3 HW1 out [Scribe 1, latex] Video
1-Apr Convex Optimization Basics BV Ch. 4.1-4.2 Notes [Scribe 2, latex] Video
2 6-Apr Canonical problem forms BV Ch 4.3-4.7 Notes [Scribe 3, latex] Video
8-Apr Gradient Descent BV Ch 9.1-9.4 Notes [Scribe 4, latex] Video
3 13-Apr Subgradient and subdifferential Boyd's subgradient notes HW2 out / HW1 Due Notes [Scribe 5, latex] Video
15-Apr Subgradient method and proximal gradient descent (Part I) Boyd's subgradient method notes Notes [Scribe 6, latex] Video
4 20-Apr Proximal Gradient Descent (Part II) Section 1-4 of Parikh and Boyd ) Notes [Scribe 7, latex] Video
22-Apr Stochastic (sub)gradient methods Section 1-5 of Boyd's SGD notes Notes Video
5 27-Apr Duality Lecture 11 and 12 of CMU 10-725 HW3 out / HW2 due. [Scribe 9] Video
29-Apr KKT conditions and its usage Lecture 13 and 14 of of CMU 10-725 [Scribe 10, latex] Video
6 4-May Newton's method BV Ch 9 and 10 Notes Video
6-May Interior point methods BV Ch 11, Nemirovski Ch 2 Notes [Scribe 12, latex] Video
7 11-May Intro to online learning: Learning from expert advice Hazan Ch 1 [Notes on OCO intro, latex] Video
13-May Online (Projected) Gradient Descent Hazan Ch 3 HW4 Out / HW3 Due [Scribe 14, latex] Video
8 18-May Follow the Regularized Leader Hazan Ch 5 Video
20-May Exponential-Concavity and Online Newton Method Hazan Ch 4 [Scribe 16, latex] Video
9 25-May No class. Memorial day.
27-May Modern Stochastic Gradient Methods [Johnson and Zhang (2013), Ghadimi and Lan (2013) Notes Video
10 1-Jun Alternating Direction Method of Multipliers [ Ramdas and Tibshirani, Candes et al.] HW#4 due Video
3-Jun Adaptive Online Learning: Dynamic Regret and Adaptive Regret. Zinkevich (2003) Besbes et al (2013) [Scribe 19, latex] Video
3-Jun Not covered: Bandits. Hazan Ch. 6 [Scribed notes for bandits, latex] [More scribed notes for bandits, latex]