CS292F (Spring 2020) Convex Optimization

Syllabus
[
link
]
Instructor:
Prof. YuXiang Wang
TA:
Kaiqi Zhang
Lecture Section: Monday/Wednesday 1:002:40 pm Location: on
Zoom (Password sent to you via email.)
Piazza: piazza.com/ucsb/winter2020/cs292f/home
Piazza is our main channel of communication. Questions should be posted here.
Gradescope: https://www.gradescope.com/courses/109563
This is where you submit your homeworks and reading summaries.
Office hours:
Instructor: by appointment.
TA: Kaiqi Zhang 46 pm every Friday on Zoom:
[ Zoom link]
You will need a password from your email sent by Kaiqi.
Course evaluation:
80% Homework, 15% Reading Notes, 5% Participation. Bonus 5% for scribing. There are more bonus points in each homework.
Scribing:
Please volunteer here, use this latex template
Course Schedule / Scribed Notes
=
Week 
Date 
Topic 
Reading 
Assignment 
Notes 
Scribe 
Lecture Video 
1 
30Mar 
Intro + Convex Set and Convex Function 
BV Ch.1, Ch.2, Ch.3 
HW1 out 

[Scribe 1, latex] 
Video 

1Apr 
Convex Optimization Basics 
BV Ch. 4.14.2 

Notes 
[Scribe 2, latex] 
Video 
2 
6Apr 
Canonical problem forms 
BV Ch 4.34.7 

Notes 
[Scribe 3, latex] 
Video 

8Apr 
Gradient Descent 
BV Ch 9.19.4 

Notes 
[Scribe 4, latex] 
Video 
3 
13Apr 
Subgradient and subdifferential 
Boyd's subgradient notes 
HW2 out /
HW1 Due 
Notes 
[Scribe 5, latex] 
Video 

15Apr 
Subgradient method and proximal gradient descent (Part I) 
Boyd's subgradient method notes 

Notes 
[Scribe 6, latex] 
Video 
4 
20Apr 
Proximal Gradient Descent (Part II)

Section 14 of Parikh and Boyd
)


Notes

[Scribe 7, latex] 
Video 

22Apr 
Stochastic (sub)gradient methods

Section 15 of Boyd's SGD notes 

Notes 

Video 
5 
27Apr 
Duality

Lecture 11 and 12 of CMU 10725 
HW3 out / HW2 due. 

[Scribe 9] 
Video 

29Apr 
KKT conditions and its usage

Lecture 13 and 14 of of CMU 10725 


[Scribe 10, latex] 
Video 
6 
4May 
Newton's method

BV Ch 9 and 10 

Notes 

Video 

6May 
Interior point methods 
BV Ch 11, Nemirovski Ch 2 

Notes 
[Scribe 12, latex] 
Video 
7 
11May 
Intro to online learning: Learning from expert advice

Hazan Ch 1 


[Notes on OCO intro, latex] 
Video 

13May 
Online (Projected) Gradient Descent

Hazan Ch 3 
HW4 Out / HW3 Due 

[Scribe 14, latex] 
Video 
8 
18May 
Follow the Regularized Leader

Hazan Ch 5 



Video 

20May 
ExponentialConcavity and Online Newton Method

Hazan Ch 4



[Scribe 16, latex] 
Video 
9 
25May 
No class. Memorial day.






27May 
Modern Stochastic Gradient Methods

[Johnson and Zhang (2013), Ghadimi and Lan (2013) 

Notes 

Video 
10 
1Jun 
Alternating Direction Method of Multipliers

[ Ramdas and Tibshirani, Candes et al.] 
HW#4 due 


Video 

3Jun 
Adaptive Online Learning: Dynamic Regret and Adaptive Regret. 
Zinkevich (2003) Besbes et al (2013) 


[Scribe 19, latex] 
Video 

3Jun 
Not covered: Bandits. 
Hazan Ch. 6 


[Scribed notes for bandits, latex] [More scribed notes for bandits, latex] 