NSF CAREER: Exact Optimal and Data-Adaptive Algorithms and Tools for Differential Privacy
Principal Investigator
Yu-Xiang Wang, UCSB (2021 - 2024), UCSD (2024 -)		  
Project Summary
Products

Funded by NSF CNS 2048091.

This material is based upon work supported by the National Science Foundation under Grant No. 2048091. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Project Summary

This project is motivated by the increasing public concerns on privacy issues, new legislations and the high demand for privacy enhancing technologies such as differential privacy (DP) in applications from both private and public sectors. The overarching theme of the project is to address the pressing new challenges that arise as differential privacy transforms from a theoretical construct into a practical technology. The project advances the state-of-the-art of research in the area of DP, and contributes to privacy education. On the research front, the project develops new algorithms and analytical tools that enable more precise privacy accounting and higher utility in DP. On the education front, the project involves training future leaders in DP areas, creating educational materials and expanding an open-source software library called autodp that makes state-of-the-art differentially private computation more accessible. Collectively, the integrated research and educational activities contribute to ongoing collaborative efforts in building innovative applications of differential privacy.

Fig 1. The exact optimal DP accouinting involves describing a DP mechanism by a function.

Talks and media coverages

  1. Paper "Neural Collapse Meets Differential Privacy" is selected for an Oral Presentation at ICML 2024
    Among the top 0.3% of all submissions.
  2. Invited talk on "Open Problem: Near Linear time algorithm for DP-ERM"
    PI Wang spoke at TTIC Federated Learning Workshop, 2023
  3. Invited talk on "How to Squeeze out the Last Drop of Utility in your DP application"
    PI Wang spoke at TTIC Federated Learning Workshop, 2023
  4. Invited talk on "Towards Practical Differential Privacy"
    PI Wang spoke at Nokia Bell Lab, UCSD, Princeton in 2023
  5. Keynote talk on "Practical Differential Privacy"
    PI Wang spoke at NeurIPS'22 Workshop on "Challenges in Deploying and Monitoring Machine Learning Systems"
  6. Invited talk on "Towards Practical Differential Privacy with Differential Privacy-like ideas"
    PI Wang spoke at Google Research Workshop on "Security, Privacy, and Anti-Abuse Research Workshop"
  7. Invited talk on "Optimal Accounting of Differential Privacy"
    PI Wang spoke at MIT Media Lab and LinkedIn Data Science Seminar Series
  8. Invited talk on "Optimal Accounting of Differential Privacy via Characteristic Function"
    PI Wang spoke at Rutgers. Jinshuo and Yuqing spoke at Google.
  9. Invited talk on "Per-instance Differential Privacy and How to Privately Publish Them"
    PI Wang spoke at Google and at University of Albany
  10. Invited talk on "Advances in Differential Privacy and Private Federated Learning"
    PI Wang spoke at Amazon Web Services.
  11. News article "A Matter of Privacy"
    Coverage on this project on UCSB Magazine "The Current" [Link]
  12. Invited talk on "Privacy Amplification by Sampling and Renyi Differential Privacy"
    PI Wang spoke at Berkeley Simons Institute [Link to the talk]

Research Results

  1. Neural Collapse meets Differential Privacy: Curious behaviors of NoisyGD with Near-Perfect Representation Learning
    Chendi Wang, Yuqing Zhu, Weijie Su, Yu-Xiang Wang
    ICML 2024. [arxiv] (*Oral Presentation)
  2. Privacy Profiles for Private Selection
    Antti Koskela, Rachel Redberg, Yu-Xiang Wang.
    ICML 2024. [arxiv]
  3. Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy
    Yingyu Lin, Yi-An Ma, Yu-Xiang Wang, Rachel Redberg.
    ICLR 2024. [arxiv]
  4. Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deployment
    Participants of 2022 June Workshop "Differential Privacy: Challenges Towards The Next Fronter".
    Harvard Data Science Review [paper, arxiv]
  5. Improving the Privacy and Practicality of Objective Perturbation for Differentially Private Linear Learners
    Rachel Redberg, Antti Koskela, Yu-Xiang Wang.
    NeurIPS 2023. [arxiv]
  6. Threshold KNN-Shapley: A Linear-Time and Privacy-Friendly Approach to Data Valuation
    Jiachen T. Wang, Yuqing Zhu, Yu-Xiang Wang, Ruoxi Jia, Prateek Mittal.
    NeurIPS 2023. [arxiv] (*Spotlight Presentation)
  7. Offline Reinforcement Learning with Differential Privacy
    Dan Qiao, Yu-Xiang Wang.
    NeurIPS 2023. [arxiv]
  8. "Private Prediction Strikes back!" Private Kernelized Nearest Neighbors with Individual Renyi Filter
    Yuqing Zhu, Xuandong Zhao, Chuan Guo, Yu-Xiang Wang.
    UAI 2023. [arxiv] (*Spotlight Presentation)
  9. Near-Optimal Differentially Private Reinforcement Learning
    Dan Qiao, Yu-Xiang Wang.
    AISTATS 2023 [arxiv]
  10. Generalized PTR: User-Friendly Recipes for Data-Adaptive Algorithms with Differential Privacy
    Rachel Redberg, Yuqing Zhu, Yu-Xiang Wang.
    AISTATS 2023. [arxiv] (*Plenary Oral Presentation)
  11. Differentially Private Linear Sketches: Efficient Implementations and Applications
    Fuheng Zhao, Dan Qiao, Rachel Redberg, Divyakant Agrawal, Amr El Abbadi, Yu-Xiang Wang.
    NeurIPS 2022. [arxiv]
  12. SeqPATE: Differentially Private Text Generation via Knowledge Distillation
    Zhiliang Tian, Yingxiu Zhao, Ziyue Huang, Yu-Xiang Wang, Nevin Zhang, He He
    NeurIPS 2022. [openreview]
  13. Provably Confidential Language Modelling
    Xuandong Zhao, Lei Li, Yu-Xiang Wang.
    NAACL 2022. [arxiv] (*Oral Presentation)
  14. Adaptive Private-K-Selection with Adaptive K and Application to Multi-label PATE
    Yuqing Zhu, Yu-Xiang Wang.
    AISTATS 2022. [arxiv]
  15. Optimal Accounting of Differential Privacy via Characteristic Function
    Yuqing Zhu, Jinshuo Dong, Yu-Xiang Wang.
    AISTATS 2022. [arxiv]
  16. Privately Publishable Per-instance Privacy
    Rachel Redberg, Yu-Xiang Wang.
    NeurIPS 2021. [arxiv]
  17. Revisiting Model-Agnostic Private Learning: Faster Rates and Active Learning
    Chong Liu, Yuqing Zhu, Kamalika Chaudhuri, Yu-Xiang Wang
    Journal of Machine Learning Research. Shorter version appeared in AISTATS 2021. [arxiv]
  18. Voting-based Approaches For Differentially Private Federated Learning
    Yuqing Zhu, Xiang Yu, Yi-Hsuan Tsai, Francesco Pittaluga, Masoud Faraki, Manmohan chandraker and Yu-Xiang Wang
    Manuscript. [arxiv]
  19. Subsampled Renyi Differential Privacy and Analytical Moments Accountant
    Yu-Xiang Wang, Borja Balle, Shiva Kasiviswanathan
    Journal of Privacy and Confidentiality, 2021 [ paper ]. A shorter version appeared at AISTATS 2019 and received a Notable Paper Award.

Educational materials

  1. CS291A Introduction to Differential Privacy: Theory, Algorithms and Applications [Course website]
    Instructor: Yu-Xiang Wang, 2021 Fall
  2. [Open source project] New API / examples / tutorials / unit tests for autodp
    Contributors: Yu-Xiang Wang, Yuqing Zhu, Borja Balle (DeepMind), Stefan Mallem (Google)
  3. Guest Lecture on "Composition, RDP and Amplification by Sampling"
    at Mijung Park's CPSC 532P at University of British Columbia [Course website]

Fig 2. An illustration of the new API of autodp.