Usability of Programming Languages

CSE 291I, Fall 2022

Programmers of all kinds express their ideas using programming languages. Unfortunately, languages can be hard to use correctly, resulting in lengthy development times and buggy software. How can these languages be designed to make programmers as effective as possible?

In this course, we will learn techniques to analyze and improve the usability of programming languages. Students will apply these techniques to languages of current and historical interest, and in the process, expand their knowledge of different ways to design languages. This course is intended as preparation for conducting independent research on the usability of programming languages, and will include homework assignments as well as a project.



Michael Coblenz, Assistant Professor

Office hours: Mondays 4-5 PM, CSE 3246


Dylan Lukes, PhD Student

Office hours: Thursdays, 2-4 PM, CSE B215

Course Learning Outcomes

Some students in the course are research-focused, and are primarily interested in learning how to do research on usability of programming languages. For you, my goal is to prepare you to do high-quality, impactful research. Others are practice-focused, and are primarily interested in future careers in the software industry. For you, I hope to provide insight and background that will help you make wise choices (which language or tools should my team use for this project?); and to read and critically analyze scientific literature in this area so you are ready to take advantage of future discoveries and innovations. Both kinds of students are valued participants in the class.

Upon completion of this course, students will be able to:

  1. Apply qualitative and quantitative research methods to obtain insights about programming language design choices;
  2. Critically analyze design questions in the context of languages and programming systems;
  3. Read and interpret research papers in the area of usability of programming languages, and summarize major findings to date.

Course Format

The course will be face-to-face. We will follow the UC San Diego policies regarding masking; currently, masking is required in all classroom settings.

The course is about using research methods to study the impact of programming language design decisions on people. Therefore, the course is divided into three main segments.

  • First, we will study how to use methods from human-computer interaction research to obtain insights about the effects of language design decisions on programmers and software engineers. During this time, the homework will focus on reading papers and book chapters. To encourage thorough study of the readings, the readings will be accompanied by reading responses, which must be submitted before we discuss the readings in class.
  • Next, we will briefly introduce two new programming languages with which most students will be unfamiliar. There are two goals for this: first, to broaden your perspectives on what programming could be like; second, to provide inspiration for your project. You will have an opportunity to practice programming in each language in homework assignments.
  • In the third segment of the class, we will study results of applying these and other methods to familiarize ourselves with what is already known about the usability of programming languages. During this time, you will work with a partner to conduct a preliminary research project according to your own interests. You will present your results to the class at the end of the quarter.

I have made readings optional in cases where I expect to cover the material thoroughly in class. Those readings might be used for exam study, for reference while doing homework, or to provide an alternative explanation of the material when needed.

The class will have a midterm exam covering the research methods taught in the class. There will be no final exam; instead, students will submit reports on their project results.


Some readings may be from Research Methods in Human-Computer Interaction by Lazar, Feng, and Hochheiser. The full text of the book is available for free via the UCSD library. A copy is available on course reserves at the library for students who prefer the paper version. This book may be a useful reference as you work on your assignments and projects and as you study for the midterm exam.

You may wonder why my PLIERS paper is not on the reading list. The answer is that the entire course is about PLIERS! You may find it useful, however, as reference material.


Assignment Weight Due Date
Project 35%  
Programming assignments 10%  
Reading responses 25% Before lecture
Qualitative study assignment 10% 10/12
Methods Exam 20% 10/28 in class


The project represents an opportunity to investigate a research question of your choice using one or more research methods discussed in class. The research question should be relevant to programming languages, broadly construed. This means that the question might pertain only to a language, or it might pertain to a combination of a programming language and environment (e.g., an IDE), or it might focus on the IDE itself.

Once you have identified the research question, you should plan how you will gather data to address your question: what artifacts or processes will you observe? How will you study those artifacts or processes in a structured way? Then, you should have a plan for analyzing the data and drawing conclusions.

Grading for the project, which overall represents 35% of the grade, will be broken down as follows. The proposal and plan are identical in structure; the idea is that the proposal is a low-stakes opportunity to receive feedback on the plan before executing it.

Component Weight Due Date
Project pitch slides 5% 10/24
Project pitch presentation 5% 10/24
Project plan with one task specified; other tasks outlined 25% 11/4
Checkpoint: experience report from one pilot and revised materials 15% 11/18
Project report 25% 12/2
Project presentation 25% 11/30 (presentations will take place 11/30 and 12/2)


All assignments may be found and submitted on Canvas.

Course Schedule

Project/Homework due Slides  
9/22 Introduction to usability: definition, methods overview. (none)   PDF  
9/26 No class (instructor out)        
9/28 Research methods intro Andreas Stefik and Stefan Hanenberg. 2014. The Programming Language Wars: Questions and Responsibilities for the Programming Language Community. In Onward! 2014. DOI. Optional reading: Chapter 10 of “Research Methods in HCI.”   PDF  
9/30 Designing and running usability studies Brad A. Myers, Amy J. Ko, Thomas D. LaToza, YoungSeok Yoon. Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools. PDF   PDF  
10/3 Qualitative methods: interviews and surveys PDF      
10/5 Guest lecture, Prof. William Griswold Optional readings: Getting Started with AspectJ; An Overview of AspectJ.   PDF  
10/7 Qualitative methods: thematic analysis, grounded theory Optional reading: Braun and Clarke. Using Thematic Analysis in Psychology   PDF  
10/10 Qualitative methods example Justin Lubin and Sarah E. Chasins. 2021. How statically-typed functional programmers write code. OOPSLA. DOI Group matching survey PDF  
10/12 Programming in Rust   Teams announced (Credit: Michael Hicks) Rust introduction  
10/14 Programming in Rust     (Credit: Michael Hicks) Ownership  
10/17 Programming in Rust   Qualitative methods assignment (Credit: Michael Hicks) Collections, Structs/enums  
10/19 Programming in Rust     (Credit: Michael Hicks) Traits  
10/21 Programming in Rust Optional: History of Rust video   (Credit: Michael Hicks) Boxes, Interior Mutability  
10/24 Project pitch presentations, day 1   Project pitch (slides)    
10/26 Project pitch presentations, day 2        
10/28 Research methods midterm        
10/31 Quantitative methods Optional: Amy J. Ko, Thomas D. LaToza, and Margaret M. Burnett. A practical guide to controlled experiments of software engineering tools with human participants. Empirical Software Engineering. PDF Rust programming assignment PDF  
11/2 Cognitive Dimensions of Notations Thomas R. G. Green and Marian Petre. 1996. Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. Journal of Visual Languages & Computing 7, 2 (1996), 131–174.   PDF  
11/4 Gender HCI: programming tools and gender Margaret Burnett, Anicia Peters, Charles Hill, and Noha Elarief. Finding Gender-Inclusiveness Software Issues with GenderMag: A Field Investigation. CHI 2016. PDF   PDF  
11/7 Discuss and give feedback on group projects        
11/9 Argument type checking RCT L. Prechelt and W. F. Tichy, “A controlled experiment to assess the benefits of procedure argument type checking,” TSE. Apr. 1998, IEEE Final project plan PDF  
11/11 Veteran’s Day (no class)        
11/14 Rust Michael Coblenz, Michelle L. Mazurek, and Michael Hicks. 2022. Garbage collection makes Rust easier to use: a randomized controlled trial of the Bronze garbage collector. ICSE 2022. DOI   PDF  
11/16 Natural programming Brad A. Myers, John F. Pane, and Andy Ko. 2004. Natural programming languages and environments. CACM. DOI   PDF  
11/18 Static vs. Dynamic types Endrikat, Hanenberg, Robbes, Stefik, “How do API documentation and static typing affect API usability?”, ICSE 2014. ACM Project checkpoint PDF  
11/21 Theory No reading; focus on project   PDF  
11/23 Corpus Studies     PDF  
11/25 No class: Thanksgiving        
11/28 PLIERS: putting it all together        
11/30 Project presentations   Project presentations (split class in two groups)    
12/2 Project presentations   Project report