Usability of Programming Languages
CSE 291I, Fall 2022
Programmers of all kinds express their ideas using programming languages. Unfortunately, languages can be hard to use correctly, resulting in lengthy development times and buggy software. How can these languages be designed to make programmers as effective as possible?
In this course, we will learn techniques to analyze and improve the usability of programming languages. Students will apply these techniques to languages of current and historical interest, and in the process, expand their knowledge of different ways to design languages. This course is intended as preparation for conducting independent research on the usability of programming languages, and will include homework assignments as well as a project.
Michael Coblenz, Assistant Professor
Office hours: Mondays 4-5 PM, CSE 3246
Dylan Lukes, PhD Student
Office hours: Thursdays, 2-4 PM, CSE B215
Course Learning Outcomes
Some students in the course are research-focused, and are primarily interested in learning how to do research on usability of programming languages. For you, my goal is to prepare you to do high-quality, impactful research. Others are practice-focused, and are primarily interested in future careers in the software industry. For you, I hope to provide insight and background that will help you make wise choices (which language or tools should my team use for this project?); and to read and critically analyze scientific literature in this area so you are ready to take advantage of future discoveries and innovations. Both kinds of students are valued participants in the class.
Upon completion of this course, students will be able to:
- Apply qualitative and quantitative research methods to obtain insights about programming language design choices;
- Critically analyze design questions in the context of languages and programming systems;
- Read and interpret research papers in the area of usability of programming languages, and summarize major findings to date.
The course will be face-to-face. We will follow the UC San Diego policies regarding masking; currently, masking is required in all classroom settings.
The course is about using research methods to study the impact of programming language design decisions on people. Therefore, the course is divided into three main segments.
- First, we will study how to use methods from human-computer interaction research to obtain insights about the effects of language design decisions on programmers and software engineers. During this time, the homework will focus on reading papers and book chapters. To encourage thorough study of the readings, the readings will be accompanied by reading responses, which must be submitted before we discuss the readings in class.
- Next, we will briefly introduce two new programming languages with which most students will be unfamiliar. There are two goals for this: first, to broaden your perspectives on what programming could be like; second, to provide inspiration for your project. You will have an opportunity to practice programming in each language in homework assignments.
- In the third segment of the class, we will study results of applying these and other methods to familiarize ourselves with what is already known about the usability of programming languages. During this time, you will work with a partner to conduct a preliminary research project according to your own interests. You will present your results to the class at the end of the quarter.
I have made readings optional in cases where I expect to cover the material thoroughly in class. Those readings might be used for exam study, for reference while doing homework, or to provide an alternative explanation of the material when needed.
The class will have a midterm exam covering the research methods taught in the class. There will be no final exam; instead, students will submit reports on their project results.
Some readings may be from Research Methods in Human-Computer Interaction by Lazar, Feng, and Hochheiser. The full text of the book is available for free via the UCSD library. A copy is available on course reserves at the library for students who prefer the paper version. This book may be a useful reference as you work on your assignments and projects and as you study for the midterm exam.
You may wonder why my PLIERS paper is not on the reading list. The answer is that the entire course is about PLIERS! You may find it useful, however, as reference material.
|Reading responses||25%||Before lecture|
|Qualitative study assignment||10%||10/12|
|Methods Exam||20%||10/28 in class|
The project represents an opportunity to investigate a research question of your choice using one or more research methods discussed in class. The research question should be relevant to programming languages, broadly construed. This means that the question might pertain only to a language, or it might pertain to a combination of a programming language and environment (e.g., an IDE), or it might focus on the IDE itself.
Once you have identified the research question, you should plan how you will gather data to address your question: what artifacts or processes will you observe? How will you study those artifacts or processes in a structured way? Then, you should have a plan for analyzing the data and drawing conclusions.
Grading for the project, which overall represents 35% of the grade, will be broken down as follows. The proposal and plan are identical in structure; the idea is that the proposal is a low-stakes opportunity to receive feedback on the plan before executing it.
|Project pitch slides||5%||10/24|
|Project pitch presentation||5%||10/24|
|Project plan with one task specified; other tasks outlined||25%||11/4|
|Checkpoint: experience report from one pilot and revised materials||15%||11/18|
|Project presentation||25%||11/30 (presentations will take place 11/30 and 12/2)|
All assignments may be found and submitted on Canvas.
|9/22||Introduction to usability: definition, methods overview.||(none)|
|9/26||No class (instructor out)|
|9/28||Research methods intro||Andreas Stefik and Stefan Hanenberg. 2014. The Programming Language Wars: Questions and Responsibilities for the Programming Language Community. In Onward! 2014. DOI. Optional reading: Chapter 10 of “Research Methods in HCI.”|
|9/30||Designing and running usability studies||Brad A. Myers, Amy J. Ko, Thomas D. LaToza, YoungSeok Yoon. Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools. PDF|
|10/3||Qualitative methods: interviews and surveys|
|10/5||Guest lecture, Prof. William Griswold||Optional readings: Getting Started with AspectJ; An Overview of AspectJ.|
|10/7||Qualitative methods: thematic analysis, grounded theory||Optional reading: Braun and Clarke. Using Thematic Analysis in Psychology|
|10/10||Qualitative methods example||Justin Lubin and Sarah E. Chasins. 2021. How statically-typed functional programmers write code. OOPSLA. DOI||Group matching survey|
|10/12||Programming in Rust||Teams announced||(Credit: Michael Hicks) Rust introduction|
|10/14||Programming in Rust||(Credit: Michael Hicks) Ownership|
|10/17||Programming in Rust||Qualitative methods assignment||(Credit: Michael Hicks) Collections, Structs/enums|
|10/19||Programming in Rust||(Credit: Michael Hicks) Traits|
|10/21||Programming in Rust||Optional: History of Rust video||(Credit: Michael Hicks) Boxes, Interior Mutability|
|10/24||Project pitch presentations, day 1||Project pitch (slides)|
|10/26||Project pitch presentations, day 2|
|10/28||Research methods midterm|
|10/31||Quantitative methods||Optional: Amy J. Ko, Thomas D. LaToza, and Margaret M. Burnett. A practical guide to controlled experiments of software engineering tools with human participants. Empirical Software Engineering. PDF||Rust programming assignment|
|11/2||Cognitive Dimensions of Notations||Thomas R. G. Green and Marian Petre. 1996. Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. Journal of Visual Languages & Computing 7, 2 (1996), 131–174.|
|11/4||Gender HCI: programming tools and gender||Margaret Burnett, Anicia Peters, Charles Hill, and Noha Elarief. Finding Gender-Inclusiveness Software Issues with GenderMag: A Field Investigation. CHI 2016. PDF|
|11/7||Discuss and give feedback on group projects|
|11/9||Argument type checking RCT||L. Prechelt and W. F. Tichy, “A controlled experiment to assess the benefits of procedure argument type checking,” TSE. Apr. 1998, IEEE||Final project plan|
|11/11||Veteran’s Day (no class)|
|11/14||Rust||Michael Coblenz, Michelle L. Mazurek, and Michael Hicks. 2022. Garbage collection makes Rust easier to use: a randomized controlled trial of the Bronze garbage collector. ICSE 2022. DOI|
|11/16||Natural programming||Brad A. Myers, John F. Pane, and Amy J. Ko. 2004. Natural programming languages and environments. CACM. DOI|
|11/18||Static vs. Dynamic types||Endrikat, Hanenberg, Robbes, Stefik, “How do API documentation and static typing affect API usability?”, ICSE 2014. ACM||Project checkpoint|
|11/21||Theory||No reading; focus on project|
|11/25||No class: Thanksgiving|
|11/28||PLIERS: putting it all together|
|11/30||Project presentations||Project presentations (split class in two groups)|
|12/2||Project presentations||Project report|