Chen Chen

Publications Main venues include HCI (e.g., CHI, UIST), A11y (e.g., ASSETS), XR (e.g., ISMAR), and Sensing (e.g., SenSys, IPSN, IMWUT).

MemoVis: A GenAI-Powered Tool for Creating Companion Reference Images for 3D Design Feedback

Chen Chen, Cuong Nguyen, Thibault Groueix, Vladimir G. Kim, Nadir Weibel
In the Journal of Transactions on Computer-Human Interaction (TOCHI 2024)

Providing asynchronous feedback is a critical step in the 3D design workflow. A common approach to providing feedback is to pair textual comments with companion reference images, which helps illustrate the gist of text. Ideally, feedback providers should possess 3D and image editing skills to create reference images that can effectively describe what they have in mind. However, they often lack such skills, so they have to resort to sketches or online images which might not match well with the current 3D design. To address this, we introduce MemoVis, a text editor interface that assists feedback providers in creating reference images with generative AI driven by the feedback comments. First, a novel real-time viewpoint suggestion feature, based on a vision-language foundation model, helps feedback providers anchor a comment with a camera viewpoint. Second, given a camera viewpoint, we introduce three types of image modifiers, based on pre-trained 2D generative models, to turn a text comment into an updated version of the 3D scene from that viewpoint. We conducted a within-subjects study with 14 feedback providers, demonstrating the effectiveness of MemoVis. The quality and explicitness of the companion images were evaluated by another eight participants with prior 3D design experience.

 Preview  Teaser

---

Enhancing Accuracy, Time Spent, and Ubiquity in Critical Healthcare Delineation via Cross-Device Contouring

Matin Yarmand, Chen Chen, Michael V. Sherer, Yash N. Shah, Peter Liu, Borui Wang, Larry Hernandez, James D. Murphy, Nadir Weibel
Proceedings of the 2024 ACM International Conference on Designing Interactive Systems (DIS '24), Copenhagen, Denmark

Improving accuracy, time spent, and ubiquity of delineation has been a long-standing design aim, yet many HCI works have overlooked high-stakes and complex healthcare annotation. We explore contouring, a critical workflow aimed at identifying and segmenting tumors, usually performed on immobile desktop computers in clinics, in which limited support for mobile access leads to prolonged and subpar treatment planning. Following interviews and think-aloud studies (N=10 physicians), we report key contouring behaviors, and later design a novel cross-device prototype that enables contouring on everyday touch devices. We compared contouring via desktop and touch in a lab study (N=8 residents) and found that mobile phones not only yielded similar accuracy, but also took significantly less time. Our results point to three broad design guidelines for cross-device solutions deployed within standalone healthcare workflows, and highlight how incorporating different device and input modalities can improve treatment delivery in today's distributed healthcare environments.

 Project Page

---

“I’d be watching him contour till 10 o’clock at night”: Understanding Tensions between Teaching Methods and Learning Needs in Healthcare Apprenticeship

Matin Yarmand, Chen Chen, Kexin Cheng, James D. Murphy, Nadir Weibel
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI '24), Honolulu, HI, USA (Acceptance Rate = 26.3%)

Apprenticeship is the predominant method for transferring specialized medical skills, yet the inter-dynamics between faculty and residents, including methods of feedback exchange are under-explored. We investigate contouring: outlining tumors in preparation for radiotherapy, a critical skill that when performed subpar, severely degrades patient survival. Interviews and design-thinking workshops (N = four faculty; six residents) revealed misalignment between teaching methods and residents who desired timely, relevant, and diverse feedback. We further discuss reasons: overlapping learning content and strategies to ease tensions between clinical and teaching duties, and lack of support for exchange of cognitive processes. The follow-up survey study (N = 67 practitioners from 31 countries), which contained annotation and sketching tasks, provided diverse perspective over effective feedback elements. We lastly present sociotechnical implications in supporting the teaching duties of the faculty and the cognitive models of learners, such as systematically leveraging senior learners in providing case-based guidance and supporting double-sided flow of cognitive information via in-situ video snippets.

 Project Page

---

PaperToPlace: Transforming Instruction Documents into Spatialized and Context-Aware Mixed Reality Experiences

Chen Chen, Cuong Nguyen, Jane Hoffswell, Jennifer Healey, Trung Bui, Nadir Weibel
Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST '23), San Francisco, CA, USA (Acceptance Rate = 25.1%)

Paper instructions are a mainstream medium for sharing knowledge. However, consuming such instructions and translating them into activities can be inefficient due to the lack of connectivity with the physical environment. PaperToPlace is a novel workflow comprising an authoring pipeline, which allows the authors to rapidly transform and spatialize existing paper instructions into an MR experience, and a consumption pipeline, which computationally places each instruction step at an optimal location that is easy to read and does not occlude key interaction areas.

 Preview  Teaser  Slide

---

Embodied Exploration: Facilitating Remote Accessibility Assessment for Wheelchair Users with Virtual Reality

Siyou Pei, Alexander Chen, Chen Chen, Mingzhe "Franklin" Li, Megan Fozzard, Hao-Yun Chi, Nadir Weibel, Patrick Carrington, Yang Zhang
Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '23), New York, NY, USA (Acceptance Rate = 27%)

Assessing accessibility for wheelchair users can be challenging, due to lack of accessibility details needed for individual users. Embodied Exploration is a VR technique to deliver the experience of a physical visit while keeping the convenience of remote assessment. Embodied Exploration allows wheelchair users to explore high-fidelity digital replicas of physical environments with themselves embodied by avatars, leveraging the increasingly affordable VR headsets.

 Teaser

---

Screen or No Screen? Lessons Learnt from a Real-World Deployment Study of Using Voice Assistants With and Without Touchscreen for Older Adults

Chen Chen, Ella T. Lifset, Yichen Han, Arkajyoti Roy, Michael Hogarth, Alison A. Moore, Emilia Farcas, Nadir Weibel
Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '23), New York, NY, USA (Acceptance Rate = 27%)

Voice-first user interfaces refers to those where voice is the primary interaction modality. Standalone voice + touchscreen Voice Assistants (VAs), such as Echo Show, are specific types of devices that adopt such interfaces and are gaining popularity. However, the affordances of the additional touchscreen for older adults are unknown. We demonstrate the lessons learnt through a real-world deployment study of VAs in the residence of older adult living in San Diego area.

 Slide  Project Page

---

Ascertaining Whether an Intelligent Voice Assistant Can Meet Older Adults' Health-Related Needs in the Context of a Geriatrics 5Ms Framework

Ella T. Lifset, Kemeberly Charles, Emilia Farcas, Nadir Weibel, Michael Hogarth, Chen Chen, Janet G. Johnson, Mary Draper, Annie Nguyen, Alison A. Moore
Journal of Gerontology and Geriatric Medicine, 2023

The Geriatrics 5Ms: Medications, Mind, Mobility, what Matters most and Multicomplexity is a framework to address the complex needs of older adults. Follow up with our previous research, we attempt to understand how voice assistants may address their health-related needs and ascertained their fit into the Geriatrics 5Ms framework, from the perspective of older adults.

 Project Page

---

Mites: Design and Deployment of a General-Purpose Sensing Infrastructure for Buildings

Sudershan Boovaraghavan, Chen Chen, Anurag Maravi, Mike Czapik, Yang Zhang, Chris Harrison, Yuvraj Agarwal
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT/UbiComp '23), Cancun, Mexico

Real-world deployment of building-scale, general-purpose and high-fiedlity sensing system is challenging! Mites is the first scalable end-to-end hardware-software system for supporting and managing distributed general-purpose sensors in buildings, and has been successfully deployed in Carnegie Mellon University of more than five years.

 Project Page

---

VRContour: Bringing Contour Delineations of Medical Structures Into Virtual Reality

Chen Chen, Matin Yarmand, Varun Singh, Michael V. Sherer, James D. Murphy, Yang Zhang, Nadir Weibel
The 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR '22), Singapore (Acceptance Rate = 21.6%)

Contour delineation is an indispensable step in radiotherapy treatment planning, yet today’s contouring software (such as Eclipse) is constrained to only work with a 2D display, which is less intuitive and requires high task loads. VRContour is the first effort that aims to explore how to use VR to support an easier and more intuitive contour delineation workflow.

 Teaser  Talk  Project Page

---

Investigating Input Modality and Task Geometry on Precision-first 3D Drawing in Virtual Reality

Chen Chen, Matin Yarmand, Zhuoqun Xu, Varun Singh, Yang Zhang, Nadir Weibel
The 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR '22), Singapore (Acceptance Rate = 21.6%)

Precision-first 3D drawing refers to the 3D drawing process where the precision is prioritized, which is indispensable for many precise 3D tasks. However, due to lack of physical support, limited depth perception, and the non–planar nature of 3D curves, it is challenging to adjust mid–air strokes to achieve high precision. We demonstrated a first effort to understand how task geometric shapes and input modalities affect such precision–first drawing performance in a within–subject study (n = 12) focusing on 3D target tracing in commercially available VR headsets.

 Dataset

---

Understanding Barriers and Design Opportunities to Improve Healthcare and QOL for Older Adults through Voice Assistants

Chen Chen, Janet G Johnson, Kemeberly Charles, Alice Lee, Ella T Lifset, Michael Hogarth, Alison A Moore, Emilia Farcas, Nadir Weibel
The 2021 ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '21), Virtual Event, USA (Acceptance Rate = 29%)

One key challenge of aging-in-place is to allow older adults to easily access and interact with digital information, which however is challenging! We conducted a first qualitative study with real-world older adults, geriatricians and senior care nurses to understand potential design opportinuity to improve healthcare and quality of life for older adults through voice assistants.

 Teaser  Project Page  News

---

ExGsense: Toward facial gesture sensing with a sparse near-eye sensor array

Chen Chen, Ke Sun, Xinyu Zhang
The 2021 ACM International Conference on Information Processing in Sensor Networks (IPSN '21), Nashville, TN, USA (Acceptance Rate = 24.5%)

Immersive face-to-virtual-face telecommunication is one unique use case for virtual reality (VR) technologies. Existing camera-based telephony systems cannot be used for such immersive VR video chat, due to the physical occlusions of head-mounted displays (HMDs) and/or unwieldy positioning of cameras. We present ExGSense, a new VR input modality that can sense and reconstruct both upper and lower facial gestures, by only using lightweight biopotential sensors embedded within the HMDs.

 Slide

---

"Alexa, stop spying on me!" Speech Privacy Protection Against Voice Assistants

Ke Sun, Chen Chen, Xinyu Zhang
The 2020 ACM International Conference on Embedded Networked Sensor Systems (SenSys '20), Virtual Event, Japan (Acceptance Rate = 18%)

Voice assistants (VAs) are becoming highly popular recently as a general means of interacting with the Internet of Things. However, the use of always-on microphones on VAs imposes a looming threat on users' privacy. We propose MicShield, the first system that serves as a companion device to enforce privacy preservation on VAs.

Articles, Short Papers, Posters, and Demos

How do Older Adults Set Up Voice Assistants? Lessons Learned from a Deployment Experience for Older Adults to Set Up Standalone Voice Assistants

Chen Chen, Ella T. Lifset, Yichen Han, Arkajyoti Roy, Michael Hogarth, Alison A. Moore, Emilia Farcas, Nadir Weibel
The 2023 ACM International Conference on Designing Interactive Systems (DIS '23 Companion), Pittsburgh, PA, USA

 Teaser  Project Page

---

Professional Experience

Research Scientist Intern, Microsoft Research, Redmond, WA, USA, June 2024 - Sep. 2024

Ph.D. Student Collaborator, Adobe Research, Remote, CA, USA, Nov. 2023 - Now

Research Scientist Intern, Adobe Research, San Francisco, CA, USA, June 2023 - Nov. 2023

Ph.D. Student Collaborator, Adobe Research, Remote, CA, USA, Sep. 2022 - April 2023

Research Scientist Intern, Adobe Research, Remote/San Francisco, CA, USA, June 2022 - Sep. 2022

Research Associate, Carnegie Mellon University, Pittsburgh, PA, USA, Jan. 2018 - June. 2018

Research Assistant, Carnegie Mellon University, Pittsburgh, PA, USA, Dec. 2016 - Dec. 2017

Research Assistant, The University of Nottingham, Nottingham, UK, May 2015 - Aug. 2015

Awards & Honors

Best Paper Honorable Mentioned Award, ACM CHI '24

Special Recognition as Useful (or Highly Useful) Review, ACM UIST '23, ACM CHI '24

Inclusion, Diversity, Equity & Accessibility (I.D.E.A) Scholarship, IEEE ISMAR '22

Adobe Research Fellowship (Finalist), '21

Qualcomm Innovation Fellowship (Abstract Selected) '21 [Abstract]

Best Poster Award, ACM SenSys '20

Qualcomm Innovation Fellowship (Abstract Proposal) '20 [Proposal]

Qualcomm Innovation Fellowship (Finalist) '19 [Abstract] [Proposal] [Poster]

British Petroleum (BP) Fellowship '15

Teaching

Teaching Assistant and Lab Instructor, CSE 165 3D User Interaction (Spring '24), UC San Diego [Final Project Video Teasers] [Student Evaluation] [Instructor Evaluation]

Teaching Assistant and Lab Instructor, WES 237A Introduction to Embedded Computing (Winter '24), UC San Diego [Student Evaluation]

Teaching Assistant and Lab Instructor, CSE 118/218 Ubiquitous Computing (Fall '23), UC San Diego [Undergraduate Student Evaluation] [Graduate Student Evaluation]

Teaching Assistant and Lab Instructor, CSE 118/218 Ubiquitous Computing (Fall '22), UC San Diego [Undergraduate Student Evaluation] [Graduate Student Evaluation]

Teaching Assistant, WES 237A Introduction to Embedded Computing (Winter '21), UC San Diego [Student Evaluation] [Instructor Evaluation]

Teaching Assistant, CSE 291H Data Center System (Fall '19), UC San Diego [Student Evaluation] [Instructor Evaluation]

Teaching Assistant, CSE 112 Advanced Software Engineering (Fall '20), UC San Diego [Student Evaluation]

Teaching Assistant, COGS 118 AI Algorithms (Summer '19), UC San Diego [Student Evaluation]

Teaching Assistant, 08-672 Web Application Development (Fall '17), Carnegie Mellon University

Teaching Assistant, 08-672 Web Application Development (Fall '16), Carnegie Mellon University

Student Mentored

Borui Wang, HCDE Undergraduate @UW (incoming CS Ph.D. Student at Cornell)

Anish Rajeshkumar, Math + CS Undergrad Student @UCSD (now MS Student @USC), Fall '22 - Summer '23

Arkajoyti Roy, Math + CS Undergrad Student @UCSD (now Staff Engineer @Meta), Fall '22 - Summer '23

Varun Singh, CS Undergrad Student @UCSD, Fall '21 - Summer '23

Yichen Han (now MS Student @Carnegie Mellon University), Winter '22 - Summer '22

Christopher B. Han, Winter '22 - Summer '22

Peng W. Lee, Winter '22 - Summer '22

Ella T. Lifset, Biology Undergrad Student @UCSD, Summer '20 - Summer '21

Robin Z. Xu (now Staff Engineer @Intuitive), Winter '21 - Summer '21

Kemeberley Charles, now Medical Student at UCSD Health

Alice Lee, now UXR at Microsoft

Chenghong Bian, now Ph.D. student at Imperial College London

Services

IMWUT/UbiComp Video Co-Chair ('23)

UCSD CSE Ph.D. Visit Day Area Lead (HCI) ('22 - '23)

UCSD CSE Ph.D. Admission Committee ('19 - '23')

Program Committee/Associate Chair (AC) for ACM CHI '23 (Late Breaking Work Track), CHI '25 (Full Paper Track, User Experience & Usability Subcommittee)

Conference Reviewers for ACM CHI, UIST, CSCW, IUI, DIS, EICS, ICMI, SUI, VRST ('20 - Now)

Conference Reviewers for IEEE VR, ISMAR ('20 - Now)

Journal Reviewers for IMWUT, IJHCI, PLOS Digital Health ('20 - Now)

Student Volunteer at ASSETS ('21 - '23), DIS '23

Co-Host for DLab Research Meeting, UCSD

Volunteer at the International Humanity Foundation, Banda Aceh, Indonesia, Summer '13