I am a Postdoctoral Researcher at Caltech where I am mentored by Anima Anandkumar, Mike Alvarez, and Andrew Hung. I received my PhD in Human Centered Design & Engineering from the University of Washington, where I was advised by Gary Hsieh.
My primary interests focus on Leveraging AI for Behavior Change as well as Empowering Users to Inspect AI Systems. I have unique experience in Human-Computer Interaction and Design in combination with Applied ML. I worked on projects in various areas including Explainable AI, Surgical AI, Health Behavior Change, Workspace Productivity and Social Media Moderation.
Over the summers, I've been lucky to intern with Jonathan Bragg and Doug Downey at Allen Institute for AI, Saleema Amershi and Andrés Monroy-Hernández at Microsoft Research Redmond, and Daniel Avrahami at Fuji-Xerox Palo Alto Research Lab.
Dec 2024 | "Human AI Collaboration for Unsupervised Categorization of Live Surgical Feedback" accepted to Nature npj | Digital Medicine |
Nov 2024 | Two proceedings papers accepted at ML4H 2024: ▶ "Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment" Best Paper Award ▶ "Automating Feedback Analysis in Surgical Training: Detection, Categorization, and Assessment" |
Oct 2024 | "ChatGPT Based Data Augmentation for Improved Parameter-Efficient Debiasing of LLMs" accepted at COLM 2024 |
May 2024 | Invited talk at USC ISI on "Human-AI Interaction: From Supporting Surgical Training to Inspecting Social Bias in LLMs". |
Feb 2024 | Invited talk at University of Oxford's Future of Tech & Society Discussion Group on "Testing Fairness in Generative AI". |
Dec 2023 | "Deep Multimodal Fusion for Surgical Feedback Classification" received a Best Paper Award at ML4H 2023 |
Challenges in Moderating Disruptive Player Behavior in Online Competitive Action Games
Rafal Kocielnik, Zhuofang Li, Claudia Kann, Deshawn Sambrano, Jacob Morrier, Mitchell Linegar, Carly Taylor, Min Kim, Nabiha Naqvie, Feri Soltani, Arman Dehpanah, Grant Cahill, Animashree Anandkumar, R Michael Alvarez
Frontiers in Computer Science: Human-Media Interaction, 2024
Publisher
Artificial Intelligence-Based Video Feedback to Improve Novice Performance on Robotic Suturing Skills: A Pilot Study
Runzhuo Ma, Dani Kiyasseh, Jasper A Laca, Rafal Kocielnik, Elyssa Y Wong, Timothy N Chu, Steven Cen, Cherine H Yang, Istabraq S Dalieh, Taseen F Haque, Mitch G Goldenberg, Xiuzhen Huang, Anima Anandkumar, Andrew J Hung
Journal of Endourology, 2024
Publisher
Deep Multimodal Fusion for Surgical Feedback Classification
Rafal Kocielnik, Elyssa Y. Wong, Timothy N. Chu, Lydia Lin,
De-An Huang, Jiayun Wang, Anima Anandkumar, Andrew J. Hung
Machine Learning for Health (ML4H), 2023 Best
Proceedings Paper
Publisher
Exploring Social Bias in Downstream Applications of
Text-to-Image Foundation Models
Adhithya Saravanan, Rafal Kocielnik, Roy Jiang, Pengrui Han, Anima Anandkumar
NeurIPS workshop on Failure Modes in the Age of Foundation Models, 2023
(accepted for special issue of PMLR)
arXiv
OpenReview
BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models
Rafal Kocielnik, Shrimai Prabhumoye, Vivian Zhang, Roy Jiang, R. Alvarez Michael, Anima Anandkumar
ICML workshop on Deployment Challenges for Generative AI 2023
arXiv
HuggingFace Tool
Project Website
Dataset
Large Language Models and Political Science
Mitchell Linegar, Rafal Kocielnik, R Michael Alvarez
Frontiers in Political Science, 2023
Publisher
Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions
Rafal Kocielnik, Sara Kangaslahti, Shrimai Prabhumoye, Meena Hari, Michael Alvarez, Anima Anandkumar
NeurIPS 2023 Transfer Learning for Natural Language Processing Workshop
Publisher
Medium Blog
From Who You Know to What You Read: Augmenting Scientific Recommendations with Implicit Social Networks
Hyeonsu B Kang, Rafal Kocielnik, Andrew Head, Jiangjiang Yang, Matt Latzke, Aniket Kittur, Daniel S Weld, Doug Downey, Jonathan Bragg
CHI 2022
Publisher
Using real-time feedback to improve surgical performance on a robotic tissue dissection task
Jasper A Laca, Rafal Kocielnik, Jessica H Nguyen, Jonathan You, Ryan Tsang, Elyssa Y Wong, Andrew Shtulman, Anima Anandkumar, Andrew J Hung
European Urology Open Science, 2022
Publisher
Can I Talk to You about Your Social Needs? Understanding Preference for Conversational User Interface in Health
Rafal Kocielnik, Raina Langevin, James S George, Shota Akenaga, Amelia Wang, Darwin P Jones, Alexander Argyle, Callan Fockele, Layla Anderson, Dennis T Hsieh, Kabir Yadav, Herbert Duber, Gary Hsieh, Andrea L Hartzler
Conversational User Interfaces, 2021 Best Paper Honorable Mention
Publisher
Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems
Rafal Kocielnik, Saleema Amershi, Paul N Bennett
CHI 2019
High Impact: 240+ citations
Publisher
MSR Blog
Presentation Video
HarborBot: A Chatbot for Social Needs Screening
Rafal Kocielnik, Elena Agapie, Alexander Argyle, Dennis T Hsieh, Kabir Yadav, Breena Taira, Gary Hsieh
AMIA Annual Symposium Proceedings 2019
Publisher
Reflection Companion: A Conversational System for Engaging Users in Reflection on Physical Activity
Rafal Kocielnik, Lillian Xiao, Daniel Avrahami, Gary Hsieh
UbiComp / IMWUT 2018
High Impact: 150+ citations
Publisher
Designing for Workplace Reflection: A Chat and Voice-Based Conversational Agent
Rafal Kocielnik, Daniel Avrahami, Jennifer Marlow, Di Lu, Gary Hsieh
Design of Interactive Systems 2018
Publisher
Reciprocity and Donation: How Article Topic, Quality and Dwell Time Predict Banner Donation on Wikipedia
Rafal Kocielnik, Os Keyes, Jonathan T Morgan, Dario Taraborelli, David W McDonald, Gary Hsieh
CSCW 2018
Publisher
Send Me a Different Message: Utilizing Cognitive Space to Create Engaging Message Triggers
Rafal Kocielnik, Gary Hsieh
CSCW 2017
Publisher
Calendar. help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop Authors
Justin Cranshaw, Emad Elwany, Todd Newman, Rafal Kocielnik, Bowen Yu, Sandeep Soni, Jaime Teevan, Andrés Monroy-Hernández
CHI 2017
Publisher
You get who you pay for: The impact of incentives on participation bias
Gary Hsieh, Rafal Kocielnik
CSCW 2016 Best Paper Award
Publisher
Smart technologies for long-term stress monitoring at work
Rafal Kocielnik, Natalia Sidorova, Fabrizio Maria Maggi, Martin Ouwerkerk, Joyce HDM Westerink
International Symposium on Computer-based Medical Systems 2013
High Impact: 170+ citations
Publisher