Research Highlight: CRA Board Member Mary Czerwinski

By Mary Czerwinski, Microsoft Research

My research revolves around tracking and understanding users’ emotional states and leveraging that information as additional context for the design of emotionally sentient systems. Some of the systems we have built have been designed for a user’s own personal reflection. Our first application, AffectAura, provided users with their own behavior patterns over time, such as what they were doing, where they were, who they were with and how they felt. This information could be used to make personal decisions about behavior change—if certain activities usually result in your feeling good or bad, perhaps you want to increase or decrease those behaviors.

We have also built wearables that allow other people to see how you are feeling in real time. These systems are more controversial. While it can be useful for others, such as work colleagues or family members, to know how you’re feeling, not everyone is comfortable divulging this kind of personal information.

We have also designed systems to help people learn how to regulate their emotions—either with just-in-time interventions or through prolonged use of a skills app as a complement to therapy. For these systems, we either rely on physiological emotion signals from wearable computers or through self-reporting by the user. Machine learning can then be used to determine which psychological interventions make sense for that particular user at that time and in that context. We are referring to this latter line of research as “precision psychology”—where we are monitoring a user 24×7 in order to understand which psychological treatments are most effective for them at which moments. Our preliminary studies have shown that using machine learning to target interventions in this way leads to positive changes in behavior, with users learning new skills that they enjoy using and reducing their reported negative behaviors. However, it is very important that the content that is provided to users, when targeted in this manner, is diverse enough so it isn’t boring or tedious. Our PopTherapy and ParentGuardian systems were good research vehicles for exploring these topics.

More recent research efforts have moved toward designing emotionally sentient conversational agents that partner with the user to improve their health and well-being. Our studies have shown that, just like in human conversations and interactions, design parameters like attractiveness, personality type, and speaking style are all important for a user to trust the agent and want to keep interacting with it. We have also seen success in using a known personality as the conversational agent. Of course, gamifying behavior-change applications can also help to ensure that people use the agent for a longer period of time. Our latest app, Pocket Skills, was designed with this philosophy in mind. PocketSkills delivers Dialectical Behavior Therapy (DBT) training skills to users who are already in DBT, but need an app to help them practice their new skills when they aren’t at their doctor’s office.

Another research avenue of mine centers around stress, focus, and control in the workplace. Together with my colleagues Gloria Mark (University of California, Irvine) and Shamsi Iqbal (Microsoft Research), we have spent years examining what factors contribute to stress, or the lack of it, at the office. There are many factors that we’ve identified as related to stress, including email overload, work deadlines, poor dietary habits, and even personality type. One notable factor that is associated with less stress in the workplace appears to be a sense of control over your day. A sense of control can allow a worker to see stressful events more as opportunities or challenges. We have also observed that those workers with a higher sense of control over their multitasking (e.g., accessing non-work related websites or social media) tend to be able to focus more readily and don’t need tools or technologies to assist them with concentrating. Employees who don’t control their access to social media and non-work related websites might benefit from technology that limits access until a work break is truly needed or enough focus time has been achieved. Future work will be to combine what we’ve learned about behavior change and conversational agent interaction technology with ways to reduce stress and improve focus throughout the workday.

About the Author

Mary Czerwinski is a research manager of the Visualization and Interaction Research Group at Microsoft Research. Mary’s research focuses primarily on emotion tracking, information worker task management, health and wellness for individuals and groups. Her background is in visual attention and multitasking. She holds a Ph.D. in cognitive psychology from Indiana University in Bloomington. Mary was awarded the ACM SIGCHI Lifetime Service Award, was inducted into the CHI Academy, and became an ACM Distinguished Scientist in 2010. Mary became a Fellow of the ACM in 2016. She also received the Distinguished Alumni award from Indiana University’s Brain and Psychological Sciences department in 2014.