RESEARCH STATEMENT
I research and design digital products designed for millions of users.
I continue to build on my doctoral research, which explores how a digital therapeutic intervention’s aims can be embodied in the moment-to-moment user experience. For example: Can we convert teen passion for dramatic stories into real-world self-improvement outcomes? This is an example of ‘transfer’, a long-studied problem in educational games, VR therapy, and other digital entertainment based approaches to impact. In my research I argue that transfer is not easy, but it’s possible.”
My work responds to 30 years of mostly failed attempts to apply video game and digital entertainment product design to non-entertainment products. I agree with theorists who argue these points:
- Video games have great promise as tools for change
- Past approaches, such as gamification[1] and edutainment[2], have failed to deliver on that promise
- Activity-goal alignment theory can address those failures
- A good method to advance this research area is to apply design theory in building innovative products
From 2012 to 2017, my colleagues and I explored how to apply activity-goal alignment theory to Cognitive Behavioral Therapy (CBT), the leading evidence-based theory to prevent and treat depression and anxiety. In recognition of the many self-directed digital CBT therapy products that have failed to engage their users enough to have major impact, I argue our prioritization of engagement is appropriate, and have the potential to be more effective than top-cited work (e.g. SPARX) because the users discover, internalize, and employ strategies of CBT via playful, iterative, challenge-based practice to master systems whose rules embed CBT.
METHODS
Formal (e.g. NIH-funded) research design might include
- Two studies (N=30 pilot, N=300 final)
- Five Stages (recruit, screen, pre survey, treatment, post survey)
- Justification for setting (lab vs natural), participant selection, exposure
- Analysis of methodological weaknesses (e.g. biases due to compensation) and mitigation efforts
- Quantitative data from valid, reliable published measures of knowledge and attitudinal change
- Power and bias reviews from statistics, subject matter, other research experts, prior to study
- IRB approval
My study designs begin with literature reviews, and address known weaknesses. For example, many studies of game-based interventions were unable to distinguish engagement effects attributable specifically to the novelty of a video game experience, or rely on anecdotal data to support their claims of engagement levels comparable to commercial games. To address this in a recent project, I added a second control group to create a three-group design: one control group plays a top-selling commercial game, while experimental group plays our prototype, and a third group engages in Treatment As Usual – an online elearning course with known levels of efficacy).
- Third stage (‘promising signs’ / exploratory user research) research prior to pilot, during production
As an applied researcher, I often combine modern product design approaches (iterative, user data-driven), combining and modifying methods from the following academic and commercial fields:
- psychology (behavioral, social)
- digital health
- design
- media and communication
- commercial video game production
I often work at early stage (concept) projects, typically employing the following methods:
- Basic ethnographic (e.g. natural and laboratory behavioral observation, semi-structured interview, and/or attitudinal / knowledge surveys)
- Secondary source research (academic meta-analyses, competitive product analyses, business reports based on prior user surveys, published essays from designers and user researchers)
For example, I might evaluate a nonfunctional low-fidelity prototype using this method:
- Typical N=2-10
- Recruiting via friends & family, online services (mechanical Turk, Sermo), compensated
- Screening, survey, 60 minute videoconference, intro-play-discuss
- Often single-blinded (we show two prototypes per session)
- Data, qualitative (notes and transcripts from semistructured interviews, observation of behavioral and social interactions with prototypes (function/feature definition)
- Data, quantitative (sort, speech-based, observed actions/body language, data recorded by prototype)
As an applied researcher, my aims are often broader than traditional academic research. In addition to typical UX research aims (adoption/engagement/commercial success), my work often has an efficacy assessment aim as well. I customize methods to achieve these combinations. For example, for a nonprofit-funded socioemotional treatment experiment in 2015, my colleagues and I conducted an innovative pre-pilot study method I named “Rapid Evaluation”.
- Aim: estimate (rapidly inaccurately measure) efficacy and engagement prior to pilot
- Timeline: iteration, in 1-3 week cycles
- Product: functional prototype of proposed product
- Protocol: online recruiting via mTurk, pre-screening, pre-interview 10-item survey, 60 minute play-discuss session via videoconference
- Coding: single-rater coding of body proximity, presence/absence of emotion (frustration/confusion), conversation topic (social rejection experience),
- Data: spreadsheet of coded behaviors, brief written statements 3 topics per playtest, videos of playtests.
- Outcome: we feel the “Rapid Eval” method fills an important gap between non-behavioral methods (surveys, secondary sources) and a typical 8-month academic pilot test. Its findings are too low quality to substitute for pilot, but can be used to justify resources for one.
For preproduction or larger-scale research during production that is sufficiently resourced, I collaborate with experts in laboratory settings to design experiments suitable for the budget and specific project.
[1] Gamification is adding generic mechanics from video games such as point scoring and leaderboards.
[2] Edutainment is didactically delivered educational content interleaved with entertaining experiences.
HIGHLIGHTS
- Twice invited presenter and design participant, “Depression Game Jam” workshop, University of Southern California (2014), Radboud University (2015)
- Presentation title: “Prototyping Methods to find High Alignment between Game Mechanics and CBT for Teen Depression.”
- Accepted panel, Foundations of Digital Games (FDG) 2015, Asilomar CA.
- “Game Design Prototyping Methods To Find High Alignment Between Game Mechanics And CBT For Teen Depression For Android Devices.”
- Invited speaker, “Youth Technology Health Live 2015”, San Francisco, April 2015. “Methods and Findings for Engagement Studies of “Surviving Independence” Game-Based Behavior Change Intervention”
- Invited participant, HealthFoo 2015
- Won $250k grant to Northwest Media Inc., titled “Online Training for Foster and Primary Parents of Neglected Children”, NIH. Co-PI. Designed, developed product, completed pilot study, 2014.
- Won $2.4M grant to Northwest Media Inc., titled “VSG”: a video game that teaches independent living skills to at-risk youth, NIH. Co-PI. Designed, developed product, completed final study, 2014
- 2011-2012 “Lead Researcher” (PI), successful $30,000 YAWCRC “Young People and Game Developers Working Together: Modelling a Process for Video Game Design Co-Creation” link