Hung, Pei-Yao, Jin-Ling Lo, Hsin-Yen Wang, Hao-Hua Chu, Ya-Lin Hsieh (2009). CuttingGame: A Computer Game to Assess & Train the Visual-motor Integration Ability for Preschool Children with Autism. Interactive Creative Play with Disabled Children Workshop, ACM SIGCHI Interaction Design and Children (IDC), June 3, Como, Italy. [Slides][BibTex]
Enjoyable Training to Improve Visual-Motor Integration for Children with Autism
Background: This is my master thesis project, and also my first research project in the area of Human-Computer Interaction in which I completed the process from initial observation, co-design with domain experts, prototyping, to evaluation in the form of one-month field deployment with real target users. This project inspired me to pursue research in the area of Human-Computer Interaction as I saw first-hand how the system I design and develop can make a meaningful and joyful impact on people, even just 7 children and their families.
The research lab I was associated with when I was pursuing my master degree in Computer Science had a series of successful collaborations with another research lab at the Department of Occupational Therapy within the same university. Through a series of brainstorming sessions, we identified an opportunity to transform a therapy activity into the computer game. I then led the design, development, and evaluation of a computer game, in collaboration with occupational therapists, under the mentorship of two faculty, one from each department.
My role(s): Project Leader, Researcher, Designer, Developer
Collaborator(s): Dr. Jin-Ling Lo, Dr. Hsin-Yen Wang, Dr. Hao-Hua Chu, Ya-Lin Hsieh
Method(s):
Skill(s):
Motivation: Children with autism are likely to show difficulty with fine motor skills and visual-motor integration. The conventional solution is for children to attend training sessions led by occupational therapists. Unfortunately, such sessions usually happen at most weekly, which is too infrequent for them to gain the maximum benefit.
Problem Solving: Drawing from my 5-month field observations of training sessions, I developed a computer paper-cutting game that mimicked a training activity. I worked with 7 children with autism and their families to conduct a one-month field deployment. I also developed additional tools for occupational therapists to review children's gameplay and provide more detailed feedback.
Field Observation
Co-design
2 occupational therapists
Development
Field Testing
7 families for a month
Data Analysis
Outcomes:
- Ability improved: children's visual-motor integration capability, approximated by in-game performance, had improved.
- Children enjoyed it: children enjoyed playing the 'game' daily, which enabled intensive training.
The outcomes of this project include CuttingGame, a desktop training game for children, a game manual, a set of tools for occupational therapists to review children's progress, and two scholarly publications (link).
Summary:
Date: 2007 ~ 2008
CuttingGame: A Computer Game to Assess & Train The Visual-Motor Integration Ability for Preschool Children with Autism from PEI-YAO HUNG
Research & Design Strategy
Use Field Observation to Understand the Characteristics of Children with Autism and Occupational Therapy
This was a project conducted in Taipei, Taiwan, where being autistic was even more stigmatized than in the U.S., and people seldom talked about such condition. To get a deeper understanding of the target population, children with autism, I collaborated with occupational therapists in a teaching hospital. In this hospital, the therapists conducted weekly sessions where parents of autistic children brought their children to attend sessions. I shadowed the sessions for two semesters, as part of the student group consists of students majoring in occupational therapy, and participated in the post-session debrief. The post-session debrief was designed to allow students to discuss their observations with occupational therapists leading the sessions and make a plan for future sessions.
Identify Important Design Requirements to Support System Design to Accommodate Practical Constraints of Training
Through observing these sessions, I identified several design requirements for a training application. First, the application needed to have a simple interaction flow that was easy to learn, as children with autism might have trouble concentrating on the training activity and could easily get distracted. Second, the design should employ activity that children were already familiar with as they needed to use such training application at home without the assistance and guidance of occupational therapists. Lastly, the terminology and concepts in the application needed to be simple as these children were young (e.g., around 5 years old) and did not yet have the capacity to understand advanced concepts.
Develop a Computer Game for Home-based Training with Careful Considerations of Children and Feedback from Occupational Therapists
Based on these requirements, I designed a computer game that allowed users to perform the paper cutting and drawing activity typically used for training visual-motor integration ability. As I observed, children typically were instructed to follow these steps: first, use a scissor to cut the figure provided. Second, choose a scene as the background. Third, put the figure on the scene, add other accessories to decorate the figure and the scene. Lastly, color the figure and the accessories to make a picture.
I implemented this flow in an application, CuttingGame, where children could select a list of figures and a list of scenes to compose their picture and colored them. I didn't use any words on the interface but instead relied on symbols (e.g., a scissor), to label each GUI widgets (e.g., button). All the figures and scenes were selected based on feedback from occupational therapists regarding how likely children knew these concepts.
Use Field Deployment to Evaluate the Effectiveness of the Proposed System in Training and Evaluating Children's Progress
To evaluate the possibility of using CuttingGame to train visual-motor integration ability for children with autism, the occupational therapists and I worked with 7 children and their parents to field-test the game at home for a month. To evaluate the effect of CuttingGame, children underwent a standardized test that was used to evaluate visual-motor integration ability each week in the hospital before, during, and after the one-month training period.
Based on the standardized test, I developed an auto-scoring algorithm that scored children's performance in CuttingGame by measuring how accurately they were able to follow the contour of the figure during cutting. To help occupational therapists to provide more detailed feedback, I also developed a playback tool so that occupational therapists could observe how each child was cutting the figures in CuttingGame.
Outcomes
Findings & Implications
CuttingGame Provides an Enjoyable Training Experience for Children at Home
My preliminary analysis using children's gameplay logs showed that children achieved better accuracy after a month of training. Moreover, qualitative feedback suggested that children enjoyed playing the game at home frequently and thus potentially achieved the goal of extensive training beyond the hospital.
Systems & Applications
Multiple applications were created as the outcomes of this project. First, a desktop application that allowed children to play the paper-cutting and drawing game, which recorded children's in-game performance (i.e., cutting path) for measuring progress. This was accompanied by a game manual created for parents to understand the game flow and different features in case they needed instructions at home. Second, a script was developed based on the evaluation strategies in Southern California Motor Accuracy Test-Revised (MAC-R) and the Visual Motor Test of Integration (VMI) to evaluate children's performance based on game logs. Lastly, a gameplay viewing application was created for occupational therapists to replay and review children's gameplay to provide further analysis.
Background & Problems
One well-established goal of public health officials has been the early detection of developmental delays. One aspect of the delays is visual-motor integration ability, which exhibits in certain children with autism. Clinicians often use standardized tests to assess visual-motor integration ability of children, with treatment activities such as having the children clipping and pasting paper for children to practice. These activities are designed and facilitated by therapists and require professionals to observe or videotape the sessions for further analysis. The time-consuming human resource requirement makes this more extensive training/evaluation (proven to be effective in the treatment of developmental delay) almost impossible.
Research Process
Findings and Outcome
We designed a computer game, called CuttingGame, to approximate a standardized test in the assessment of the visual-motor integration abilities of children. By making the test a game, we created a playful experience that both assesses and trains children in visual-motor integration and does so in a natural and enjoyable way.