upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

A System in the Wild: Deploying a Two Player Arm Rehabilitation System for Children With Cerebral Palsy in a School Environment

Raymond Holt, Andrew Weightman, Justin Gallagher, Nick Preston, Martin Levesley, Mark Mon-Williams, and Bipinchandra Bhakta

Journal of Usability Studies, Volume 8, Issue 4, August 2013, pp. 111 - 126

Article Contents


Mehtods

The goal of the research was not to assess the clinical efficacy of the AMD approach but to determine whether it was feasible to utilize a dual-player system in a school environment, and if not, what prevented this from taking place. This section outlines the hardware and software that made up the system as deployed, the process used to deploy it in both single- and two-player modes, and the data gathering plan to evaluate its usage.

Hardware

Our research group designed 4 two-user AMD computer game systems. The systems were designed and manufactured to provide safe physical assistance to children with arm impairments while undertaking therapeutic exercises. We developed these systems to be an extension of the single-user technology that had been previously deployed for home use (Weightman et al., 2011). We designed the original system using a MicrosoftTM Force-Feedback Sidewinder II joystick that was modified to provide the required assistive forces and that would fit in an appropriate workspace. The system was connected to a PC via USB. We loaded specially designed therapeutic games onto the PC. Originally, we envisioned that the new two-player system would take a similar form, with up to six joysticks plugging into a school PC to allow multiplayer gaming, which would have the benefit of being portable and easy to store. However, several factors made this approach not feasible. On a technical front, the Sidewinder joystick was not able to provide as much force as desired. This required a move toward a new design with larger motors and customized control software developed in LabVIEWTM and delivered via a National InstrumentsTM compact Reconfigurable Input-Output (cRIO) controller. However, the move from a home to a school environment also entailed a number of significant alterations to the design.

To ensure that the system was acceptable to the school staff supervising the students’ activity with the system, we held group meetings at three different schools, with a total of seven teachers in attendance. At the meetings, we wanted to address teachers’ concerns and questions and to obtain feedback on the design concepts. It quickly became apparent that while portability and ease of storage were considered important, this was secondary compared to ease of use and a quick setup. Teachers felt that they couldn’t accommodate any system that took more than a minute or two to set up during break times. A longer setup time would leave too little time for the system to be used and would be disruptive while students waited for the teacher to get the system organized. The teachers were enthusiastic about the system and felt that time could be made in the school day for its use, through a combination of lunchtime clubs and afterschool sessions. One teacher suggested that for it to be included in the classroom environment, the games would need to have an educational angle.

Accordingly, we designed the system as a self-contained unit with two joysticks, two monitors, a PC, and cRIO controllers as shown in Figure 1. Figure 2 shows a close-up of the joystick component. We designed the system to be portable in a school environment. To ensure an acceptable footprint before we built the system, we measured doorways and aisles in a range of schools. The deployed system had a footprint of 135 cm x 67 cm (53 in. x 26 in.) and was 116 cm (45 in.) tall. We put wheels on the system so that it could be easily moved between classrooms or out of the way as needed. The cRIO and PC had separate power supplies that needed to plug into the main power supply. All system components, including the monitors, were connected to a four-way power extension so that only a single external socket was required to power the system. In case of emergency, a large safety push button (not illustrated in Figure 1) that disconnected the power from the assistive haptic interfaces was located on the front top of the system. Each haptic interface incorporated a handgrip sensor such that no assistive forces could be applied when a user was not holding the handgrip. The system included another handgrip with a push button that was used to access some of the game functions to ensure that the game was bimanual. In this way, the children with CP had something to do with their unimpaired arm, reducing the temptation to use it (rather than their impaired arm) to operate the joystick.

Figure 1

Figure 1. The AMD system as deployed

Figure 2

Figure 2. Close-up of the joystick used in the system

The monitors and cRIO would turn on automatically once power was supplied. A single press of the PC power button is all that was required to turn the PC on. This done, the system would boot up and the teacher could launch its initialization process by selecting an icon from the desktop, after which the system was ready to launch the games. In this first version of the system, initialization entailed the teacher following a short series of onscreen prompts to make sure that the joysticks were in a central position. To register that the joystick position was acceptable, the teacher pressed the trigger button for each joystick. This ensured that the system was not starting near the edge of its working range.

Software

We developed a suite of games that required therapeutically appropriate movements to play and permitted a combination of competitive/collaborative multiplayer and single player experiences. All games were based on the same outward and inward movement from the body. The goal of the movement was to follow a defined path between targets so that the assistive force controller could identify how to assist the movement. Furthermore, all of the games were based on the same scenario delivered by a cutscene when the system first loaded. The cutscene showed the players adopting the role of “monkeys whose friends have been kidnapped by a hungry crocodile and are being taken to his lair as dinner.” The premise of each game is based on the monkeys trying to rescue their friends, which is in line with Malone and Lepper’s (1987) recommendations of the use of fantasy as a motivator in playing games. We designed each of the games such that they could be operated in a single user mode. The multiplayer aspect of each game varied depending on whether it was sequential (players taking turns to move their characters) or simultaneous (both players’ characters moving at the same time), and whether it was cooperative (either both players win or both players lose against the computer) or competitive (one of the players wins and the other loses). The following are descriptions of the four games:

We developed the four games with input from a user group of children with cerebral palsy who had participated in the previous home-based project (Weightman et al., 2011). This group was recruited because their prior knowledge of the system meant that they would not be participating in the upcoming school-based trial, and consequently, there was no concern that their exposure to the games might confound later stages of the study. Meetings took place at the University of Leeds. This gave the children the opportunity to offer feedback and make comments on early iterations, to evaluate the initial concepts and game-play proposals, to give feedback on early prototypes, and to carry out informal usability testing to help identify bugs and check the clarity of instructions. Sample screens from each game are shown in Figure 3.

Figure 3

Figure 3. Screenshots of the four game types, clockwise from top left: the Puzzle Game, the Chase Game, the River Race Game, and the Van Chase Game

In addition to the games, software was required to manage the amount of assistance the joystick provided each player. We called this the Adaptation to Player Performance Algorithm (APPA), and it served two purposes. From a therapeutic perspective, the goal is to gradually reduce the amount of assistance provided as players’ motor skills improve—but such a change has to be based on therapeutic progress, rather than simply based on the amount of time played. In addition, there is the problem of creating a level-playing field between players who have different levels of impairment; scaling the amount of assistance provided to a player’s abilities provided a convenient way of doing this. To assess players’ abilities during the field deployment, every time the system started, each player completed an assessment task in which each player had to collect as many bananas as possible in a minute. This unassisted task was used to determine how much assistance a child would receive during the play session. Each child using the system performed this assessment, regardless of whether they had CP, although the system only provided assistance for the children with CP.

In addition to the assessment routine, a child’s score on a given day was compared with his or her score on this task for previous days. Where the performance scores increased by more than 10% of the previous mean score, the assistance was reduced. Where the performance score decreased by more than 10% of the previous mean score, assistance was increased. In this way, the APPA fine-tuned the assistance to the most appropriate level for a given player on a given day.

Deployment

Our goal was to create the most realistic test possible for the system, not to assess the clinical efficacy of the system. We wanted to see whether it could be deployed in a school environment and whether it would sustain usage over an extended period of time. This section outlines the participants in this study, the process used to deploy the system, and type of evaluation data that we wanted to gather.

Participants

There were 11 children with upper limb impairments due to cerebral palsy at the schools (one at each school, except for two schools with two such children each); they were the target users for the system. These comprised eight males and three females, with a mean age of nine years (SD = 1 year 11 months). Given the two-player nature of the system, other children, without CP, at the participating schools also played the games. But for this study, we were only interested in how much therapeutic exercise the children with CP received via the system. It is these children’s usage data that we discuss in this paper.

We asked all children recruited for the study to provide written consent after we presented an age appropriate explanation of the study. All children were told that they were free to withdraw from the study at any time. The children all received age appropriate information sheets as did the parent(s)/guardians of the children. Ethical approval was granted by Leeds (East) Research Ethics Committee (REC reference 09/H1307/48).

Process

We deployed the four systems in the following three stages:

Each school received one system during each deployment. The spare systems in Stages 2 and 3 remained at the University. Each stage comprised two 4-week phases separated by a minimum of three weeks to ensure that therapeutic effects from the first deployment did not influence performance on the second deployment. In one of two phases, we deployed the system in single-player mode; in the other phase, we deployed the system in two-player mode. The phases were counterbalanced such that five of the schools received the system in the two-player mode first, and the other four schools received the system in the single-player mode first.

Our goal was for the children to use the system approximately 30 minutes a day. This was an important target. We discussed this goal with the head teacher of each school before the school agreed to participate in the project. We also discussed it with school staff on two occasions: once when they first agreed to participate in the project and again upon delivery of the system. We provided participating staff with a copy of the information sheet supplied to parents when consenting for their children to participate in the study. This information sheet stated that children would use the system for up to an hour a day. Initially this was met with some disbelief by school staff, but we explained that the hour was a maximum (included to avoid problems with ethics compliance should sessions inadvertently exceed 30 minutes), and that 30 minutes was the target. The staff accepted this with some relief. It is worth noting, however, that we did not provide the staff with any written indication of the 30-minute target, and with hindsight, including this in the written information may have been beneficial.

On delivery of the system to the school, we asked each school to designate a teacher to supervise the system (except in School H, see Table 1, where the designated staff member was the Special Educational Needs Coordinator, rather than a teacher). We showed the supervising teacher how to use the system: plug in the system to the power supply, start the PC, start the real-time sub-system software (icon on the desktop), move the haptic interface joints to set positions, and then start the game sub-system (icon on the desktop) software. After this initial setup, the onscreen software instructions guided the users. We demonstrated the emergency shutdown procedure, and then demonstrated how to restart the system if it did not initialize. We left an instruction manual with the system. The instruction manual included contact numbers to reach project team members if any problems occurred. We pointed this out to the supervising teacher or staff member as part of the initial briefing when we delivered the system. This represented some conflict with the desire to deploy the system without any input from our team, but we felt that this represented the kind of technical support that a school might receive in practice. It was better for problems to be fixed and noted and to identify any other barriers to using the system rather than have the system sit unused for four weeks because of a straightforward technical problem.

We gathered evaluation data from three sources: a record of any calls received from the schools to report problems or to request assistance, usage data stored on the systems for each child during each phase of the trial, and feedback questionnaires completed by the participating children and their teachers. As a result of difficulties in getting time in teachers’ schedules for interviews in the early phases of the project, we agreed with the participating staff that the most effective way of gathering information was to provide feedback questionnaires on paper. We provided a stamped, self-addressed envelope so that the school could return the questionnaires to the researchers at their convenience, rather than attempting to arrange face-to-face feedback interviews.

 

Previous | Next