Subjective and objective results of usability testing of a prototype performance management system among US Navy personnel

Abstract

The U.S. Navy conducted usability testing for the Navy's Human Performance Feedback and Development (HPFD) and ePerformance system to capture both quantitative and objective data as well as qualitative and subjective data from participants to identify potential sources of error and user burden. A total of 57 officer and enlisted supervisors and nonsupervisors took part in usability testing conducted at 3 Navy test sites. Each participant completed a usability test scenario and took part in focus group interview debriefings of their usability testing session to elicit recommendations for improvement and cultural factors that might affect system implementation. Results identified various usability errors, their frequency, and duration, as well as recommendations for improvement. In addition, user ratings of the system and qualitative data from focus group interviews indicate that system users believe the systems worked well. Focus group interview results indicate that the frequency and severity of connectivity problems had a negative effect on user perceptions of the HPFD and ePerformance systems, and there were several cultural factors that were key for the acceptance of a new performance appraisal system. Implications of the findings, study limitations, and recommendations for future research are discussed. (PsycINFO Database Record (c) 2013 APA, all rights reserved)