design, strategy, code, schtuff

design and development of content, sites, apps, experiences, logos, what-have-you

Service Design and Change Management for an International Non-Profit

Highlighted Skills Used:
Ethnographic and Remote User Research
Process Design/Business Strategy
Team Management
Service Blueprinting

Interaction and Interface Design


Context: A non-profit, whose servitized product was delivering digital scans of books, online, for free, and accessible in as many formats, to as many people as possible.

The Challenge: There were no standards for quality of digital images - not in evaluation process or in process training and evaluation. Additionally, standards in image quality for digitization technicians varied greatly between individuals and between digitization center locations. Standards needed to be established, implemented, trained, and evaluated - it was affecting the customer experience. People would call, email, and leave comments on specific book scans noting how frustrating it is to get 40 pages into a book, only to find that every page thereafter was too blurry or too dark to read, or that pages 41-42 were inadvertently not captured.

Research Gathered/Methods Used: Spending nearly three years embedded as an ethnographic researcher among the Quality Assurance (QA) team, I performed the QA Technician's job, as those were the users for whom I was trying to understand in order to build a standardized, easy to learn and use process. This included performing "QA batch checks", and starting group chat rooms for the QA techs for ease of gathering user needs and pain points in the moment. A number of remote usability tests with the QA staff around the globe were performed, utilizing eye tracker software in order to learn about and then teach the optimal method screen-scanning method.

Design Process: The new QA process was tested with users (QA techs), refining best methods of clarifying nebulous concepts like "how dark is too dark to read?" and standards around when to reject a batch for quality reasons and what the ramifications of that were. A new process and system for performing QA and evaluating the new skills of the QA staff, and a dashboard of push data for management, including ROI on the new processes were all built out in proof-of-concept. All of these were tested with QA techs, and iterated on over the course of the roughly 34 months spent on this project.

Result: The new quality standards, and methods for evaluating QA staff were implemented. The suggested new process and systems for creating a better team member experience may have been implemented at a later date.

Measurements of Success: Upon implementing the new quality standards and spending a few weeks training all the scanning staff - scanners and QA techs - in those new standards and what to do about them, all 15 QA staff were re-trained and had passed their evaluations with100% correct evaluations. This service design engagement ultimately resulted in a series of process, tool, and training changes for team members. But these more tangible changes were merely the tip of the service redesign iceberg. The changes to process, training and evaluation of quality assurance staff, in turn, resulted in both better quality images for users, as well as an easier, less costly on-boarding process for new quality assurance team members. The clearer quality guidelines filtered to the scanning technicians, who were able to optimize their own processes, resulting in fewer quality issues needing to be flagged by quality assurance technicians, ultimately to be rescanned. Since this resulted in fewer books needing rescanning due to poor quality of scans, while the quality defect rate went down, raw production – pages scanned per hour – increased by nearly 100% over the course of 18 months. Taken holistically, this resulted in shaving approximately $0.02 off operational costs per page scanned, which equated to a savings, on average, of 30% per page scanned.