While driving the beltway last year, you may have noticed road signs with the Washington Nationals’ Curly W (see photo). On first impression, this seems like a great idea. The symbol would presumably aid in recognition while the highway was congested with hurried fans trying to make that first pitch. However, while the ‘Curly W’ logo blankets the ballpark and Nationals merchandise, it can’t be found on any road signs. So what happened to all those Curly W’s?!
In a Washington Post article from February, Dan Steinberg investigated this and uncovered some interested findings. The D.C. Department of Transportation (DDOT), conducted a survey with ballpark attendees to determine if ‘the Curly W’ assisted drivers in getting to the ballpark. The results from this survey were inconclusive so DDOT tasked the Federal Highway Administration (FHWA) with a follow-up human factors evaluation. In Dan Steinberg’s interview with the FHWA, the agency reported that “the addition of the graphic logo (‘the Curly W’) to the word legend on the sign took observers longer to process, requiring longer glance times to the signs containing the graphic logos. These longer glances did not necessarily allow for an unfamiliar user to comprehend the meaning of the graphic logo.” The human factors evaluation also found that the sign did not assist unfamiliar drivers and further, for familiar drivers, ‘the Curly Ws’ did not aid in comprehension.
I think it’s great that DDOT took a risk and attempted to help drivers get to the ballpark more safely. And then they actually evaluated the sign to ensure that it was meeting their objectives. Despite this, DDOT could have done research up-front to determine if the ‘Curly W’ was going to help drivers. This initial small up-front cost could have saved the taxpayers money and the DDOT from scrutiny.
Many parallels can be drawn from this story to usability testing. At its core, I think it stresses the importance of testing design changes before they happen. For example, making a change on a website or app may at first seem like a great idea. In this case, the addition of a ‘Curly W’ to a road sign seems very cool. However, until the change is tested with actual users, there is no evidence that the cool idea will actually work. Unfortunately, well intentioned organizations with great ideas often make design changes that do not work well for their users and cause issues. The organization may react and be able to fix the issue quickly; however, it is not always clear what needs to be done to correct the problem. And reacting to the problem is more expensive than identifying it up front. In this case, DDOT had to shut down highway lanes to cover up or remove the ‘Curly W’ from the signs.
If a problem is recognized with a website or app, a ForeSee survey can help to collect initial data and provide preliminary findings. But just as with DDOT’s initial survey with ballpark attendees, the survey results are often limited and inconclusive.
Bottom line: Save yourself money (and a potential headache) and build usability testing into your design plan and process, and avoid the trappings of the cool idea.