Email Newsletter Usability

Nielsen Norman Group Report Email Newsletter Usability Executive Summary

		The most significant finding from our usability research on email newsletters is that users have highly emotional reactions to them. This is in strong contrast to research on website usability, where users are usually much more oriented toward functionality. Even a website that users visit daily seems to feel like a tool: users want to get in and get out as quickly as possible rather than “connect” with the site. Users tend to glance at websites when they need to accomplish something or to find the answer to a specific question. In contrast, newsletters feel personal because they arrive in users’ inboxes, and users have an ongoing relationship with them. Newsletters also have a social aspect, as users often forward them to colleagues and friends. The positive aspect of this emotional relationship is that newsletters can create much more of a bond between users and a company than a website can. The negative aspect is that newsletter usability problems have a much stronger impact on the customer relationship than website usability problems. For example, in one of our studies, a user received an error message that read “Email address is not valid.” This would be a poorly worded error message in any user interface, but the emotional aspect to newsletters increased the user's anger: “Mine’s as valid as the next person's! ... It's questioning my validity as an entity in cyberspace.” Sixty-nine percent of users said that they look forward to receiving at least one newsletter, and most users said a newsletter had become part of their routine. Very few other promotional efforts can claim this degree of customer buy-in.
User Research		To assess how people use email newsletters, we conducted three rounds of user studies, as well as pilot studies to refine the test methodology. In total, 93 users participated in our testing. Most participants were in the United States (in 12 states across the country), but we also studied users in Australia, Hong Kong, Japan, Sweden, and the United Kingdom. The first study focused on testing newsletter usability in terms of subscribing, unsubscribing, and maintaining the user’s account. For this study, we systematically tested 10 different newsletters, which we assigned to users to ensure that they hadn’t previously used a newsletter’s subscription interface. Most of the study was conducted as a traditional laboratory test: we observed users individually as they read newsletters and attempted to subscribe and unsubscribe. We conducted additional parts of the study remotely, through telephone calls. We conducted our second study remotely, using a diary methodology that allowed us a much wider geographical distribution of participants. The second study included all of our international participants and all of our U.S. users who were not on the East Coast. The first study had the benefit of systematically testing a set of design variations, with multiple users for each design. However, it also had the distinct disadvantage of people getting newsletters that they had not selected themselves. For the second study, we looked in detail at users’ experience receiving and reading newsletters that they’d already subscribed to on their own initiative. In total, the participants subscribed to 345 different newsletters, and we studied 101 of those. We studied users’ newsletter experience over a four-week period for most participants, and over two weeks in a few cases. This longitudinal approach allowed more emphasis on how people deal with incoming newsletters during their workday. We were also able to test many more B2B and intranet newsletters than we could cover in the first study, which mainly tested B2C newsletters. Of the newsletters received by the users in our second study, 65% were for personal purposes and 40% were for business purposes (users viewed 5% of newsletters as both personal and business, so we counted those twice). We conducted the third study using an eyetracker. Eyetracking let us record where users were looking on websites as they subscribed or unsubscribed. We also recorded how users looked at their inboxes and how they read individual newsletters. During the third study, we systematically tested 12 newsletters (using a controlled methodology to ensure that all newsletters were used evenly) and tested 40 newsletters in a less controlled manner (users were free to pick newsletters of interest from an inbox, so some newsletters were read much more than others). Finally, we tracked users’ eye movements as they read a total of 65 newsletters from their personal inboxes. By definition, each of those newsletters was read by only a single user. In addition to studying newsletters, the third study included a component in which people used a variety of RSS (Real Simple Syndication) readers to read news feeds. This let us compare the newer medium of feeds with newsletters, which are now an established media form. Finally, our third round of research included a field-study component in which we observed users in their offices during a normal workday. This ethnographic approach let us learn about the use of newsletters and news feeds in an environment with many competing information sources and demands on users’ time.
High Nominal Usability		Our test users experienced unprecedentedly high levels of task completion in their attempts to subscribe and unsubscribe to the newsletters in the study: 81% for subscribing and 91% for unsubscribing in our most recent study. Although high, these rates could still be improved. If, for example, a newsletter with 50,000 subscribers ensured that everyone could correctly operate its subscription interface, it could add an estimated 11,700 subscribers on average. Still, most usability studies find success rates of around 66% for other Web design areas. Clearly, relative to this, the newsletter usability success rates are incredibly high—even though they’re still lower than anything we would deem a truly great user experience. There are probably two reasons for the high success rates here. First, the tested functionality is very simple: Get on or off a mailing list. In fact, the main failures came on websites that complicated this functionality, such as by combining newsletter subscriptions with site registration. In general, it’s easier to design a simple user interface when the underlying functionality is simple. The second reason that the subscription process had much better usability than other Web designs is that newsletter designs are highly accountable. In many other Web design areas, project managers can delude themselves and their bosses that user-hostile designs, such as splash pages, offer some benefits. Create a design where people can’t find what they want and page views might even go up as users wander aimlessly before they leave (and give up doing business with the company). With a newsletter subscription design, users either subscribe or they don’t. In the latter case, websites will eventually tone down their design excesses and focus on simplicity, and subscriptions will increase accordingly. If a site were to replace a simple design with a complex one, it would soon notice a decline in new subscriptions and revert to the previous design, writing off the bad design as an expensive usability lesson.
Low Perceived Usability		Even though users successfully unsubscribed 91% of the time during the test sessions, they often refrained from even trying to get off mailing lists that they no longer wanted to receive. The four main reasons people didn’t attempt to unsubscribe were: Emotional attachment to the newsletter: Users said that it didn’t feel good to sever the relationship, even when they no longer read the mailings. Low expectations for the website’s usability: People assumed that it would be difficult and time-consuming to unsubscribe, so they postponed the job for another day and simply deleted the newsletter’s current issue. Fear that unsubscribing would fail and would subject the user to even more mail: Many people have heard that asking to get off spam lists only confirms the validity of their email address to the spammers; this notion has become an urban legend that contaminates users’ mental model of legitimate newsletter publishers as well. Easier options: It’s often easier to simply use a spam-blocking feature to stop future issues than it is to unsubscribe. Whatever the reason, it’s clear that mailing list owners shouldn’t assume that all subscribers actually want to receive their newsletters. Many users might have simply neglected to unsubscribe. Some newsletters deliberately make it difficult to unsubscribe by hiding the instructions or making them overly complex. The motive is probably to retain as many subscribers as possible to maximize the reach of permission marketing programs. In reality, however, you don’t have users’ “permission” once they stop wanting the newsletter, regardless of whether they jump through the hoops required to get off the list. If users keep getting unwanted newsletters, the messages will start to backfire and become regular reminders that they’re annoyed with your company. Better to let them go.
Speed Matters		In our latest study, the subscribe process took 4 minutes, and the unsubscribe process took 1.5 minutes. Even though these task times are not prohibitive, they’re much too long for the simple functionality involved. We recommend setting a usability goal of allowing an existing user to unsubscribe in less than a minute, assuming that the user has a recent copy of the newsletter at hand. New subscriptions should also take less than a minute when subscription requires only the user's email address. Even if additional information is required, users should be able to subscribe to free newsletters in less than 2 minutes. Only newsletters that involve a subscription fee should be allowed so many steps that the average user can’t subscribe in 2 minutes. Users are very demanding with respect to the efficiency of operations like subscribing or unsubscribing. For both tasks, we found extremely strong correlations between the task time and the users’ subjective satisfaction: r = -.63 and -.95, respectively. These correlations basically say that the slower the subscribe or unsubscribe process, the less people will like the site. For each additional minute it takes to subscribe, you will lose 0.3 satisfaction points on a 1 to 7 scale, and for each additional minute it takes to unsubscribe, you will lose 0.6 satisfaction points. As indicated by the numbers, users are substantially more critical of a slow unsubscribe process. Once they want out, they want out quickly. A perfect satisfaction rating of 7 would require instantaneous task performance according to the regression estimates. It seems impossible to create a design that allows users to subscribe and unsubscribe in 0 seconds, but that’s ultimately what users want. It's nobody’s goal in life to “manage subscriptions,” so any overhead becomes an annoyance. Extreme simplicity and ease of use are necessary to make a positive impact on customers.
Improving Usability		In our first study, the average time to subscribe was 5 minutes. Only four years later, the average time for this task had dropped to 4 minutes. It’s also faster to unsubscribe: this task time dropped from 3 minutes in the first study to 1.5 minutes four years later. These are substantial improvements in usability over a fairly short time, showing that companies are investing resources in advancing the newsletter user experience. The improvements in getting off mailing lists are particularly impressive. In part, this might be because companies have recognized that there are no benefits in continuing to annoy customers who don’t want their newsletters. But the emphasis on easier unsubscribe features is more likely due to legislation that requires companies to provide users with more information about how to stop receiving email.
Significant Platform Diversity		The Web is a fairly uniform environment. Almost all users have either Internet Explorer or Firefox, and almost all run these browsers on Windows. Yes, a few people use Macs and browsers like Safari or Opera, but each browsing environment offers pretty much the same features. The differences between Web browsing platforms are like the differences between Indian and African elephants, not like those between crabs and eagles. In contrast, email newsletters must contend with platform diversity that is much more like the biodiversity of the Cretaceous Period (before the comet hit). Although Yahoo! was the most commonly used email reader in our recent study, it accounted for only 31% of users. Eight additional platforms were represented, but people also commonly use others, including Eudora, Lotus Notes, and a variety of mainframe systems and Unix mail variants. Each email platform has a different way of displaying the From line, the Subject line, and the newsletter content. They also have different approaches to spam filtering and other things that influence the subscriber’s user experience. This diversity makes it crucial that newsletter designers test their subscribe and unsubscribe processes—as well as the actual newsletter delivery and display—on all major email platforms.
Spam is a Fact of Life		There’s a little good news (but mostly bad news) about the impact of spam on email newsletters. The good news is that users in our recent studies were better able to differentiate legitimate opt-in newsletters from unsolicited messages than they could in the past. In our earlier newsletter usability studies, users sometimes confused the two. Now, spam has a very prominent profile in terms of popular awareness, press coverage, and the sheer amount of it hitting inboxes. Users have thus developed a reasonable understanding of the spam phenomenon as opposed to simply being baffled about unexpected messages. The bad news is that the increased burden on email users has caused people to become even more stressed and impatient when processing their inbox. Users have less tolerance for newsletters that waste their time. We have also found that people often use their spam filters as a shortcut to eliminating newsletters they no longer want. Instead of unsubscribing, which users often view as too cumbersome, they simply tell their spam-blocker that the newsletter is spam. Voila, that newsletter no longer shows up in the inbox. The fact that many users will declare a newsletter to be spam when they tire of it has terrifying implications: legitimate newsletters might get blacklisted and thus be undeliverable to other subscribers who still welcome new issues. This is a compelling reason to increase the usability of the unsubscribe process: better to lose a subscriber than to be listed as spam.
The Battle for the Inbox		Users are getting pickier and pickier about which newsletters they’ll read. Some are purposefully cutting back on the number of newsletters they receive. These users view newsletters as being in direct competition with each other for a limited number of slots in the inbox. Users will unsubscribe from a newsletter or stop reading it—even if it’s good—if they come across a different one on the same topic that better serves their needs. People get a lot of email. They don't have time to read a lot of text. In our most recent study, users spent an average of 51 seconds on each of the newsletters they read from their own inbox. Users spent an additional 33 seconds on information found by pursuing newsletter links to websites. Not only do you have to compete with other newsletters to get people to subscribe, you must compete with all other email and get users to open your messages, pay attention to your content, and click through to your site. This has always been true, but the competition is becoming more intense as users are getting more swamped by online information sources than they were in the past. Email services are offering increasing amounts of storage space, often in the gigabyte range. This allows users to save more old newsletters than was possible in the past. On the one hand, having users archive your newsletter means that it becomes a form of permanent outreach and will show up when they search their personal information space. On the other hand, many newsletters might be saved and never read. It’s worth it to use informative and enticing subject lines that encourage users to read a newsletter while it’s fresh.
Scannability and Immediate Utility		The most frequent complaint in our study was about newsletters that arrived too often. And, when we let them vent, the most frequent advice our study participants had for newsletter creators was to “keep it brief.” Newsletters must be designed to facilitate scanning. In our first study, 23% of the newsletters were read thoroughly. In our third study, four years later, only 19% of the newsletters were read thoroughly. The drop in percentage of thoroughly read newsletters is a good indication of the increased volume of email that users have to process. The dominant mode of dealing with email newsletters is to skim them: that’s what happened to 69% of the newsletters in our most recent study. Of the remaining newsletters, users only glanced at them or at most read a few items. Eyetracking heatmaps showing how many users read each part of two newsletters. The areas where users looked the most are colored red; the yellow areas indicate fewer views, followed by the least-viewed blue areas. Sometimes users will simply skim the headlines to get an update or overview of what’s going on in the field covered by the newsletter. As one user said, “I like to keep up-to-date in the industry, but rarely delve deeper than the cover page.” Other times, users deliberately pick out those few elements that are most important to them and ignore the rest. As another user said, “I review the contents by company and only read the companies of interest to me.” Designing for users who scan rather than read is essential for a newsletter’s survival. Scannability is important for websites as well, but it’s about 50% more important for newsletters. This implies the need for layouts that let users quickly grasp each issue’s content and zero in on specifics. Content and writing styles must support users who read only part of the material. Newsletters must be current and timely, as indicated by three of the four main reasons that users listed for why a certain newsletter was the most valuable they received. All of the following four reasons were given by more than 40% of users: Informs of work-related news or company actions (mentioned by two-thirds of users) Reports prices/sales Informs about personal interests/hobbies Informs about events/deadlines/important dates There is pretty much a “what have you done for me lately” phenomenon at play, where newsletters have to justify their space in the inbox on a daily basis. Having been relevant in the past is not enough. Because of the immediacy of the medium, newsletters must be relevant today and address users’ specific needs in the moment. Because newsletters build relationships with readers and because it’s so easy to ignore individual issues, newsletters do get some leeway if they are predictably relevant at certain times. During those periods when a newsletter isn’t relevant to the user’s immediate needs, the user might simply ignore it rather than unsubscribe. For example, a speech pathologist at an elementary school said that she could only purchase new products at the end of the school year, and so ignored product-related newsletters most of the year. Still, she didn’t unsubscribe, and simply receiving the sales newsletters reminded her of the brand when she received her budget. Users will often avoid signing up for newsletters because they feel crushed by information overload. It is the job of the newsletter publisher to convince users that the newsletter will be simple, useful, and easy to deal with. A predictable publication frequency that is not too aggressive is usually best, except for newsletters that report breaking news. A regular publication schedule lets users know when to look for the newsletter and reduces the probability that they’ll confuse it with spam and delete it. Also, writing good subject lines is crucial, both in encouraging users to open the newsletter and helping them distinguish the newsletter from spam. We recommend including content from the issue in each subject line, even though it's a difficult job to write good microcontent within the fifty- to sixty-character limit that many email services impose.
News Feeds (RSS)		The first, and strongest, guideline about news feeds is to stop calling them RSS. In our most recent study, 82% of users had no idea what this term referred to. In general, it’s typically wrong to use implementation-oriented terminology, because most users don’t understand the underlying technology and don’t care about it. It’s better to use terms that indicate what the concept does for users, and “news feeds” does this far better than “RSS.” Some users were familiar with the general idea of feeds, even if they didn’t know the term “RSS.” This was typically because they were receiving feeds on their My Yahoo! page or a similar personalized portal. Users had very mixed feelings about feeds. Some people liked viewing information from multiple sites in a single centralized location instead of having to go to each site. Some users also liked scanning a list of headlines without seeing any content that they didn’t ask for. A final benefit some users appreciated was the ability to determine when they would go and view their news items. This is in contrast with newsletter arrival times, which users can’t control. On the other hand, many users had negative feelings about feeds. People who are already suffering from information overload resent having to go to yet another source of information. In contrast, email newsletters arrive in a tool that people already use, so they don’t add yet another thing for over-burdened users to do. Email is also easier to archive for later use, whereas feeds have an ephemeral nature. Several participants in our study had stopped using the feeds on their My Yahoo! page. Many previous studies have found that users are reluctant to spend time customizing portals, so it’s not surprising that some users simply decided to stop looking at that part of the page rather than edit their preference settings. Finally, some users resented the fact that news feeds are divorced from the context of the publisher’s website. These users preferred the serendipity that came from visiting a full-fledged website that offered options beyond the current headlines. News feeds are definitely not for everybody, and they’re not a replacement for email newsletters. Feeds can supplement newsletters for sites that cater to users who prefer a centralized view of headlines. These are primarily newspaper sites and other sites with a heavy focus on news and breaking stories, as well as sites that target Internet enthusiasts. For sites that target mainstream business users or a broad consumer audience, news feeds may be less important. Such sites might be better off emphasizing higher-quality newsletters and a choice of publication frequency. Also, our eyetracking of users reading news feeds showed that people scan headlines and blurbs in feeds even more ruthlessly than they scan newsletters. When you appear in somebody’s news reader, your site has a diminutive footprint that’s rubbing shoulders with a flood of headlines from many other sites. Under these conditions, users often read only the first two words of a headline, so it’s crucial to have brief headlines and to start them with the most information-carrying words. Feeds are a cold medium in comparison with email newsletters. Feeds don’t form the same relationship between company and customers that a good newsletter can build. We don’t have data to calculate the relative business value of a newsletter subscriber compared to a feeds subscriber, but we wouldn’t be surprised if it turns out that companies make ten times as much money from each newsletter subscriber. Given that newsletters are a warmer and much more powerful medium, it is probably best for most companies to encourage newsletter subscriptions and promote them over feeds on their website.
Future of Email Newsletters		Four years ago, in our first report about newsletter usability, we said about the future of email newsletters: “There may be none. Legitimate use of email is at war with spam, and spam may be winning.” Although four years is a very short period in which to assess big trends, we now believe that this assessment was too negative. Email newsletters are so powerful that the best of them do have a future, despite ever-more adverse conditions. Ever-increasing information overload is definitely making users reluctant to sign up for more email. And once newsletters arrive in the user’s inbox, they might simply be deleted as part of the ruthless mass deletion procedure aimed at the morning’s spam. Finally, as discussed above, fear of spam and other email abuse is keeping users from dealing rationally with newsletter subscriptions. When we asked users why they liked email newsletters, more than one-third highlighted the following three benefits: Email newsletters are informative and keep users up-to-date (mentioned by two-thirds of the users). Email newsletters are convenient and are delivered straight to the user’s information central; they then require no further action beyond a simple click. Email newsletters have timely information and real-time delivery. Newsletters that leverage these advantages (along with other points that users mentioned) have a stable future. But they must continually deliver specific benefits that help users with life or work issues in the here and now. Comparing email newsletters with other media, one user said: “Bottom line, I’d rather have it in an email newsletter than in the regular mail. I can click Delete if I don’t want it; I don’t have to throw anything away; and it is usually easier to unsubscribe if you don’t want to get anymore.” Convenience rules. This is one of the few times we have found that the virtual world was better and more convenient than the physical world. Usually, websites have such poor usability that they compare very unfavorably with real-world stores or in-person services and communities. In contrast, email newsletters have a very strong position.