Internet Surveys E-mail Surveys A Discussion of Web Surveys

I attended the November 1996 Market Research on the Internet conference and found it to be very informative. Below is a synopsis of what I heard at the conference, mingled with my own opinion on the value, use, and future of Internet surveys.

There are two main types of Internet surveys being conducted at this time -- e-mail and web-based. This tip deals with e-mail surveys. Our next tip will address web-based surveys.

E-mail Surveys

Correct netiquette says that you do not "spam," that is, send unsolicited e-mails in large numbers. This presents a challenge in terms of obtaining a sample representative of the population being measured. Some researchers are attempting to do e-mail surveys by asking people who stop by particular Web pages if they would be interested in participating in future e-mail surveys. This presents an element of self-selection bias, that is, of the respondents rather than the researcher determining who is eligible to participate in a survey. At this time, all Internet surveys suffer from this problem, some to a lesser extent than others.

Perhaps the best use of e-mail surveys is in measuring the opinions of customers or employees who communicate using e-mail. That may sound strange, or it may sound intuitive, depending upon your point of view. I say that customers make a good target because it is not bad netiquette to send an unsolicited survey, e-mail-based or otherwise, to a customer. It is important that these customers communicated by e-mail, otherwise you will be in a position of reaching conclusions about the total sample of customers based upon the segment who communicate (with you) by e-mail. If a large portion of your customer or employee base uses e-mail, an e-mail survey may be a correct approach for you to consider. However, there are other limitations to be understood and overcome.

Key among these is the size limitation of an e-mail survey. As many of you know, large e-mail files are difficult to download given the current widely-available technology. E-mails exceeding 15k are impossible for some to download. Thus, at this point in time, e-mail surveys should be limited to subjects that can be handled with a simple questionnaire format.

Bad e-mail addresses are a further problem. Even recently compiled lists of e-mail addresses can contain 10% to 20% "undeliverable" addresses. Some of this is caused by movement of people to different e-mail addresses; some is caused by software difficulties. I expect these difficulties to reduce with the passage of time.

At this point in time, there are difficulties in taking the data from the e-mail responses and entering it into a database. Many respondents will change the format of the text surrounding their answers, thereby playing havoc with most automated data recording programs. You will need to allow for manual checking of all data entry, at least until technology catches up with the data-entry needs of researchers.

The technology allowing people to easily implement e-mail surveys will be here very shortly. This presents additional problems. If too many people implement e-mail surveys, people will have more of a tendency to ignore them, making samples less representative for all e-mail researchers. Also, if amateur market researchers begin implementing surveys because e-mail makes it easy to do so, erroneous conclusions could be drawn and reported, damaging the credibility of the market research industry.

If you want to conduct an e-mail survey because you believe it will not take much "people" time, you may be in for a big surprise. You will receive responses from people asking you questions, to which you will feel obligated to respond. If you are perceived as having "spammed" people, you will have a public relations nightmare on your hands. Data entry and scrubbing will take quite a bit of time. You also have to allow for extra analysis to adjust for demographic differences between the sample and the population being measured.

As with all Internet surveys, you will need to be certain that none of the questions in the survey are correlated with the survey delivery mechanism. For example, you wouldn't want ask people in an e-mail survey whether they preferred communicating by e-mail or regular mail, because those who do not use e-mail will not have their answers represented.

The proper way to determine the methodology to use in a survey is to first determine the objectives and information needs. It is a severe error to determine the methodology before going through this process. Too much emphasis on quick timing and cost reduction can result in the use of an inferior survey methodology. E-mail surveys certainly are not immune to this problem. As the number and type of people on the Internet increase, e-mail will become an appropriate methodology in more circumstances. Researchers should regard e-mail as one additional tool in their arsenal of methodologies, and should use it when it is the most appropriate tool to gather representative, unbiased information.

Do you own or manage a small business, or a department of a large business? Check out our special E-Value2003 Employee Satisfaction Survey, specially designed and priced for businesses with 1000 or fewer employees.