This article was originally written as part of an internal training manual. While a few of the statistics cited within have changed somewhat since the original writing, the content of the article still applies. We'll print this article in several installments:

1. Telephone Sampling
2. Mall Intercept Sampling
3. Mail Samples, Finding Low Incidence Respondents
4. Statistical Issues

Every study we do entails drawing a samplefrom a samplingframe which is part of a universeor population.

• The universe or population is the group we want to make inferences about - denture wearers, orange juice purchasers or whatever.
• The sampling frame is the sub-universe from which we select our sample - telephone households, shoppers at specified malls, persons on a certain list, etc.
• The sample is the respondents we end up with.

Technically, all inferential statistics apply only to probability samples, though this assumption is routinely violated.

Telephone surveys are the most common type of marketing research, partly because very good samples can be achieved at reasonable cost. We'll cover just a few highlights of sampling for telephone studies.

A. Random Digit Dialing Vs. Listed Sample

Telephone sampling used to mean sampling from directories. However, large proportions of households are unlisted (i.e., non-published).

• Nationally, 27.6% of households have unlisted numbers.
• In many cities the unlisted rate is over 50%.
• California is especially bad, with most cities over 50% (Los Angeles is 56% unlisted).

• Directories don't include people who want to be unlisted.
• Directories don't include people who've moved recently.

• Listing increases with age; over half of households headed by 18-34 year olds are unlisted and about 40% of the 35-54's are unlisted.
• Among people who have moved in the past two years, 59% are unlisted. This figure undoubtedly covers some who want to be unlisted, but for the most part these are persons who moved since the last directory went to press.
• Unlisted households are not higher-income. But, as one might expect unlisted householders tend disproportionately to be unmarried and to be renters.

The solution to the unlisted problem is Random Digit Dialing (RDD).

In RDD, sampling houses generate random four digit numbers in known exchanges, excluding "blocks" known to be unassigned (most are not assigned). For residential samples they also exclude blocks assigned to businesses. For a fee they'll also have a computer dial each number to detect whether or not it is a working number.

RDD assures that both listed and unlisted households are included in the sampling frame. Inevitably, however, not all the numbers are working and not all are households, so the efficiency is not as great as a listed sample.

B. Replicates

A sampling frame, whether listed or unlisted, usually contains at least as many "elements" (i.e. listings or people) as you will need to meet the quota of completed interviews. Which means that perhaps some elements will never be called. If so, you no longer have a random sample.

Commonly, samples are divided into "replicates," each a mini version of the full sample. Before a new replicate is "opened," the preceding one is worked to exhaustion. In this way, you can be certain that when your quota of interviews is complete the sample did not come disproportionately from one segment of the list.

C. Random Selection Within Household

Why go to great effort to achieve a random sample of households and then interview whoever answers the phone!

In many studies in which there can be more than one eligible respondent in the household, random selection is used to determine who should be interviewed.

There are various techniques for doing this:

• The oldest is known as the "Kish" technique (after its inventor, Leslie Kish), in which the interviewer asks for the names of all household members who meet the screening requirements and for their birthdays. The selected respondent is the one whose birthday was most recent.
• Another is the Troldahl-Carter method, which uses a set of forms with matrices for the selection of the oldest male, youngest female, etc.
• Another asks for respondents to be listed in order of age and uses a random number list to select which one is interviewed.

D. Callbacks

Whether a probability sample or not, it's common to specify a certain number of callbacks to reach not-at-homes, busy signals, etc. Most typical is two callbacks to reach a household and another two to complete an interview with the correct respondent.

Some clients (most notably the government) specify that a certain completionrate must be achieved (often 60% or more). Callbacks are the only way to achieve such a rate.

Various studies have investigated whether people reached on the first call differ from those reached on the second or third call. There are differences.

What do you suppose those differences are?

E. Answering Machines

Reportedly, about 25% of households now have answering machines.

However, while answering machines are a problem, Survey Sampling reports that they still have not made a major impact on the ability to contact households:

• In a recent large study, only 6% of first attempts reached a machine.
• With four attempts, 57% of machine households can be reached.
• Machine households are no more likely to refuse than non-machine households.

F. CATI Sample Management

Many CATI systems have sample management built in. The sample is provided on tape or disk. As a dialing is completed, the computer randomly selects a sample element and pops it up on the interviewer's screen.

The program can make sure it is in the right time zone for the call, and will keep track of the "tally sheet" for the job.

Some large systems have auto-dialers built in to pull up and dial numbers at specified intervals and, feed that dialing to the next available interviewer.

Do you own or manage a small business, or a department of a large business? Check out our special E-Value2003 Employee Satisfaction Survey, specially designed and priced for businesses with 1000 or fewer employees.