Previously, no one had ever done a survey to look at rates of autism among adults so to do this a research team needed to develop an approach. Surveys are going on all the time around us. They seem to be in the news every day—for example, opinion polls on who should be the next leader of the country—so we all know a bit about surveys and lots of us have been a respondent in one. If you are a survey specialist there will probably be no need to read the next few paragraphs and indeed you may prefer to read the technical scientific reports on our work (Brugha et al., 2009, 2011; Brugha, McManus, et al., 2012; McManus, Meltzer, Brugha, Bebbington, & Jenkins, 2009).

Any scientific or technical topic can be daunting and confusing so we are setting out a brief introduction to survey methods here. If you would like to read more, an experienced survey expert colleague and I have put much of what you need together in a recently published chapter in a public health textbook (Brugha & Meltzer, 2008). More knowledgeable readers can skip over much of the next section.

Let’s start with the two questions we were trying to answer with our survey. One, how many people have the condition and, two, what are the characteristics of people who have it (e.g., gender difference and likelihood of having been exposed to a possible environmental or genetic cause)? The ideal way to answer these kinds of questions is to ask them of everyone, as is done every 10 years in the UK Census. However, this is not a feasible solution for many more complex and detailed questions. So we do a survey—which means we select a much smaller group of people from the whole population and just ask our questions of them. But how can we be sure the answers for our group also apply to the whole population? We can’t, but the answers are more likely to be representative if everyone in the whole population, that means you and me, has a known chance of being in that group (we call the group a probability sample, or a purposive sample). We achieve this by taking a random sample: that means drawing up a list of “everyone,” for example an electoral roll, and choosing a group of people from that list on the basis of chance (using a computer program to speed things up considerably!). Compared to a convenience sample, such as persons responding to an advertisement, a randomly chosen group like that is far more likely to be representative, with about the same proportion of men and women, old and young, working and not at work, as that found in the population as a whole. (Indeed, in a survey, we compare the characteristics of the sample group with the census and where the two differ we make adjustments for those differences by weighting the responses).

Before saying more about the survey approach, let’s not rule out the idea of a census completely. For example, if you want to know how common autism is among children aged 8 years old in a city or a county, you could, with permission, examine all health and educational records on 8-year-old children for information suggesting that a child might have the condition of interest. And then you could ask a qualified health professional to examine those records more closely, or, more expensively, you could actually examine those children to find out which ones have the condition.³ The CDC studies, referred to at the start of this chapter, do something quite like this. Child records are examined in defined geographical areas, although the children themselves are not re-examined. The problem with this approach is that it is only as good as the records—and indeed only as good as the services are at deciding which children should have a health check or an educational test that is accurately recorded. The fact that the number of children with autism found by the CDC researchers varies so much from one place to the next suggests that the quality of the records, or the services, is very variable between places. This puts some limitations on the value of statistics derived in this way, as the CDC authors have spelt out carefully and in detail.

So, returning to the survey sampling method and how to choose a group at random from the population—what are the advantages and drawbacks there? The big advantage is that the survey goes directly to representatives of the whole population—making no assumption about the quality of existing records (except for the quality of the list [sampling frame] from which the survey sample group is drawn). So when it came to studying adults, where there are known to be very poor records of who is affected by autism (National Audit Office, 2009), it was a “no brainer”: we had to choose the survey, with a direct assessment method. Since no one had done this before we had to invent specific methods for surveying autism in the adult population. Now that we’ve done it, lots of people have asked how we did it when no one else had managed to do it before. It’s a story worth telling—if only in the hope that others will now do the same in other populations and improve on our approach!

By now you will have realized that there is a survey cost issue. Examining people for a condition is a lot more expensive than checking records or asking people easy to answer questions about themselves. A medical or psychological examination for a complex condition like autism is very expensive so how can it be done at a reasonable cost? The answer is that we conduct the survey in two phases, and examine fewer people in the second phase. The first phase in our survey included screening questions for autism, and was based on interviews conducted face to face in people’s own homes. In the first phase we selected a large group of people and an interviewer went in person to their home and asked them many questions the answers to which might indicate that they have autism. But these questions are not enough to establish whether they definitely have autism: that requires a clinical assessment. But it is too expensive to assess everyone in that way as it takes a long time and can only be done by a specially trained interviewer.

By taking the survey sample method to a second phase, we resample from our first phase survey sample. Having taken a sample from a population and asked those willing to answer questions about their health in phase one, we then chose an even smaller subgroup to agree to a more detailed clinical examination, in a second phase, at a later date.⁴ And to choose for the second phase a balanced or unbiased sample, we chose for the clinical examination some who look more likely to have the condition and some who do not. A tricky part of this is how to work out how likely someone is to have the condition. If this were the common cold or the flu you could just ask “in the last week have you had a cold or felt like you had the flu?” But most people would struggle with the same question if you replaced the words “cold” and “flu” with “autism” and “Asperger syndrome!” How we tackled this we’ll come back to later.

Now that we know roughly when and how to select a sample we need to consider how large the survey needs to be. For this we need a statistician who will turn this question around and ask you what exactly is the question that you are trying to answer? Let’s think of another example to illustrate this. In an example “leader of our nation” question a good answer is we want to know which candidate is ahead; but a better question might be to ask is: “is the most popular candidate far enough ahead in the voting population to win the election?” Thus, because it is an opinion poll of about 1,000 adults and not the election itself, we need to know if the front-runner’s lead also exists in the population at large or whether it just happens that lots more of his supporters were sampled by chance. A second poll on a different random set of 1,000 electors on the same day might show a 1 % difference but the other way around. 1,000 may not be enough if we are interested in looking at small differences.

So what questions do we want to ask about autism in the adult population? If it is to ask how many adults have autism that could imply an interest in making a comparison with the frequency of this compared with other conditions; or it could be that an estimate is needed that is precise enough to inform the planning of services and to help to decide what they would cost to deliver. Or the question we might want to answer could be: “does autism affect as many older people as younger people?” For example if the true rate of autism is rising, as surveys of children over recent decades seem to suggest, you would expect fewer cases in older than in younger adults. This turned out to be the question that was of the greatest public interest when the results emerged, although we should emphasize that our study was not designed to answer this question specifically. Given the general interest in this question it proved fortunate that the adult survey began with over 7,000 willing respondents which provided some degree of precision to answer the question—“is autism associated with age?” Had our survey been much smaller we could not have done this.