Skepticism about the 2011 Korean Study

In The Politics of Autism, I discuss the uncertainty surrounding estimates of autism prevalance.  Jessica Wright reports at SFARI:

A widely cited 2011 study that pegged autism prevalence in South Korea at a whopping 2.6 percent is inherently flawed because of the assumptions the researchers had to make, suggests a new analysis of the data1. The findings have implications for estimates of autism prevalence anywhere in the world.

Intense controversy surrounds estimates of autism prevalence, which have been rising steadily over the past 30 years. How researchers collect their data can have a huge impact on the numbers, and may even account for the rapidly rising rates.

The 2011 study garnered attention for its two-phase design, in which researchers screened more than 20,000 children for autism and then clinically assessed a small proportion2. They extrapolated these findings to South Korea’s population to arrive at their estimate.

Three-quarters of the children the study tagged as having autism did not have a prior diagnosis and were attending mainstream schools. Flagging children based on signs noted in medical and school records — the standard method in the U.S. — may have missed these children.

But the two-stage design is also flawed. The screening method the researchers used is not perfectly accurate, and the results can be skewed by which parents choose to participate. Taking these limitations into account, the analysis, published 29 June in Autism, says the 2011 study’s range for autism prevalence could have been at least 2 to 5.4 percent.

This numerical spread is too wide to allow researchers to pinpoint any single rate with certainty, says lead researcher Dan Kennedy, assistant professor of psychological and brain sciences at Indiana University in Bloomington.

“The estimate can be misleading if incorrect assumptions are made,” Kennedy says. “If we are overconfident in the assumptions we make, then we end up with an estimate we believe, but probably shouldn’t.”

Korean American Outreach and Cross-Cultural Adaptations

Roy R. Grinker and colleagues have an article in The Journal of Autism and Developmental Disorders titled "Cultural Adaptation and Translation of Outreach Materials on Autism Spectrum Disorder."

The abstract:

In order to connect with families and influence treatment trajectories, outreach materials should address cultural perceptions of the condition, its causes, and postdiagnostic care. This paper describes the cultural adaptation and translation of the Autism Speaks First 100 Days Kit into Korean for the purpose of improving autism spectrum disorder (ASD) diagnosis, assessment, and interventions. The goal of this study is to describe a methodology for future cross-cultural adaptations and translations of outreach materials on ASD, using the Autism Speaks First 100 Days Kit as an exemplar. The research involved two stages of qualitative interviews: unstructured individual and group interviews with 19 Korean child health and education professionals in Queens, NY, followed by structured cultural consensus modeling interviews with 23 Korean mothers, with and without children with ASD, in Queens, NY and the greater Washington, DC area. We conclude that a systematic approach to cultural translation of outreach materials is feasible. Cultural consensus modeling yielded information about numerous barriers to care, had a demonstrable effect on the translation of the kit, and was efficient when employed with coherent segments of a relatively homogeneous population and focused on a single condition.

From the discussion:

Stigma—a negative evaluation of the child—was a sub-text throughout the research, because interviewees stigmatized people with ASD, or the parents of someone with ASD. One woman, when asked to provide a list of words or ideas that came to mind when she thought about children with developmental problems, said, “I feel sad because there is no bond between the child and the mother and father.” In addition to “lack of a bond between parents and child” (coded here under the category of poor childrearing), the additional terms she included in her cards were “stubborn,” “egotistical,” “disobedient,” and “unstable.” Stigma was also related to the whether a developmental problem or functional impairment is a medical or non-medical problem, with the latter being more highly stigmatized. Deafness, for example might cause only mild shame, while autism would suggest that, in the words of one mother, “the child’s needs are not being fulfilled.”

Outreach to Korean Americans

Earlier posts described a study about the prevalence of autism in Korea, and the reluctance of Korean parents to address the problem. The New York Times reports on an effort by Autism Speaks to encourage Korean Americans to get help for autistic kids.

“More so than other populations, Korean-Americans really measure their own self-worth, and the worth of the family, in terms of what the child is able to achieve and what the child means to the family,” said Roy Richard Grinker, a professor of anthropology at George Washington University and the senior author of the South Korea study.

...

It is a crucial moment for autism across the United States. The number of children who receive a diagnosis of autism has been rising for years, without any consensus about why, other than increased awareness of the condition. At the same time, autism itself is being redefined: the newest edition of the country’s manual for mental disorders, released weeks ago, collapsed some categories of autism, including Asperger syndrome, under the umbrella of “autism spectrum disorder.” Some experts have predicted the change will lead to fewer diagnoses, and hence cuts in public spending on therapy and special education.

In New York City, the number of public school students classified as having autism this year, 10,199, or roughly 1 percent of enrolled students, is up 50 percent from four years ago, according to the city’s Education Department. Diagnoses among Asian students have also jumped. But while they make up 16 percent of the school system, they account for only 8 percent of those with autism diagnoses.

...

“We are trying to build a model, for outreach and facilitation, that would support immigrant families, minority families, to access services available from school systems and from cities and states,” said Andy Shih, an official at Autism Speaks who is managing the initiative.

As diagnoses of autism have become more common, some early intervention providers have taken advantage of the growth in public spending, and lax oversight, by billing for services that were not needed or never provided. Dr. Shih acknowledged that some businesses might “exploit parents scared and confused about how to best support their children.”

...

Unscrupulous providers are not the only potential pitfall. Young Seh Bae, 48, who leads a committee of the Korean American Behavioral Health Association and is the mother of a 16-year-old boy with autism, said she worried that a focus on Koreans, in both the South Korea study and the Flushing effort, could exacerbate stereotypes.

...
And though the study in South Korea was “rigorous,” Dr. Winston Chung, an assistant professor of psychiatry at the Geisel School of Medicine at Dartmouth, said it should be viewed carefully because the researchers used tools designed by a Western culture to measure children in an Eastern one. Typical behaviors in a “Confucian society,” where the norms for eye contact, gesturing social reciprocity and expressing oneself are “profoundly different,” and where the skill of nunchi — measuring someone’s mood and desires without speaking — is valued, could be misconstrued as autistic in some cases, he said.

Counting Cases of Autism

The South Korean study showed a greater prevalence of autism than many other studies. At the American Journal of Psychiatry, Tony Charman writes:

One reason why counting autism is so difficult is that when you set a high threshold for who has autism, the count is lower, and when you set a low threshold, the count is higher. There is also some evidence that culture drift in our understanding of what constitutes autism has changed over the decades in ways that are hard to quantify and study.

...

The authors describe how they examined whether measured child and school factors, such as grade, sex, and school size, affected participation at each stage, and broadly they did not. However, there are a host of possible unmeasured factors that might have influenced participation in such a way as to introduce bias—in the direction of overestimating prevalence. These include school and parent knowledge and interest in participating in an autism study and parental concerns about their child's development and behavior.

...
A final intriguing and perhaps more hopeful point is made in Kim and colleagues' discussion of how it is that so many pupils with an (undiagnosed) ASD can apparently manage to be educated and integrated in mainstream classrooms. This is a useful reminder that developmental conditions such as autism are affected by the environment in which a child develops and that accommodations to such environments can sometimes have a positive impact on children with a "disorder."

Korean Families React to the Korean Study

The Los Angeles Times reports on the families of the children whom the Korean study diagnosed with ASD:

"The majority were upset but understood," said Dr. [Young Shin] Kim, who led the study. But "some did not accept it and thought we were nuts."

Only three families were willing to talk with a reporter. Kim said several mothers had agreed to participate but their husbands would not let them, refusing to accept that anything was wrong with their children.

One mother said her son was a little weak on social skills but otherwise normal. "I believe that in Korean society, boys are often misunderstood," she said. "So it's harder for them to socialize." The researchers said the boy's diagnosis was Asperger's.

In several cases, children identified as autistic had already been found to have "reactive attachment disorder," a condition that resembles autism in that children struggle with social interactions. But unlike autism, which has a strong genetic basis, it is by definition caused by disregard for the child's emotional needs.

The diagnosis resembles "refrigerator mother" theories embraced in the U.S. decades ago: the idea that bad mothering made children detached and unable or unwilling to communicate. It inflicted enormous guilt on mothers. But in Korea, it is more palatable than the possibility that the condition was inherited.

"They don't want to see it as genetic, because it's damaging to the family," [coauthor Richard Roy] Grinker said. Instead, "the mother can take the bullet and say, 'I failed with this child.'

Prevalence and the Korea Study

At The Guardian, Oxford professor Dorothy Bishop writes about the Korean study:

In 1965, Leo Kanner, the father of autism research, expressed his concerns about overdiagnosis: " ... it became a habit to dilute the original concept of infantile autism by diagnosing it in many disparate conditions which show one or another isolated symptom found as a part feature of the overall syndrome. Almost overnight, the country seemed to be populated by a multitude of autistic children."

There followed a period of standardisation of diagnostic criteria. Psychiatrists worked hard to develop instruments to make diagnosis more objective, and cases of autism became less common and more clearcut. The pendulum, however, is now swinging back.

Earlier this month, a press release from Yale University announced: "Prevalence of Autism in South Korea Estimated at 1 in 38 Children". This estimate was considerably higher than recent UK studies (Baird et al, and Baron-Cohen et al). There are three possibilities: either children in South Korea are at unusually high risk of autism; previous studies have dramatically underestimated the prevalence of autism; or this new study gives an overestimate.

She suggests that we cannot dismiss the third possibility.

The South Korean study was not restricted to classic autism, but included the milder condition of ASD. To assess this, the authors used culturally appropriate versions of diagnostic interviews and observational instruments. Expert clinicians then reached a consensus diagnosis on the basis of information from these.

As I have discussed elsewhere, both interviews and observations are often conducted because the conclusions of either – when taken alone – are not watertight. For instance, the observational instrument only assesses how a child behaves during a 40-minute period, and so may miss behaviours that are rare or that occur in less structured settings.

These assessments can also over-diagnose autism, however. For instance, it is possible for children to appear impaired on the domains of social interaction and communication if they are shy, anxious, sullen, or have language difficulties. For all these reasons, it is seen as dangerous simply to rely on numerical scores to make a diagnosis.

TV Covers the Korea Study

On "The Today Show," Dr. Nancy Snyderman looks prevalence, with the Korea study as a news peg. (Note that she says "incidence" when she means "prevalence.") She also describes Asperger's as a distinct disorder without mentioning the pending change in the DSM.



On The NBC Nightly News, Robert Bazell also mixes up incidence and prevalence.

On CBS, Dr. Jon LaPook makes sense about the need for early diagnosis of HFA:

From KCTV in Kansas City:

KDVR in Denver:

Korea: One in 38

Previous posts have dealt with the question of prevalence: just how much autism is there? The New York Times reports:

An ambitious six-year effort to gauge the rate of childhood autism in a middle-class South Korean city has yielded a figure that stunned experts and is likely to influence the way the disorder’s prevalence is measured around the world, scientists reported on Monday.

But experts said the findings did not mean that the actual numbers of children with autism were rising, simply that the study was more comprehensive than previous ones.

“This is a very impressive study,” said Lisa Croen, director of the autism research program at Kaiser-Permanente Northern California, who was not connected with the new report. “They did a careful job and in a part of the world where autism has not been well documented in the past.”

For the study, which is being published in The American Journal of Psychiatry, researchers from the Yale Child Study Center, George Washington University and other leading institutions sought to screen every child aged 7 to 12 in Ilsan, a community of 488,590, about the size of Staten Island.

...

“From the get-go we had the feeling that we would find a higher prevalence than other studies because we were looking at an understudied population: children in regular schools,” said the lead researcher, Dr. Young-Shin Kim, a child psychiatrist and epidemiologist at the Yale Child Study Center.

South Korea was chosen not only because autism prevalence had not been measured there, but also because its national health care system, universal education and homogeneous population made it a promising region for a planned series of studies that will also look at genetic and environmental factors in autism.

The study, which was largely financed by the research and advocacy group Autism Speaks, raises the question of whether a similarly high prevalence would be found in the United States if all children were screened.

From the AP report:

A study in South Korea suggests about 1 in 38 children have traits of autism, higher than a previous U.S. estimate of 1 in 100. By casting a wider net and looking closely at mainstream children, the researchers expected to find a higher rate of autism characteristics. But they were surprised at how high the rate was. They don't think South Korea has more children with autism than the United States, but instead that autism often goes undiagnosed in many nations. U.S. estimates are based on education and medical records, not the more time-consuming survey conducted in South Korea.

Two-thirds of the children with autism traits in the study were in the mainstream school population, hadn't been diagnosed before and weren't getting any special services. Many of those undiagnosed children likely have mild social impairments, rather than more severe autism.