Cambridge academic critiques Govt research on Conversion Therapy

My attention has been drawn to the empirical basis for proposed legislation in favour of criminalizing the provision of therapeutic services to those with unwanted feelings of same-sex attraction or ‘conversion therapy’ (hereafter, “CT”). 

It is my understanding that the current discussion regarding the decision to criminalize CT is predicated on two pieces of research: the first conducted by the Ozanne Foundation Faith & Sexuality Survey 2018 (hereafter, “FSS”) and second by the Government Equalities Office’s National LGBT Survey (hereafter, “NS”).

It is clear that neither has been subject to any kind of empirical peer review.  I have therefore undertaken a careful examination of the methodological design used for the FSS and the NS.

By way of brief summary, the inadequacies in survey design mean that neither the FSS nor the NS is close to being commensurate with reputable empirical studies in the social sciences.  Both surveys suffer from three crucial methodological flaws:

1. Significant ambiguities with respect to the definition of CT
2. Value-laden terminology
3. Non-representative sampling

These methodological flaws are interlocking and co-dependent, compounding the problematic nature of this research.  In sum, it is difficult to see how any social scientist could support the findings and conclusions made by this research in good conscience.

1. Definition of ‘Conversion Therapy’

Neither survey provides a methodologically adequate definition of what constitutes CT.  NS refers to CT as ‘it’ in three of the four answer options in Q142-143.  That is a clear violation of construct validity, a methodological principle which requires the observable indicator to represent the underlying construct researchers are seeking to measure.  It is therefore not at all clear that ‘conversion’ is what is being measured, since respondents would have been free to interpret the term as myriad of activities that could not be considered a form of ‘therapy.’ 

The definitional problem is exacerbated by the use of the term ‘so-called’ in references to CT in Q142-143, Q144, and Q145 in NS.  That means CT could have been expansively interpreted by respondents to include prayer, fasting, or any number of other harmless religious practices.  Direct evidence for this comes from Q28 of FSS, where researchers included under the definition of CT ‘private prayer,’ ‘prayer with close friends,’ ‘fasting,’ ‘deliverance ministry,’ ‘emotional healing,’ ‘counselling with official religious ministry,’ and ‘other.’  Of the 458 participants in the FSS, 282 (61.6%) listed ‘private prayer’ or ‘prayer with close friends’ as the form of CT they had undergone.

The indeterminate definitional scope of CT as a term undermines any conclusions one could sensibly draw from research that deployed the term in this way.  It is not at all clear that CT as it is being measured in FSS and NS would correspond to CT as legislators propose to define it in the relevant legislation.  Presumably any legislative definition would exclude from the scope of the relevant statute the kinds of harmless religious practices that both FSS and NS include within their respective applications of CT as a term.  

2. Terminological Problems

The second methodological flaw relates to the loaded and value-laden language used liberally in both FSS and NS.  

Put plainly, both FSS and NS framed its questions using the kind of loaded language that is likely to have encouraged respondents to provide answers that they would not have provided had the language not been used.  On the Ozanne Foundation website, the introduction to NS states:

‘Working alongside Humanists UK their survey aims to unearth the types of practice that are still prevalent in the UK and the impact that this has on an individual’s mental health … Today, Conversion Therapy has gone underground – it takes place ‘in the dark.’  This research will help us illuminate its prevalence and discover where it is still taking place.’

The clear connotation of the terms used in this programmatic paragraph for the FSS is plainly not neutral, since it explicitly depicts subterranean forces ‘in the dark’ conspiring to damage the mental health of members of the LGBT+ community, when the language could and should have depicted CT from an impartial perspective that left open the possibility that it did not damage their mental health. 

Q142-143 of the NS states:

Whether respondents had been offered, or had received, so-called ‘conversion’ or ‘reparative’ therapy in attempt to ‘cure’ them of being LGBT.

The use of quotation marks for the term ‘cure’ imputes bad faith to CT as a practice, plainly conveying to respondents the impression that CT pathologizes feelings of same-sex attraction.  The use of loaded language is likely to have have created a social desirability bias where respondents may have provided answers they knew would conform with the thinly disguised attitudes of the organisations conducting the surveys. 

This methodological flaw distorts the accuracy of responses by inscribing the biases of the researchers into the questions.  Consequently, neither FSS nor NS could reasonably be considered accurate measurements of the views of the respondents.

3. Sampling Problems

Both FSS and NS used non-representative samples.  Within social science research, the central objective of any empirical study is to obtain a representative sample in order to produce results that can be extrapolated to the population at large.  This goal is typically achieved through random sampling where the demographical categories comprising the overall population are included via random selection.  In other words, the characteristics of the sample must resemble the characteristics of the population.

The use of purposive selection methods in both FSS and NS produced samples that were unlikely to resemble the true population distribution of characteristics of the LGBT community.  To this extent, each survey suffers from a self-selection bias where participants actively volunteered to participate.  Any social scientist would rightly be sceptical of any results produced by self-selected samples for the simple reason that individuals who actively volunteer to be part of a sample are inherently different from those who do not volunteer.  In many cases, self-selected participants are likely to be more invested in the relevant issue than those who do not self-select.

Since FSS was conducted online, respondents were almost by definition people invested in cultural and political issues involving same-sex attraction.  Such people could not reasonably be considered members of a randomly selected group for a survey seeking to measure attitudes among the general LGBT population towards CT.  Moreover, it is likely that disproportionately many members of the LGBT population with a pre-existing hostility towards CT would have volunteered to participate. 

In the case of the NS, respondents were self-selected from those attending “Pride” celebrations in 2017.  Remarkably, the NS itself admits that ‘the sample was self-selected, and there is no guarantee that it was representative.’  Given that both surveys contained non-representative samples, the views expressed in the responses as a whole could not reasonably be considered an accurate reflection of the experiences of the wider LGBT community.  The accuracy of the results of the FSS and the NS, and any conclusions based on those results, could not and should not provide any basis for primary legislation to be passed.

4. Ancillary Concerns

There are other concerns with the FSS and the NS that should be noted.  

First, it is unclear when and where CT was provided to respondents.  Indeed it is possible, given the ambiguity of the term, that respondents may have been referring to forms of CT provided many decades ago that are no longer practised today.  This is especially plausible given that the NS reveals that 5.4% of those above age 65 and 3.5% of those between the ages of 55 and 64 received CT, the highest figures among all age demographics featured in the NS.  

Second, it may be worth noting the NS suggests that the decision to ban CT is likely to disproportionately restrict the rights of religious minorities, since 9.8% of British Muslims and 8.3% of British Hindus respectively report receiving CT.

Finally, of the 107,850 individuals that participated in the NS, only 2,588 (2.4%) admitted to receiving CT.  However, given the ambiguity of term it is virtually impossible to determine the actual number of recipients.  If we base an estimation on the 38% of participants from the FSS who received some form of CT that was not prayer, this number goes down to 983 or 0.9% of the sample of 107,850.  Given the serious methodological flaws already identified, it seems imprudent to pass primary legislation based on a sub-sample smaller than 1% of the total sample.  It is difficult to see how such legislation could be devised using statistical outliers that may or may not reflect the views of the general LGBT+ population.

Given the nature and extent of these flaws, it is my recommendation that no conclusions could responsibly be drawn from these two surveys for the purposes of policy-making in relation to the indisputably controversial question of CT in the UK.