SEGM Isn’t Interested in Science

A closer look at the organization’s response to a study showing high satisfaction and low regret among top surgery patients.

by Evan Urquhart

A recent study that followed up with transgender patients who had received top surgery years prior found very high satisfaction, and low regret. This result, though quite similar to other studies of post-surgical outcomes for transgender patients, received wide coverage in the mainstream press which engendered an instinctive pushback from opponents of transgender healthcare. One of the most sophisticated efforts to cast aspersions on the study came from the Society for Evidence Based Medicine, a group that claims to advocate for high quality healthcare for gender dysphoric youth free of any political, ideological, or religious motivation, but exists to oppose gender-affirming care regardless of the evidence.

SEGM is not part of the medical mainstream, and has ties to conversion therapists and religiously-motivated opponents of gender affirming care. Their schtick is to present a superficially science-y rationale for opposing mainstream treatments for gender dysphoria, and to the casual observer they can seem totally legit. That’s why we thought we’d take a closer look at their response to the study finding low regret among top surgery patients and dissect exactly how they do this thing they do.

On Sunday, SEGM published what they called a “critical appraisal” of the JAMA-published study “Long-Term Regret and Satisfaction With Decision Following Gender-Affirming Mastectomy.” Their subheading claims the study suffers from a “critical risk of bias” which is where we’ll start. After a couple paragraphs describing the study and its reception in the press, SEGM writes this:

screenshot from SEGM

Sounds concerning! So, uh, what’s a ROBINS-I analysis, anyway? ROBINS-I is a tool for assessing risk of bias in non-randomised studies of interventions. The JAMA paper isn’t a randomized study. Top-surgery is an intervention. It certainly seems superficially plausible that SEGM is evaluating this study in an appropriately smart science-y people way.

Except.

ROBINS-I is intended to help gauge how far from a randomized ideal study (which may be impossible or unfeasible to actually conduct) any particular non-randomized study diverges. It helps researchers evaluate how much trust to have in non-randomized results that seek to show the effect of an intervention. But the top-surgery study under discussion doesn’t measure the effects of an intervention.

It measures how satisfied (or regretful) people who got an intervention were.

Science, it turns out, doesn’t only do one thing. Often a medical study will measure the effects of an intervention compared to another intervention, a placebo, or no intervention at all. Other times, science just measures something in a population, like how many spots an average cheetah has, or how much regret top surgery patients at a particular clinic experienced 2+ years after the fact. You don’t control the spotted cheetahs with a control group who had no spots to determine how much difference having spots has, and you don’t ask how much surgical regret someone who never got surgery had.

SEGM acknowledges this, sort of, much later on. They complain that the study didn’t look at mental health outcomes among the patient group, just satisfaction and regret. [insert SEGM2]

screenshot from SEGM

Of course, it’s very important to measure whether transgender treatments improve mental health and overall functioning, and many studies have sought to do so. However, the potential for regret has been introduced by opponents of medical transition as a key concern and a driving factor behind legislation banning gender-affirming care for youth. SEGM’s website has raised the specter of regret numerous times. Studies of regret seek to directly answer those concerns.

But SEGM doesn’t want their concerns answered. It just wants the concerns. It wants to use concerns about regret to taint the process of building an evidence base supporting gender-affirming care, and it won’t accept results that show regret is very low.

SEGM’s claim that the JAMA paper suffers from a critical risk of bias is based on a misapplication of a tool designed to evaluate a completely different type of study is important because of how prominently SEGM placed this charge. But it’s not the only criticism SEGM leverages, so it seems fair to quickly run through the rest. For example, the response complains that a long-term follow-up after an average of 3.6 years isn’t long-term enough. In this criticism it cites two studies that looked over a much longer period and also found very low regret rates.

As other critics have, SEGM also brings up the JAMA paper’s response rate of 59 percent. This would be a perfectly valid concern to have over a single study but becomes less persuasive in the context of every other study of transition regret finding similar rates.

Another criticism leveraged by SEGM is that a complete lack of reversal procedures among the patient group is not the same thing as a complete lack of regret, which seems like it would be more convincing if they hadn’t also studied regret, and found it to be very low. A complete lack of any patients seeking reversal is just one more data point that tends to support the overall picture that most patients are satisfied with their results.

They also bring up the fact that this study only looked at adults. This is the one criticism that, if SEGM was really an apolitical, non-ideological group of concerned medical professionals, you would expect. It’s included as the fifth of six “key issues” and falls roughly three-quarters of the way down the page.

Very few minors in the U. S. receive surgery as a treatment for gender dysphoria. However, in cases of a young patient with an urgent medical need, it’s normal for doctors to look at treatments that have proven successful for adults and consider whether it might be the best available option for the young person involved. That SEGM is more interested in providing a plausible-sounding but ultimately inapplicable tool as a pretense to undermine a finding of low regret in adults speaks volumes about the real priorities and methods of this faux-scientific activist group.