Why I won’t be using Pangram AI scoring

TL;DR Biases, profits, IP protection, lack of consent, and my answers to the 5 questions I asked my subscribers

On July 21, Substack Team announced the integration of Pangram “AI detection” scoring into the Substack platform on our longer Notes, comments, and articles. I’ve been seeing lots of commentary on Pangram integration since then. This article is to summarize the issues and to state clearly where I stand.

(Thanks to everyone in my 6 ‘P’s in AI Pods subscriber chat and SheWritesAI subscriber chat, JHong, Code Like A Girl, Cheyenne Dominguez | M(AI)VENS, Vanessa, Dee McCrorey, Effrosyni Paza, Kim Doyal, Caroline Vrauwdeunt, Reality Re-Thunk, Soul Hacked AI Labs, Lisa van der Linden, Sharon Greene, Valentina, Sam Illingworth, Rebecca Mbaya, Shannon Kimberly Edwards, Hessie Jones, The Raven Scholar, Rebecca Wicker, Leanna Bishop, Jessie Mannisto, Ida-Emilia Kaukonen, Tumithak of the Corridors, Her Majesty of Ink and Exits, Daria's Tech Musings, and many more for their thoughtful posts.)

Biases in AI-based detection of AI writing

Pangram claims very low false positive rates (1 in 10,000). But many people challenge those claims (e.g. JHong), pointing out that their numbers are from lab tests and are based on analyzing pre-COVID writing. Others have pointed out detection biases which have much greater false positive rates for certain groups (by race, ethnicity, neurodivergence, second language writers, academics). Still others have pointed out that women are disproportionately penalized for using AI, compared to men, so this kind of scoring is more likely to harm women writers than men writers.

How Pangram profits

JHong uncovered that Substack will be paying Pangram based on the number of items they scan and score. That means that by using Pangram scoring features here, we are contributing to funding their work.

Using Pangram also means giving them access to our writing data. For those of us who have opted out of allowing AI bots to scrape our newsletters, this means that any time we or anyone else click the button for Pangram to Scan for AI text, we are giving Pangram access to that article. By testing how Pangram scoring worked on my Notes reply to JHong (below), I gave them all of those words. ☹️ Those who go further and use the Pangram option to add a Create “How I make this” statement are giving Pangram even more free data. There’s almost zero chance that Pangram won’t use this data to do further model training and refinement.

Screen shot of Substack Pangram result dialog showing the 3 categories of scoring: AI, AI-assisted, or Human. This is the scoring result for a Notes post I made in reply to JHong.

[Update] And if you choose to challenge a score you think is inaccurate, via Report detection error, guess what? You are giving Pangram more data they can use to refine their models. That’s what “Reinforcement Learning from Human Feedback” (RLHF) is.

Screen shot of the options under the … menu for Pangram scoring, including Report detection error and Disable detection.

[Update] Substack team member Ben Cohen shared a link to the Pangram privacy policy and stated that it meant Pangram won’t use our data. I am skeptical of that interpretation. My questions (not answered as of this update) are:

I’m not a Pangram customer. I’m a Substack customer. Pangram is Substack’s supplier or partner. What does Pangram’s agreement with Substack say about use of our writing data?

And that privacy policy says “we use outside datasets to train our system”. Is all of Substack now an “outside dataset”?

My position on Pangram

Fundamentally, I believe we the people deserve choices. What’s right for me might not be right for someone else. So I’m not going to say that Pangram should be removed; some readers and writers might find it of use. I am going to say that we deserve to have more control over whether it is used on our writing or not.

The current ‘choices’ require that we have to expose our writing to Pangram scoring before we can opt out of the scoring being used or shown. That’s not a meaningful choice for many of us. It’s not informed consent. It’s a catch-22.

I have chosen to opt all of my newsletters out of AI training here on Substack for various reasons, including the trust my interview guests have placed in me to protect their discussions with me, and respect for collaborators who have also opted out of AI training. I do not want Substack to make it low-friction to allow any random person to override my opt-outs by sharing my writing with Pangram.

Based on the July 22 Pangram feature set in Substack, here are my answers to the questions I posed about Pangram in my subscriber chats, on the choices I am making:

1. Will you use it on what you READ here?

NO.

• As I’ve said repeatedly, I’m not interested in judging or shaming people for using AI.

• I don’t trust the tools to be accurate and fair — especially to people of different races, ethnicities, neurodiversity, academic training, ESL backgrounds, or women. So I don’t want to be a party to discrimination by triggering a writer to be unfairly scored by Pangram.

• Publishers who may have opted out of AI training on their writing shouldn’t have their work exposed to Pangram’s AI against their wishes. (Over the past year, about 1/3 of our SheWritesAI community members have opted out, as have I.) I won’t do that to them.

• Since JHong uncovered that Substack will pay Pangram for each use, I don’t want to drive more profit to them.

2026-07-23 updates:

• Via Steve [Sage-Outlaw-Caregiver]: Substack updated their Publisher Agreement on July 20 to refer to Pangram and to state that they will not use our writing data to train AI models. It’s still not clear whether that promise applies to partners such as Pangram.

• Via Kim Doyal: Apparently Substack is picking up the tab for the API calls to Pangram. Financial terms between the two companies are still unclear.

• See Carey Lening’s excellent article on potential legal ramifications, such as GDPR violations, from Substack’s use of Pangram integration. Others are also raising concerns about potential defamation claims from Pangram false positives.

2. Will you check it on what you WRITE?

I’ll check a few this week as tests, as I did above on my Note to JHong, then NO. I don’t want Pangram profiting from my work, either by Substack paying them $ per post or by me ‘paying’ them with access to my writing data (or inflated user counts that prop up their company).

3. Will you block it on your own articles?

YES, if I can find a way to block it without having to run the Pangram scanner on my article first. Primarily as an act of solidarity with others who face biases in these tools that I don’t, and to try to stop Pangram from profiting from my work.

If Substack refuses to provide a way to block Pangram access to an article without running Pangram scoring on it, then I’ll have to trust that my audience will respect my wishes on protecting my IP from AI and won’t use it either, at least not on my writing.

As I commented in my follow-up note to Ben Cohen, I am not worried about being lumped in with slopmongers who might also want to block Pangram. Anyone who is reading my work on AI ethics isn’t likely to misread me blocking Pangram as meaning that I AI-generate slop. They will read it as me being consistent about believing and advocating for consent and IP protection.

2026-07-23 update: Now that we have a way to block Pangram scoring on our articles without having to run it, I will definitely be blocking it on all of my text articles. (There is no blocking option on audio or video articles because Pangram won’t run anyway.) Others, such as Code Like A Girl, are also committing to block Pangram on their posts. Join us? Let’s make blocking an AI detection tool as normal as not smoking in restaurants. Keep the Substack air cleaner. 🚭

4. Will you add explanations through Pangram on how you use AI, or elsewhere?

I already posted my AI use policy publicly here more than 2 years ago. I expect that anyone subscribing or following me knows where I stand and what I do. And I don’t want Pangram compensated for my efforts by me posting explanatory notes through their scoring tool. I’ll update my policy to link to this explanation of why I don’t use Pangram.

AI without AI 🗣️

Inconvenient Principles: No (unfairly trained) AI images 🗣️

5. What do you think Substack should do differently here?

Let newsletter publishers opt out of any Pangram scanning on their writing at the profile level and at the newsletter level. Not this per-item crap that requires us to allow Pangram to scan it before we can block it.

2026-07-23 update: It’s now possible on newsletter articles (not on comments, though) to opt out of Pangram scanning on the article without having to run the tool. See this note from J.M. Gooding. After you proceed to the final page before publishing your article, scroll to the bottom to look for Text Analysis. There is now a Disable AI detection button below the Scan for AI text button.

Screen shot of the two buttons at the bottom of the page for submitting a new article for publication.

This still requires us to opt out on every single article. It’s a start. But (1) we don’t yet have this option on our comments, and (2) we still deserve to have a newsletter-wide opt out. Keep protesting!

What We Can Do

You all know by now that I try to never rant without proposing something we can DO, right? So here you go. Let’s use the agency we have to make a difference.

Thanks to JHong connecting with Ben Cohen on the Substack Team, I’ve asked that they consider implementing these newsletter-level and profile-level opt-outs. If you support opt-outs, please chime in here or here (Substack) or here (Pangram).

Substack’s announcement about Pangram integration, for reference: [link]

---

Updates since initial publication on July 22

2026-07-22

• Added more names and links for more people whose comments contributed to the thoughts in this article

• Added caption to screen shot

• Added a second screen shot showing the options for reporting a score or disabling detection, along with a paragraph on how this data supports RLHF

• Added Pangram privacy policy link and some comments about why I’m not convinced it protects our writing

2026-07-23

• Added info from J.M. Gooding on how to opt out an article from detection without having to run Pangram on it, with screen shot

• Added info from Steve [Sage-Outlaw-Caregiver] on the July 20 update to the Substack Publisher Agreement about Pangram and (non-)use of our data for training AI models

• Added info from Carey Lening about possible GDPR violations from this AI scoring tool

2026-07-24:

• New response from Substack: https://on.substack.com/p/how-writers-are-reacting-to-substacks (found via Stuart Miller; also see his article)

• July 21 video interview by Substack CEO Chris Best with Pangram founder Max Spero, where they talk about the tool: https://cb.substack.com/p/pangram-founder-max-spero

• Articles by Luis Campos AI Engineer on the legal aspects of Pangram, Lisa Kostova on reclaiming our cognitive agency, Dr Sam Illingworth on witch hunts and fragility of the AI detection, Lisa-Marie Cabrelli, Ph.D. encouraging us to opt out of participating in the witch hunts, Kim Doyal on building assets instead of fretting over AI tool use, Sara da Encarnação on the new blue pencil hiding behind the 3 dots, and Monica Hebert on comparing AI tools for writing to paintbrushes.

2026-07-25:

• Articles by Rebecca Mbaya [link] pointing out that Substack has prioritized this tool integration above global inclusion for all writers to be able to get paid, and Daria's Tech Musings [link]

• Added my own “How I make this” statement when I replied to a Note from a writer asserting that false positives “aren’t a threat”: