Frustrated by Substack search? Try this instead.

I found a viable workaround that lets me & others easily find articles across one or more Substack newsletters. Try mine, and learn how to make your own.

The problem: Substack Search sucks.

I’m loving many things about Substack. Their site search is NOT one of them. Examples:

❌ Search can’t find posts even when I know the exact title or URL.

❌ Search can’t look across just two newsletters with one search query.

❌ Search often can’t find posts I know are in a specific newsletter.

❌ Search often can’t find people, even when I know the exact spelling of their name in their profile. I’ve typed in names and gotten search results with different spellings and name variants before it shows me people with the actual name I entered (if it ever does).

I can tell that the Substack team has been working on Search. It’s gotten better at finding people I’m already subscribed to. But I want all of this to be easier. And I want it to be easier for my current and potential followers and subscribers to find my stuff, regardless of which of my publications it’s in.

So I’ve been experimenting with workarounds. I’ve found a search solution that works for me and may help y’all too. And:

• Setting it up doesn’t require using any AI.

• You don’t even have to write code to build it.

To learn what I did & why, read on! All of the ‘how’ is in the End Notes. If you just want to try playing with the search solution I found, jump to here.

---

What’s Wrong With Substack Search?

I usually read LinkedIn on my mobile phone. I often find out about new-to-me Substack writers through their LinkedIn posts.

• An article URL can’t be pasted into the Substack mobile app.

• Clicking the link in the LinkedIn post won’t open that URL in the Substack app. The LinkedIn app always opens the App Store to Substack, and the URL gets lost.

The logical way, then, to find the writer and read, follow, or subscribe on my phone is to open the Substack app and search for them. But searching in Substack, for the name of the writer or the title of the post, almost always fails.

And the search options are weak in general. Here’s what I got tonight when I tried to search in Substack for my two newsletters … nothing relevant in either case. “OR” simply doesn’t work as logic.

Substack search attempt from the top bar of the browser UI

Search attempt from the browser UI bringing up the page with additional options, such as choosing Publication. Still no joy.

A Workaround

One solution: Create my own custom Google Programmable Search Engine (PSE) for my Substack newsletters.

In brief: Anyone can define a custom Google search engine that searches only the sites they specify. And it works. It finds stuff that Substack’s native search doesn’t.

I could achieve the same objective by bringing up Google search myself and specifying (search terms) + site:site1.substack.com OR +site:site2.substack.com + … every time I want to find something.

But that’s a pain, right?

Once a PSE is set up, which isn’t hard at all, this approach is way easier and more mistake-proof. And it’s shareable with anyone.

Bonus: I’m currently building a new author website (external to Substack). My custom PSE will combine search results from my new author site AND my two Substack newsletters. It will be a simple and seamless way to give my readers an integrated view of my writing across the two platforms!

The Maybe Slightly-Bad News

A few caveats I noticed while building my own PSE:

• To create custom Programmable Search Engines, you have to use a Google account that isn’t administratively restricted. A garden-variety Gmail account worked for me. A Google Workspace account associated with an enterprise didn’t (which makes sense).

• In order to embed the widgets for this custom search into a website you control, you have to verify your ownership through the Google Search Console page. Five verification methods are supported. The best way, if it works, is injecting a verification TEXT or CNAME entry into the DNS configuration managed by your domain provider. Not everyone may be able to do this, though.

• If you have more than 10 newsletters (sites) that you want to search across, you might run into some problems. Google warns about this. Search might just be slow, not broken. (Most folks have at most 1 or 2 newsletters, though. Even

  Russell Nohelty
  and
  Michael Spencer
  only have 5 each ;)

• Unless you pay them, Google says they will include ads in your search results.

The Good News

Good news 1: Link to it and run it from anywhere

These custom search engines can run in a separate Google-hosted page. Just share a link to the “Public URL” for your custom search engine, anywhere you like. So even if you can’t complete the DNS verification for your own website, or you don’t have one, you and others can still USE your custom search engine!

If you have your own website, just embed your Public URL in a search hyperlink on any webpage you can edit. If Google can verify your ownership, you can embed the PSE widgets into one of your pages (more on this below).

No external website? No problem. Here are 3 ways you can link the custom search page to your Substack newsletter, via the Settings:

1. Add the Public URL as a Search link to the “Homepage links” in your right sidebar. See steps and screen shots in ¹ below.

2. Add the Public URL as a Search link in the top “Navigation bar”. See steps and screen shots in ² below. This option is probably most convenient for our online readers.

3. Add the Public URL as a Search link to the email header or footer that goes to your email subscribers. See steps and screen shots in ³ below. This seems best for people who mostly read email and don’t use the browser or app.

Good news 2: Branding

You CAN reuse your Substack theme colors and wordmark to configure the Google-hosted search results page. The design’s not fully controllable, but the Settings cover the basics (colors & logo image). See ⁴ for steps and screen shots.

If you have more than one newsletter with different branding (as I do), just pick one or improvise a new image that combines them.

Good news 3: Search Power

Best of all, the search actually works. It finds stuff, and it’s pretty fast. And you can use all of the powerful standard Google search options to refine your query.

What I Did

Here’s what I did and how you can do it, too. (If you just want to try playing with the custom search engines I set up, jump to here.)

First: Simple PSE for 2 newsletters

I went to programmablesearchengine.google.com, logged into my Google account, and configured a new PSE for my two Substack publications by following these instructions.

Screen shot of the ‘Sites to search’ part of the setup for the Programmable Search Engine

I started with a built-in Google theme (‘Greensky’), then I set my own colors. Initially, I reused my Substack brand wordmark image from

6 'P's in AI Pods

as the logo on the Results page. Fair warning: This ‘Look and Feel’ settings part of the UI is kind of clunky. And the different ‘Layout’ options are a bit confusing (and perhaps irrelevant when not embedding widgets into your own site). But it worked.

Here’s an example results page, from searching for ‘diversity’ with my custom search engine. (Note: colors in the search results section are customizable, too; I just hadn’t messed with that part of the configuration yet.)

And here’s an example results page from searching for ‘music ethics’ with the same custom search engine:

Second: customization and testing

Experimenting showed that the Google search results include the “sitemap” pages Substack automatically generates for each year (example). I want my search results to take people directly to articles in one click. So I added PSE exclusions for the sitemap URLs.

I haven’t decided yet whether to exclude some or all newsletter section pages (e.g. the home page for the Substacking section this article is in).

Screen shot of URL exclusions for the auto-generated Substack sitemap pages

Next, I created a simple combined ‘wordmark’-sized image that included logo style elements from both of my newsletters and a search icon from Flaticon ⁵. I reconfigured my PSE to use it as the logo, instead of my 6P wordmark image (in the screen shots above).

Finally, I tested the engine with more queries. I tweaked it a bit to turn off one of the default web autocomplete options ⁶. While I was typing my search text, it was showing me too many choices that I knew weren’t relevant; I found it distracting to use. I may experiment with this more.

Since I’m using a free Google Search account, I was a bit concerned about Google’s ads potentially being intrusive. So far, they haven’t been. Most of my web results pages seem to have no ads. I’ve seen only a few ads on the image results pages. (I’m not sure how useful the image results are, anyway; keeping it enabled for now).

Bottom line: This PSE solves 3 of the 4 problems I listed above (❌).

Third: Finding people on Substack

To try to solve the 4th problem, I created a second custom search engine for scanning all of Substack, to help me find people. I enabled the Safe Search option and then tested it by searching on my own name and other people’s names. At first, I didn’t configure the colors at all; I just used the default Shiny theme and added my own square logo image which combined that Flaticon search icon and the Substack icon.

Here’s what I got when I searched for my own name.

1. Top hit was my profile page.

2. Second hit was my self-named newsletter.

3. Third hit was the Business section of this newsletter.

4. Fourth hit was a recent Note I wrote.

Oddly, the default colors of the Refinement buttons above the results summary were not readable. Other than that, these results looked good.

Screen shot of results from using my custom engine for searching all of Substack for my own name.

Next, I tried a partial name search for

Amanda Rose Fadely

, leaving out her middle name. The top results seem on point:

1. Amanda’s profile,

2. My interview with her,

3. Amanda’s newsletter

   Read the Instructions
   , and

4. her post about her book becoming an international best seller.

Screen shot of results from using my custom engine for searching all of Substack for Amanda Fadely

Then I tried searching on someone who uses only her first name on Substack. The person I wanted to find, the

Mariam

who writes on

AI Brews

, was the third hit. Unless you’re searching for someone you already follow or subscribe to, this search result has been impossible to get in the native Substack search (I’ve tried).

Screen shot of my results from using my custom engine for searching all of Substack for Mariam (first name only)

Bottom line: This custom search engine solves the fourth problem (❌). It not only finds the person’s Substack profile and newsletters, it surfaces articles by or about them in other newsletters.

Bonus: Any Substack writer could use this search URL to find anything on Substack associated with their own name. I plan to keep my all-of-Substack search engine available publicly as long as web traffic (for loading the logo image) doesn’t overrun my ISP bandwidth cap. [edit: I found a way to avoid this risk; I’m pointing the search engines to the image files in this post, now that it’s published.] It’s easy to make your own PSE and focus it to your own newsletter(s), though, as I hope this post shows.

Caveat: Stating what I hope is obvious … unless you’re a wealthy philanthropist, if you pay Google per search query (e.g. to remove ads), don’t create a similar search engine configured for a site that’s not yours, and freely give away the search engine link. You’d be stuck paying the Google bill for everyone’s searches of sites and content that won’t benefit you or your business. However, if your Google account has administrator rights at a nonprofit, school, or government agency, you might be able to remove ads for free:

Screen shot of the Ads removal section of the PSE configuration console

Fourth: Embedding the widgets in my own site

For testing purposes, I created a third PSE for my new author website.

To succeed at embedding my new custom PSE in my website, I had to verify my ownership of the domain. I did the TEXT and CNAME verification steps in the Google Search Console. Even after allowing more than a day for DNS sync, it still didn’t verify successfully. So I used 2 other verification methods, and got it working almost immediately. See (⁷) for how I verified my author website domain. The ownership rights you get when using these other methods are supposedly more restricted. But my site is simple, and I haven’t yet run into any limits.

When I’m ready to take my author site live, in a few months, I’ll add its URL to the PSE for my newsletters. Voila: readers will be able to easily find topics no matter where I post them!

Try Them Out!

Here’s a link to my custom newsletter search engine. It’s configured to search my posts for “ethical AI”; try searching for whatever interests you!

Click to try Karen’s custom search engine for her two Substack newsletters

And for fun, here’s the second custom search engine I created for scanning all of Substack. The link is configured to search for my name; try it with your own, or whatever you want to find!

Click to try Karen’s custom Substack search engine

What’s Next?

Google’s Programmable Search Engine has lots of features I haven’t explored yet which could make this even more useful. There are settings for Refinements, Promotions, Query Enhancements, Autocomplete, Page Restricts, and more. For instance, it allows adding your own synonyms, categories, and filters. So I could add categories for:

• “AISW” to let people search for “AI, Software, & Wetware” interviews, or

• “music” to find only my posts on ethical AI for music, or

• “diversity” to surface posts relating to inclusion, ageism, and supporting women in STEM.

Refining my PSEs is on the “to do” list. For now, I’m thrilled that it works. (And it gives real search results - not questionable generative AI overviews.)

I would love to try creating a custom search engine for a specific set of people, such as all of the women and nonbinary folks writing about AI on Substack. That list is well over 100 people now, though, and Google warns about performance when using this tool for more than 10 site URLs. So it might not work. I may give it a try later.

What do you think of these custom search engines for Substack?

• Would you use one of these search engines on someone else’s newsletter?

• Might you create one for your own newsletter(s)? (If you do, please share it in a comment or restack!)

• Do you know of a better solution for searching Substack newsletters?

Thanks for reading Agile Analytics and Beyond! I’d love to hear if you found this article useful. (Shares, hearts, comments, & restacks are awesome 😊, and one-time tips or voluntary donations via paid subscription are welcome & appreciated!)

For more Substack-specific tips like this, subscribe (FREE) to the “Substacking” section of this newsletter. (Most of my initial subscribers in 2024 were email-only, so I don’t opt new subscribers into this section by default.) You can manage section subscriptions here.

---

The remainder of this article is all about HOW to create and use your own custom Programmable Search Engine, step by step. (If you’re reading in your email client, this article may be truncated; click here to read the rest in a browser.)

End Notes - How To’s

How to add a custom search engine link to your newsletter layout

1. In Settings, go to the Website section and click Edit to the right of Homepage links.

Screen shot of newsletter Settings in the Website section, to add a Homepage link for search

After you click Edit and add your search link, the Settings will look like:

Example screen shot showing Karen’s custom search engine link added to her Homepage links for 6 ‘P’s in AI Pods

Browse to your publication home page and find your Homepage Links (the page doesn’t show that title above them). Depending on your layout, people may need to scroll down to see this section, but it’s there. With my 6 ‘P’s in AI Pods layout, it’s in the right sidebar near the bottom. The sidebar will look something like this:

Screen shot showing the 6 ‘P’s in AI Pods newsletter sidebar with the Search link I configured in the Homepage links section of Settings

How to add the custom search engine to the top navigation bar of your newsletter

2. Go to your newsletter Settings. In the Website section, click the Add button next to “Navigation bar links”. The new link will be added to the bottom of the reorderable section of your navigation bar links.

Screen shot showing how to add Navigation bar link with the URL for the custom search engine

I moved the new link to the top of the reorderable part of the list. It showed up right after Notes & Chat:

Screen shot showing the Search link next to Chat in the top navigation bar for the 6 ‘P’s in AI Pods newsletter

I repeated these same steps on this newsletter.

Screen shot showing the Search link next to Notes in the top navigation bar for the Agile Analytics and Beyond newsletter

Now any curious person or current/potential newsletter subscriber or follower can easily search for posts across both newsletters, from either of my newsletter sites!

How to add the custom Search link to your newsletter email header or footer

3. In Settings, go to the ‘Email header & footer’ section and click Edit.

Screen shot of the Emails section in Settings for editing the headers and footers

Click the Edit button on the right side of ‘Header for free subscribers …’.

With the Substack editor, add a hyperlink (use the chain link icon in the toolbar) for Search. Example:

Screen shot of adding a link to custom search in the email header (same process for free & paid, or for footers)

Click Save at the bottom when done. Copy or repeat in the ‘Header for paid subscribers’. Your Email headers settings will now show something like:

I repeated this for the email headers of my second newsletter (this one):

Now my email subscribers will have the Search link at the top of each emailed article. (In fact, if you’re a free or paid subscriber with email enabled, you’ll see it in the email you get with this article.)

How to configure the custom search engine’s colors and logo

4. Go to the Programmable Search Engine console, open your search engine’s settings, and click “All Look and Feel settings”:

Overview for Look and Feel. Click “All Look and Feel settings” to change or customize.

Choose from six default Themes, or customize:

6 default themes for PSE configuration

To reuse your Substack theme, put in the hex color codes under Customize. (Click the down-arrows to expand each section and edit the colors or font).

Screen shot of some of the theme customization options

Screen shot of the expanded Customize dialog. Click the small circled down-arrows to put in your own hex color codes. Click Save at the top when done expanding and editing all of the elements.

Regardless of whether you use a built-in theme or customize your colors, you can add your own logo to the Google-hosted search page. (Default is no image.)

Screen shot of the PSE configuration settings for using your own logo on the Google-hosted search page. Click the pencil to edit each option.

Image credit

5. The search icon from Flaticon (one of my favorite ethical shoestring image sources) was created by NajmunNahar.

6

How to turn off an Autocomplete setting that’s more annoying than useful

Screen shot of the Autocomplete settings in the Programmable Search Engine configuration console

How to get verified as owner of a website, for embedding custom search widgets in its pages

7. Example error page from Google Search Console

I tried domain verification first, and got this error right after adding the DNS TXT entry (before it propagated).

Error dialog from Google Search Console for validating the DNS ownership

I waited 24 hours and tried again - no luck. Then I tried adding a CNAME entry to the DNS. It still didn’t work. So I tried two different verification methods. Both worked for me right away:

• uploading a special Google-provided HTML file to my site’s home directory, and

• including a special META tag inside the site home page HTML file.

After completing these two verifications, my Google Search Console showed:

Screen shot of my Google Search Console after successfully verifying with home page META tag and HTML file upload.

Within a few minutes after these two verifications finished, the custom Google search widgets I had embedded on my author site test page worked. (Unsurprisingly, the search widgets don’t work when testing the page locally, before uploading it to the site I verified.)

Example screen shot of the custom search engine embedded in my author website (in development & testing). The light tan background shows where the embedded search widget sits in my light green page background.

---

Thanks for reading Agile Analytics and Beyond! I’d love to hear if you found this tutorial useful. (Shares, hearts, comments, & restacks are awesome 😊, and one-time tips or voluntary donations via paid subscription are welcome & appreciated!)

For more Substack-specific tips like this, subscribe (FREE) to the “Substacking” section of this newsletter. (Most of my initial subscribers in 2024 were email-only, so I don’t opt new subscribers into this section by default.) You can manage section subscriptions here.