Reverse engineering AI's 'train of thought'
Welcome to the brave new world of Generative Engine Optimization (GEO), same as the old world of SEO
Here’s a thing I’ve been doing for various clients, both current and prospective, and that I recommend you do as well: I pretend to be their ideal customer, and I start typing prompts into the various AI engines.
It’s not too dissimilar from pre-AI days, when I would mess around typing search queries into Google as part of SEO marketing I do for healthcare clients1, mostly physicians who have started their own practices, or taken on senior roles at larger companies.
But in the Brave New World of marketing to the AI overlords, I wanted to understand how these models arrive at their recommendations. I think in the near future, as people get more comfortable just speaking directly to devices that are optimized to run large language models, people will increasingly just ask for direction, and trust the recommendations.
Thus, if you’re a business that wants to master this new world, it’s important to understand what’s going on in the background. Currently, the AI tools are making an attempt at transparency by publishing a “train of thought” to go along with their answers. These serve as a kind of roadmap marketers can use to reverse engineer their place on the recommendation list.
For example, here’s a prompt from today I gave ChatGPT’s o3 model, its most advanced (you can see the entire conversation here):
I can’t tell you how gratified I was to see my long-time client, Dr. Angelo Falcone of Dignity Integrative Health & Wellness, at the top of the list.
As you can see, ChatGPT thought for 46 seconds before it recommended him (along with two other doctors in the area). Here’s what it says it did:
If you read this carefully, you can see that what o3 is doing is basically a glorified Google search. Of course, the real value add of AI is its ability to synthesize large amounts of information from disparate sources, while also giving you the ability to ask follow-up questions that take into account your very individual circumstances.
In this case, ChatGPT is searching the web, reading through a bunch of websites, and then pulling together an answer. Dignity Integrative shows up in those web searches (SEO job well done ✅), and you can see in the train of thought exactly which pages it’s going to, and what it’s reading on that page to get its information.
This was so gratifying as a marketing professional because a lot of this is content that Dr. Falcone and I specifically developed together to help attract SIBO patients. Just to break it down:
We added a static page to the site with that was optimized for the kinds of searches we expected (and that I identified via real-world keyword analysis, here: https://www.dignityintegrative.com/sibo-doctor
We also published a blog post called Finding a SIBO Doctor that aimed to educate patients on what to look for: https://www.dignityintegrative.com/post/finding-a-sibo-doctor-a-comprehensive-guide-for-patients
And, we continue to publish articles that zoom in on different aspects of how to manage SIBO.
Information from all of these showed up in ChatGPT’s ultimate justification for recommending Dr. Falcone, such as his 50 hours of specific SIBO training with Dr. Nirala Jacobi, and the fact that he offers free 15-minute consultations in Rockville.
And, because I like to continually test different AI tools against each other, I entered the same exact prompt into Perplexity.ai to see what it had to say as well. Here’s what it came up with (full conversation here):
Again, I couldn’t be happier to see Dr. Falcone show up as #1.
This isn’t a Perplexity Pro search, so it doesn’t have the same kind of train of thought that ChatGPT did, but it does list its sources. Of the 13 mentioned, fully five were pieces I originated, wrote, and published for Dignity Integrative.
Again—super satisfying to see this for my client.
A B2B example
I’ve also worked with a lot of larger physician groups, i.e., emergency and hospital medicine providers who sell their services to hospitals or health systems.
One of the central marketing goals of these groups is to get on the RFP list for when an administrator is ready to put either EM or HM services out to bid. I wrote a prompt as if I were one of these administrators starting to put together a list, again using o3 (full conversation here):
This first round of suggestions is pretty obvious—TeamHealth, Vituity, US Acute Care Solutions (a former employer), Sound Physicians, SCP Health, and ApolloMD are all simply the biggest providers in the country.2
But ChatGPT didn’t just go with the biggest, though that was part of the criteria. As it explains in its train of thought, for example, at one point, it did individual searches on each group specifically looking for performance metrics:
When I saw this, I just had to smile—the part I highlighted in blue comes from a case study that I wrote more than 10 years ago! I wrote it while I was Director of Marketing for MEP Health. It was probably published sometime in 2011. MEP later sold to US Acute Care Solutions, where I became Director of Content Marketing, and in the merger, we ported over a huge number of blog posts and case studies from MEP’s website, including the one now cited here.
It’s actually kind of beautiful: content I wrote more than a decade ago—before LLMs or ChatGPT were even close to public consciousness—is now being read by AI and relied upon to make recommendations worth potentially millions of dollars.
Side note: If I’d known then how valuable the content I was writing would turn out to be—just how long of a tail it would provide to the company—I would have charged a lot more!
But this is exactly the point: content marketing is even more valuable now than it was before LLMs started dominating the public discourse.
In any case, still acting as my fictional hospital administrator, I decided I was unsatisfied with the list of national groups ChatGPT had given me. First, I pushed it to explain further how it came up with this initial list. Its answer:
One thing to note from this image: ChatGPT is reading case studies on the company websites and taking them more or less at face value. The lesson here is so obvious it’s barely worth noting: publish more case studies! Or to be more nuanced about it: where reliable data is missing in your industry, fill that void!
Next, I pushed it to expand its search to include smaller regional groups, and asked it to compare them to the larger national groups it had already mentioned (again, full conversation available here).
Here’s what it came back with:
Again, note the reliance on case studies published on the groups’ own websites.
Tell those stories, people!
Generative Engine Optimization, or GEO—same as the old SEO, except more competitive
I just want to reiterate the point: in an AI-dominated world, good content marketing is now even more valuable.
In the past, Google would serve up 10 page one results for users to browse. Conventional wisdom was that few users ever scrolled past page 1. Thus, it was every SEO marketer’s goal to get their clients on that page. As a company trying to get the word out, you effectively had 10 “slots” to try to show up in.
In the new world, there are fewer slots. My prompt asking for integrative medicine doctors yielded only three recommendations. Unless the user asks for more, that’s all the opportunities you get.
So the point is: publish now, publish often, and publish good stuff. It’s going to deliver value for years, even decades. That is valuable stuff.
At the end of the day, I wish I could say there was same Brave New World of marketing to LLMs—secret strategies that only we professionals deep in the weeds know about. I have seen marketing agencies out there publishing exactly this message: that their expansive (read: expensive) capabilities are needed to succeed in this new world.
But the truth, as evidenced above, is that the tools to succeed in this new world are the same as in the old world, only more so.
Anyway: time to double down on content marketing.
Combined, of course, with leveraging data from keyword research tools like SEMrush or Ahrefs, among other tactics.
Envision is notably absent from the list; in the train of thought, ChatGPT explains that it eliminated it due to Envision’s Chapter 11 bankruptcy and restructuring in 2023, which—fair enough.