Call · 30 min
MarketingMattia Esposito10 November 202614 min read

How to rank in Perplexity. Robots.txt opens the door, but often the firewall in front of it has the final say.

A site can have a perfectly good robots.txt and still be closed to Perplexity, because the firewall stops the bots first. Here is how to let it in, what it looks for in a source and the checklist to download.

In brief

To appear in Perplexity, a site has to let PerplexityBot in through robots.txt and allow the IP addresses Perplexity publishes through its firewall. Perplexity recommends both conditions in its own documentation.

Check the CDN first. On Cloudflare, Bot Fight Mode cannot be bypassed with a rule, and since 2025 Cloudflare no longer counts Perplexity among its verified bots.

Once you are in the index, Perplexity judges the whole site. The Government, Academic and Trusted labels look at whether a site corrects its mistakes, credits every piece and keeps news apart from advertising. They cannot be bought.

The ten-point checklist can be downloaded below, in Word and PDF, with no email required.

Written by: Mattia Esposito, AI and marketing consultant, founder of Itria AI. From Puglia, based in Turin since 2023, he works remotely in Italy and abroad. LinkedIn profile.

The “Can Perplexity read you?” checklist, to download

One sheet with Perplexity’s two bots, ten checks with where to look and space for the result, plus a log for monthly tests. In Word to fill in and PDF to print. You need access to your site’s control panel and to the CDN or firewall panel, if you have one. It takes an hour.

The ten-point checklist for ranking in Perplexity, with the table of the two bots and the monthly test log
FileContentsLink
Wordto fill in, in Italian

The two bots, the ten checks with result and date, the robots.txt line, the test log.

come-comparire-su-perplexity-checklist.docx

PDFto print, in Italian

The same checklist, to tick off with a pen.

come-comparire-su-perplexity-checklist.pdf

N.What to checkWhere to look
1

Robots.txt does not block PerplexityBot, not even through a blanket Disallow.

yoursite.com/robots.txt

2

The firewall lets through the addresses in Perplexity’s two JSON files.

CDN or hosting dashboard

3

On Cloudflare, Bot Fight Mode is off and a Skip rule sits above the blocks.

Cloudflare, Security

4

After a robots.txt change, wait 24 hours before testing again.

The checklist, date column

5

Every page says who wrote it.

Your main pages

6

The site has a visible way of correcting mistakes.

Your main pages

7

News, advertising and opinion are kept apart.

Your main pages

8

Every important page answers a question in its first lines, with figures and sources.

Your main pages

9

If you sell in the United States, product pages show availability, reviews, price and specifications.

The online shop

10

A monthly test with your customers’ questions, and the server logs for both bots.

Perplexity, server logs

On 14 October, in a LinkedIn post where I counted my own mistakes, I wrote a line that belongs on this page: “Perplexity cites me in 4 answers out of 19, and in one I am the only source.” Two months earlier, as far as the assistants knew, I did not exist: a friend had forwarded me ChatGPT’s answer, which could not find a Mattia Esposito linked to itria.io. The full post (in Italian).

In between I read the documentation of every company behind the assistants, one by one. Perplexity’s is the most practical I found: it names the bots, the addresses, the rule to put in the firewall and what happens if you block. This page sets it out in order, with the date I read each source, and goes deeper than the general check for appearing in ChatGPT and the other assistants.

How to rank in Perplexity in 2026

Perplexity searches the web in real time and writes an answer with numbered citations pointing to the original sources. To become one of those sources, a page has to clear four stages: the firewall, robots.txt, Perplexity’s index and the choice of source when someone asks a question.

How a page reaches a Perplexity answer: the firewall lets it through, robots.txt opens it to PerplexityBot, the index stores it, Perplexity-User opens it when someone asks, and the answer cites it with a number The firewallCDN, WAF, IPs Robots.txtPerplexityBot The indexPerplexity and third parties The questionPerplexity-User opens it The citationnumbered, with source checks 2-3checks 1 and 4checks 5-7 checks 8-9check 10
The path a page takes to a Perplexity citation, and which checks on the list act on each stage.

The Help Center puts it plainly: “Each answer includes numbered citations linking to the original sources”. Research mode, also called Deep Research, runs dozens of searches, reads hundreds of sources and delivers a report in 2-4 minutes.

One figure sets Perplexity apart from the other assistants. In an Ahrefs study of 15,000 prompts (August 2025, data from an SEO tools vendor), 28.6% of the pages Perplexity cited also ranked in Google’s top ten, against 8% for ChatGPT. Perplexity does not say it uses Google’s index: it says it has an index of its own and also uses third-party crawlers.

PerplexityBot and Perplexity-User: the two bots

Perplexity runs two bots with two different jobs, and only the first builds the index. PerplexityBot finds and links sites in search results; Perplexity-User opens a page when someone asks a question, to answer it and add the link. For both, Perplexity publishes a JSON file listing the IP addresses they come from.

BotWhat it doesAddresses read on 10 November 2026
PerplexityBotthe index

Finds and links sites in Perplexity’s results. Follows robots.txt. Not used to train foundation models.

13 addresses in perplexitybot.json, file created on 28 October 2026

Perplexity-Userthe question

Opens a page when someone asks. Does not crawl and is not used for training. Generally ignores robots.txt.

10 addresses in perplexity-user.json

On training, Perplexity’s documentation is clear: PerplexityBot “is not used to crawl content for AI foundation models”. If you worry about your text ending up in a model, you can let it in without that risk. The addresses change: always take them from the files, never copy them from a guide, this one included.

You can spot the user agent in your server logs: it contains “PerplexityBot/1.0” or “Perplexity-User/1.0”, with a version number that may change. A user agent can be faked, an IP address far less easily: to know whether a visit really comes from Perplexity, compare the IP with the two files. The full picture of assistant crawlers is in the glossary entry on AI crawlers.

Does Perplexity follow robots.txt?

PerplexityBot does, Perplexity-User generally does not. The Help Center says PerplexityBot will not index the full or partial text of sites that exclude it in robots.txt. For Perplexity-User, the documentation says that since a person requested the fetch, it “generally ignores robots.txt rules”. Each setting works independently, and a change can take up to 24 hours.

In robots.txt: inside Perplexity’s index

User-agent: PerplexityBot
Allow: /

A robots.txt with no rules for PerplexityBot already lets it in: the line matters when there is a blanket block higher up, such as “User-agent: *” with “Disallow: /”. How to write rules for every AI crawler, and which ones to block, is covered on the page about robots.txt for AI crawlers.

If I block PerplexityBot, do I disappear from Perplexity?

No, not entirely. The Help Center, last modified on 14 September 2026, says so clearly: “if a page is blocked, we may still index the domain, headline, and a brief factual summary”. The text of the page stays out of the index; the site name and the headline may remain.

On the same page Perplexity adds that it works with third-party crawlers to build its index and has updated those agreements so they also respect robots.txt, “particularly for news publisher sites”. The feature that summarised a blocked address at a user’s request has been switched off.

If you sell something, blocking rarely pays. A page Perplexity knows only by headline and summary is unlikely to become the numbered source of an answer, and it leaves room for a competitor with readable text. Blocking makes sense for private areas, staging sites and paid content.

Does Cloudflare block Perplexity? How to let it through the firewall

It can block it without anyone having decided to. On 4 August 2025 Cloudflare published an analysis accusing Perplexity of using undeclared crawlers when its official bots were blocked, and concluded: “we have de-listed them as a verified bot”. Perplexity disputed the account, arguing that its agents act on behalf of users.

Cloudflare sells bot protection, so in this dispute it is a party with an interest. For anyone running a site, the practical consequence is what counts: since then Cloudflare’s managed rules and Bot Fight Mode can stop or challenge Perplexity’s bots. The same article counted 20-25 million requests a day from the declared crawler.

Perplexity’s documentation explains how to let them through. In Cloudflare WAF you create a custom rule matching the user agent and the addresses of each bot, set the action to Skip for the features to bypass, and place it above any rule that blocks or challenges. The free plan allows only one custom IP list, not enough for both bots, so the addresses go straight into the rule.

There is one limit, and Perplexity spells it out: “Cloudflare’s Bot Fight Mode can’t be skipped with WAF custom rules”. With Bot Fight Mode on, Perplexity’s bots can still be challenged even with the right rule. On AWS WAF the principle is the same: one IP set per bot, a rule matching it to the user agent, and a lower priority number than your blocking rules, so it is evaluated first.

Cloudflare also has a panel for AI crawlers, with separate policies for search, agents and training, described in its AI bots documentation. If a cautious provider set up your site, it is the first place to open. On my own site a Cloudflare setting stripped the email address out of the HTML, and to the assistants the contact page had no contact details: the case is on the page about the email address AI assistants can’t see.

How Perplexity chooses sources: the Trusted, Academic and Government labels

Some Perplexity citations carry a shield with one of three labels: Government, Academic or Trusted. Perplexity explains that the label applies to the site, not the single page: “Perplexity rates the whole website a source comes from, not each individual page.” The review rests on objective questions, listed in the Help Center.

What Perplexity looks atWhat it means for your siteOn the checklist
Does the site correct its own mistakes?

A dated correction note, or a page that collects corrections.

Check 6

Does it say who wrote each piece?

The author’s full name on every page, with a profile that identifies them.

Check 5

Does it keep news, advertising and opinion apart?

Sponsored content declared as such, opinions recognisable as opinions.

Check 7

Does it appear often in results?

For “Trusted”, the site has to appear often enough and publish within its own field.

Check 8

Does it have a deal with Perplexity?

Irrelevant: partnerships and payments, Perplexity says, have no effect on the label.

None

Most sites have no label, and Perplexity stresses that the absence is not a negative judgement. Nor does it say that a label leads to more citations. Its questions, though, describe a site a system can trust. Site owners can contact support, and every source panel has a “Rate this source” option.

On page content, there is evidence measured on Perplexity itself. The study “GEO: Generative Engine Optimization” (Aggarwal and others, KDD 2024) also tested its techniques on Perplexity.ai, across 200 examples: adding quotations lifted visibility from 24.1 to 29.1, while keyword stuffing pushed it down to 21.9.

The study uploaded the texts as files, and warns that the techniques may change. A short explanation of GEO is in the glossary entry.

In the developer API, every search result carries its publication date and its last update as separate fields. Perplexity does not say dates count when it picks sources. A date shown on the page costs little and is one more fact to check.

What I measured on itria.io

Perplexity. In mid-October 2026 Perplexity cited itria.io in 4 answers out of 19, and in one it was the only source. I wrote about it in my post of 14 October.

Google and Microsoft. From 11 October to 7 November 2026 the site’s pages appeared 3,818 times in AI answers: 3,550 impressions in AI Overviews and AI Mode according to Search Console, 268 citations in Copilot according to Bing Webmaster Tools.

Another engine, another door. For Claude’s web search, on the morning of 25 September 2026 itria.io had a single page; by the evening, after I asked Brave to recrawl seven, it had five. The case is in Brave and Claude.

Other people’s sites. Of 143 websites of Italian exporters, 28 had the file that explains the company to assistants, and in 24 a plugin had written it. The full study is on food producers’ websites.

How to rank in Perplexity as a small business, in Italy and elsewhere

For an Italian small business, Perplexity’s technical rules are the same as for an American company; the commercial side is what changes. Instant Buy, shopping inside Perplexity, is available only to users in the United States according to the Help Center, and not on Enterprise plans. The merchant programme announced in November 2024 started in the United States and mentioned other markets without dates.

WhereWhat applies todaySource
Italy

Bots, addresses, robots.txt and labels as everywhere else. No shopping inside Perplexity. The Help Center is also available in Italian.

Perplexity Help Center

European Union

The same technical rules. No documented commercial programme for shops.

Help Center, Instant Buy

United Kingdom

As in the European Union: Instant Buy is not available.

Help Center, Instant Buy

United States

Instant Buy for users. Product listings with full details are more likely to be recommended.

Help Center, Instant Buy

The useful sentence for anyone selling in the United States is this one: Perplexity says its engine “is more likely to recommend merchants who provide deeper product details”, meaning availability, reviews, pricing and specifications. Products are ranked on criteria similar to those used for sources, authority and relevance, plus the user’s past searches.

For a small business selling in Italy, the lever is still the page. A complete technical sheet, a page that answers a buyer’s question, an author and an address written out in full: these are what Perplexity can read and cite. The same work pays off in Copilot, which reads Bing’s index, as the page on how to appear in Copilot explains.

Do you have to pay to appear in Perplexity?

No: there is nothing to pay to get into the index or among the sources, and Perplexity states that the spots are not for sale. The Help Center says advertisers cannot pay to place products in its “related products”, and that business deals and payments have no effect on source labels.

The Publishers’ Program, announced by Perplexity on 30 July 2024, shares revenue with the publishers it cites and counts partners such as TIME, Der Spiegel and Fortune. The programme is meant for news publishers, not as a way in for companies. A small business reading about “deals with Perplexity” should know they do not apply to it.

Nor do you need an llms.txt file for Perplexity: the documents read for this page never mention it. You can write one, and the glossary entry explains what it is, but access to Perplexity runs through PerplexityBot. How the doors of the different assistants compare, Claude included, is on the page about how to appear in Claude.

How to check whether Perplexity cites you

In two ways: in your server logs, and by asking Perplexity the right questions once a month. In the logs, look for the user agents of the two bots coming from addresses that match the JSON files: PerplexityBot tells you the site is being indexed, Perplexity-User that someone asked a question your page could answer.

For citations there is no dashboard like Bing’s for Copilot. Take five questions your customers actually ask, put them to Perplexity and note the date, whether you were cited and which page: the log at the end of the checklist is there for this. Repeated every month on the same questions, it shows whether the work on your pages is paying off.

This measurement is one part of the work on presence in search engines and AI assistants: being found by assistants is one piece of a company’s positioning. If you are looking for someone to do it, the page on GEO consultancy in Italy lists the questions to ask before you sign.

Questions and answers

How do I rank in Perplexity?

By letting in PerplexityBot, the crawler Perplexity uses to build its index, both in robots.txt and in your firewall, where the IP addresses Perplexity publishes have to be allowed through. Then the page matters: a clear answer in the first lines, figures with their source, the author named in full.

There is no form for submitting a site to Perplexity, and Perplexity says product listings cannot be bought.

Does Perplexity follow robots.txt?

PerplexityBot does: it will not index the text of sites that exclude it. Perplexity-User, the bot that opens a page when someone asks, generally ignores robots.txt according to Perplexity, because the request comes from a person. Changes can take up to 24 hours to apply.

If I block PerplexityBot, do I disappear from Perplexity?

Not entirely. Perplexity says that even when a page is blocked it may still index the domain, the headline and a brief factual summary. The text of the page stays out of its index, though, and without the text a page is unlikely to be cited as the source of an answer.

Why does Cloudflare block Perplexity?

Because Cloudflare’s managed rules and Bot Fight Mode can stop or challenge Perplexity’s bots even when nobody chose to. In August 2025 Cloudflare removed Perplexity from its verified bots, accusing it of undeclared crawling; Perplexity disputed the account.

To let it through, Perplexity recommends a WAF rule with the Skip action, matched on user agent and IP addresses, placed above your blocking rules. Bot Fight Mode, however, cannot be skipped.

Do you have to pay to appear in Perplexity?

No. Perplexity says advertisers cannot pay to place products in its “related products” and that business deals and payments have no effect on source labels. The 2024 Publishers’ Program is for news publishers, not for companies.

Notes on sources

  1. Perplexity, Perplexity Crawlers: the two bots, the 24 hours, the Skip rule in Cloudflare WAF, Bot Fight Mode, AWS WAF; the quoted sentences come from here. Address files perplexitybot.json and perplexity-user.json. Read on 10 November 2026.
  2. Perplexity Help Center, How does Perplexity follow robots.txt?, last modified on 14 September 2026, and How does Perplexity work?. Read on 10 November 2026.
  3. Perplexity Help Center, Understanding source labels and What is Instant Buy?. Read on 10 November 2026.
  4. Perplexity, blog: Introducing the Perplexity Publishers’ Program, 30 July 2024, and Shop like a pro, 18 November 2024. Developer documentation, Search API Overview.
  5. Cloudflare, Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives, 4 August 2025: the position of one party, disputed by Perplexity in August 2025. AI bots documentation, updated on 2 November 2026.
  6. Aggarwal and others, GEO: Generative Engine Optimization, KDD 2024, CC BY 4.0: section 6 and table 5, the experiment on Perplexity.ai with 200 examples.
  7. Ahrefs, AI search overlap, 11 August 2025: a study by an SEO tools vendor, 15,000 long-tail prompts, July 2025 data.
  8. The itria.io figures come from Search Console (generative AI report) and Bing Webmaster Tools, window 11 October to 7 November 2026; the Perplexity figure from Mattia Esposito’s post of 14 October 2026.
·Who does it for you

Opening the door to Perplexity takes an hour. Getting picked among its sources is the work that comes after. Positioning is built starting from the market.

Mattia Esposito, who wrote this guide, also works with Itria on positioning: in-depth market analysis to find where competitors are exposed and take that ground. He did it on his own site first. In August, asked who had founded itria.io, ChatGPT couldn’t find him. From 11 October to 7 November 2026 itria.io pages appeared 13,803 times in Google and Bing results and 3,818 times in AI answers: Google’s AI Overviews and AI Mode, and Microsoft Copilot.