1. Read the sitemap
We fetch your sitemap and update the page list. New pages are added; pages that disappear from your sitemap keep their history for as long as the website is in your account, so you can still see later what happened to them.
Indexly compares the pages in your sitemap with the pages Google actually holds in its index every day, and submits the difference. Setting up is a one-off and takes ten to twenty minutes, including the Google Cloud project of your own that Google requires for submitting; after that there is nothing left to do.
From connecting to the first pages being submitted. One step happens at Google itself: a Cloud project of your own, because without it nothing may be submitted. Set it up once, then it runs by itself.
You connect Indexly to the Google Search Console that already holds your websites. You grant permission at Google itself; we never see your password.
In Search Console you give our account access to the website. Without owner or full access, Google will not tell us anything about your pages and we cannot submit anything.
Once connected, we show which websites (properties) sit under your Google account, including the permission level for each one.
You add the website you want monitored. Every website is a separate line on your invoice and can be cancelled monthly.
For every page we ask Google what the index status is, including the reason Google gives itself. So you know not only that a page is missing, but why.
From your sitemap we take the full list of pages that should exist. A sitemap index spanning multiple files is followed automatically.
Mandatory, because Google allows 200 submissions per day per Cloud project — for the whole project, not per website. With a project of your own that quota is yours alone, and it costs you nothing: Google's API is free. A wizard in the dashboard walks you through it: create the project, upload a service account key, enable the Indexing API and add the service account as an owner in Search Console. Allow about ten to twenty minutes. We stay under that with a margin of our own: 180 submissions a day for your whole account, and at most 50 per website — so one site cannot drain the project at the expense of the others.
Every day we submit the pages Google has not indexed yet, within the limits Google sets. Pages you excluded yourself with noindex or robots.txt are skipped.
One run per website, in six steps. They are deliberately separate: each step has its own limit at Google, and a step that fails must not hold up the next one. A broken sitemap should not mean we stop checking what Google already knows.
We fetch your sitemap and update the page list. New pages are added; pages that disappear from your sitemap keep their history for as long as the website is in your account, so you can still see later what happened to them.
For every page we ask Google what the index status is, including the reason Google attaches. Pages never checked before go first, then the pages that waited longest.
We fetch your pages and compare seven parts with last time. Only what genuinely changed goes back through the pipeline — so your daily quota is not spent on pages where nothing happened.
Whatever is missing or changed gets submitted to Google and announced to the other search engines through IndexNow. Pages from your sitemap go first, and we also resubmit your sitemap to Search Console. Excluded pages are skipped.
The same pages go to Bing, Yandex, Seznam, Naver and Yep through IndexNow. No daily quota, and it keeps running if you switch off submitting to Google. Google does not take part in this channel.
Impressions, clicks and positions from Search Console. This is also where we discover pages Google does know about but that are missing from your sitemap — you would never see those otherwise.
Alongside Google we pass your changes on through IndexNow, an open protocol that several search engines are connected to. One notification from your website, and every connected search engine knows something is new or has changed.
Bing, Yandex, Seznam, Naver and Yep accept IndexNow notifications. And because the Bing index also feeds DuckDuckGo and Ecosia, a single notification reaches further than that list suggests.
With no daily quota. A new category, a bulk import or a migration of thousands of pages travels in one message instead of being spread across weeks.
IndexNow was built for ordinary pages: a blog post, a product page, a category. No type of page falls outside it, and nothing needs an exception.
For IndexNow we put a single key file on your own domain together. No Cloud project, no service account, no console to work through.
Google does not take part in IndexNow. These are two separate channels running side by side: Google is served through the Indexing API and your sitemap, the other search engines through IndexNow. And to be straight about it: Bing and Yandex together account for a small share of search traffic in the Netherlands. The gain from IndexNow is coverage and speed outside Google, not volume.
Which channel reaches which search engines, how much may pass through at once, and what you have to set up yourself. Worth knowing before you start, not afterwards.
| What you want to know | Google Indexing API | IndexNow |
|---|---|---|
| Which search engines | Google only. | Bing, Yandex, Seznam, Naver and Yep. The Bing index also feeds DuckDuckGo and Ecosia. |
| How much at a time | 200 submissions per day, per Cloud project. | Up to 10,000 URLs per message, with no daily quota. |
| For which pages | Google describes the Indexing API itself as intended for job posting pages and live streams. Other use is tolerated, but it is not what the API is meant for. | Officially for every type of page. A blog post or product page is exactly what the protocol was made for. |
| What you set up yourself | A Google Cloud project of your own with a service account that is an owner in Search Console. Allow about ten to twenty minutes. | One key file on your own domain. |
| What it costs | Nothing. Google's API is free. | Nothing. The protocol is free to use. |
| Who decides on inclusion | Google, always. | Every search engine for itself. A notification is a signal, not an instruction. |
For every page we keep a fingerprint of the parts that matter for indexing. If that fingerprint stays the same, nothing happens. If it changes, the page runs through the pipeline again and a notification goes out.
Navigation, footer, a year, a counter or a block of recommended articles changes on practically every visit. If the entire HTML counted, every page would always look changed and a notification would mean nothing. By looking only at what a search engine uses, a change gives you a reliable moment: this is where the content genuinely became different. It also saves requests that lead nowhere, on our side and on the search engines' side.
With a recognisable user agent, so your server log shows it is us. We honour your robots.txt and never hit your server faster than necessary. Of the page itself we keep the fingerprint, not the page.
For every URL we record six moments. Together they show where the time goes: in discovery, in Google fetching the page, or in it being taken into the index.
The moment the content of the page became different according to the fingerprint.
The moment the URL first appeared in our list, from your sitemap or through Search Console.
The moment Google itself reports as the last crawl of the page. This is the only moment that comes straight from Google.
The first check at which we found the page present in the index.
The first day the page was shown in the search results according to Search Console.
The first day somebody clicked through to the page from the search results.
Those moments produce figures such as the median time between discovery and indexing. You compare them per website and per part of your site, your blog against your product pages for instance. That tells you whether a slow section is down to those pages or to the site as a whole.
Google does not tell you when it indexed a page. We know when we first saw it. That is why a moment reads as 'established on' rather than 'happened on': the real event sits somewhere between the previous check and that observation. So the accuracy is never better than how often we check. Recently changed pages are checked more often than the rest, which puts their timeline closer to reality than that of a page which has not moved in years.
Every day we compare the number of indexed pages against the previous measurement. If it falls, you get an email with the breakdown already in it — so you do not have to open the dashboard first to find out where to look.
And underneath it: 1,400 of those from /products/. That second figure is the one you can act on. '1,550 pages gone' moves nobody, because it could have come from anywhere; '1,400 from /products/' points at one template, one export or one integration.
Which breakdown holds the answer depends on what broke. So we do not pick one for you: all three arrive in the same alert.
The first part of the URL: /products/, /blog/, /support/. If nearly all of the loss sits in one folder, it usually sits in one template or one section of your CMS as well.
Plenty of websites split their sitemap by content type. When the loss comes out of a single file, you know which part of your website it concerns, even when the paths themselves give nothing away.
Crawled but not indexed, another page picked as canonical, excluded by a noindex. That reason decides whether to go looking at your pages or at your settings.
Every page that disappeared shows up in all three breakdowns: it sits in a folder, it came from a sitemap file, and it carries a reason. They are three ways of dividing up the same loss, not three separate losses. Add them up and you count every page three times.
Only when the loss is at least 5 pages and at least 2% of the previous measurement. Both, because either one on its own fails in the wrong direction: on a website of 40 pages, 2% is a single page, so every ordinary fluctuation sets off an alert. The other way round, 300 pages lost on a website of half a million sits well below that 2% — a real loss you would never be told about. And anyone who gets an alert every day stops reading them within a week, which does as much damage as sending nothing at all.
A drop means exactly one thing: at our last check, Google reported fewer pages as indexed than at the check before it. That may well be a genuine deindexation, but it can just as easily be a check that ran into a daily quota halfway through. So every alert states how many pages were actually checked, and the dashboard flags 'measurement may be incomplete' in plain sight whenever that number came out unusually low. Without that figure you send somebody chasing a measurement error, which costs more than it is worth.
Beyond 'is it in or not', we give every page a number from 0 to 100. Not as a grade, but as a sum: the page starts at 100 and every finding takes a named number of points off it. So you read not only that a page is in poor shape, but what is causing it and whether it is yours to fix.
Every line in this sum also appears in your dashboard, on that one page, with the same weights. That is why it is not a black box: you can check the deductions yourself, and you see straight away which ones you can fix in your CMS and which ones are about your server.
A page with a noindex, a robots.txt block or a 404 cannot enter the index. Such a page therefore gets no small deduction but a ceiling: its score never rises above 5, however tidy the rest of that page is. A page with few internal links or without structured data is merely weaker; it is simply in the queue at Google, from a worse starting position. In the dashboard you see that difference in the colour and in the wording, not only in the number, because it is a different kind of statement and it calls for different work: repairing a setting versus improving a page.
Google does not know this score and does not use it. We add up what we can see: what Google says about the index status per page, what stands in the HTML we fetch, and how your pages link to one another. That is why every page carries the full list of deductions with their weight. A score you cannot check is an opinion with a number on it; this one you can check, line by line.
A page we have not checked yet gets no score. Not zero, not fifty: none. Those pages are counted separately and stay out of the average and the median. Otherwise everyone with a fresh website stares at a screen full of low figures that only say we have not done anything yet, which says nothing about those pages.
This score costs no extra fetching. The signals come from the same HTML the content check already pulls in to work out whether a page changed. One fetch, two answers: has anything changed, and how does this page stand. So no second pass goes over your website.
We read the HTML as it reaches us and execute no scripts; that needs a headless browser and we do not have one. What we can do is flag the risk: little text in the HTML combined with a lot of script gets reported as a suspicion. A suspicion is all it is, because what stands on the page after that script has run is something we have not seen. Google does render, and may well see more than we do. That is why this finding carries little weight and why the dashboard says 'suspicion' next to it in plain words.
The internal link findings only exist once we have seen the HTML of at least 60% of your pages. The content check works through a website over days, so after the first round of a 5,000-page website we know the links of a fraction of it. Counting then would put nearly your whole website on screen as orphans — while that says something about our progress and nothing about your site. Below that threshold we leave the link findings out, and the dashboard shows how far along we are.
Every page we fetch goes through twenty-two checks. Seven of them mean the page cannot get in at all; the other fifteen make it harder. All of them are below, with what the finding means and what you do about it — the very same explanation you get in your dashboard on that page.
Submitting does not help here, which is why we do not submit such pages: it would spend your daily quota with no chance of a result. Change something about the page itself and it joins the very next run on its own.
A noindex instruction sits in the HTML or in the HTTP header. While it is there, Google is not allowed to include the page — however good the rest of it is.
What you do about it: Is it right that this page stays out? Then nothing is wrong. If not, remove the noindex in your CMS, your SEO plugin or your server settings.
Source: From Google
Your robots.txt forbids this path, so Google is not even allowed to fetch the page and has no idea what is on it.
What you do about it: Find the rule in your robots.txt that covers this path. If the page should be findable, remove that rule or make it more specific.
Source: From Google
There is nothing at this address any more: a 404, a 410, or a page that behaves as 'not found'.
What you do about it: Take the URL out of your sitemap and out of your internal links. If a page does belong here, put it back or redirect in a single step to the new address.
Source: From Google
We got a 401, 403 or 429 back when fetching. The page may be perfectly fine, but it is unreadable to us.
What you do about it: Check whether a firewall, a login wall or a rate limit is stopping us. We fetch with an identifiable user agent, so you can find us in your server log and let us through deliberately.
Source: Measured by us
The server returned a 5xx or refused access. This is not about the content of the page but about being able to reach it at all.
What you do about it: Check your server log around the time of the check. While this lasts, no page gets through properly.
Source: From Google
Google got stuck on the redirect at this address: a loop, a chain that grew too long, or a destination that did not respond.
What you do about it: Walk the redirect yourself. Make it point to the final address in one step, and make sure that address exists and answers quickly.
Source: From Google
Google treats this page as a variant of another one and shows that other page in the results. This is Google's own choice, not necessarily yours.
What you do about it: If that other page sits well in Google, nothing is wrong. If you do want this page in the index on its own, make it distinct in content and let the canonical point at itself.
Source: From Google
The page can get in, but it stands a poorer chance. Behind each finding is what it takes off the indexability score. Those weights are fixed, and they are shown for a reason: they are what lets you check the number yourself.
The HTML of this page carries a canonical pointing at a different address. With that, the page itself declares another URL to be the original. Google may ignore it, but usually does not.
What you do about it: Decide which version is meant to be the original. If that is this page, set the canonical to its own address; that is usually one setting in your CMS or SEO plugin.
Source: Measured by us
Our own fetch failed: a timeout, a certificate error or a connection that dropped. It does not follow that Google cannot reach it — we look from one server at one moment.
What you do about it: Check whether the page is reachable from outside your own network. If it persists, look at your certificate, your firewall and your server response time.
Source: Measured by us
Not one page on your own website points at this address. Search engines can then only find it through your sitemap, and a page nobody links to reads to Google as a page that does not matter.
What you do about it: Include the page in a menu, on an overview page, or in a text link from a page that already performs well.
Source: From your internal links
Google read the page and decided not to include it. That is almost always a judgement about the content: too thin, too similar to other pages, or too little reason to show it.
What you do about it: Make the page stronger or more distinctive. Resubmitting rarely helps here — the content is the problem, not the awareness.
Source: From Google
The main text of this page is identical to that of another page on the same website. Google then picks one to show, and decides for itself which.
What you do about it: Merge the pages, make them genuinely different, or use a canonical to point out which of the two is the original.
Source: Measured by us
There is a redirect at this address. The destination may well be indexed; this address itself is not supposed to be.
What you do about it: Take this URL out of your sitemap and let your internal links point straight at the destination.
Source: Measured by us
Internal links do point at this page, but every one of them is marked nofollow. That explicitly asks search engines not to follow them.
What you do about it: Remove the nofollow attribute on the links you do want followed. It usually sits in a template or a plugin that applies it everywhere at once.
Source: From your internal links
We found fewer than 250 characters of main text in the HTML. That leaves Google little to show the page for, and little reason to rank it above another.
What you do about it: Write the page out properly, or drop it if it has no reason to exist. If you believe the text is there, also look at the JavaScript finding.
Source: Measured by us
This is a suspicion, not an established fact. The HTML as it reached us held little text and a lot of script. We do not render JavaScript, so what stands on the page after that script has run is unknown to us. Google does render, and may see more than we do.
What you do about it: Check the raw HTML yourself, with 'view source' or with URL inspection in Search Console. If your text is not in there, have your server send it along instead of building it in the browser.
Source: Measured by us
This address only reaches its destination after two or more redirects. Every hop costs time, and the longer the chain the more chance something breaks along the way.
What you do about it: Make the first redirect point straight at the final address, so a single hop is left.
Source: Measured by us
One or two internal links point at this page. It can be found, but it carries little weight within your own website.
What you do about it: Link to it from pages that already perform well: an overview page, a related article, or the menu if it matters enough.
Source: From your internal links
We know this address, but it is not (or no longer) in your sitemap. That removes the clearest signal that this page belongs to your website.
What you do about it: If the page belongs there, add it to your sitemap. If it does not, this is exactly what you want and you can ignore it.
Source: From your sitemap
Google knows the address but has not fetched the page yet. This is usually about crawl budget: Google does not spend enough time on your website, or your server was slow to respond.
What you do about it: Submitting genuinely helps here, and we do it automatically. On top of that, add internal links from pages that already sit well.
Source: From Google
Google has never seen this address. There is no judgement about the page yet; it simply has not been discovered.
What you do about it: Submitting helps most here, and we do it automatically. Also make sure the page is in your sitemap and linked to internally.
Source: From Google
We found no JSON-LD on this page. That does not hold indexing back — it is the lightest finding we have — but it costs you the chance of rich results: reviews, prices or questions shown in the result itself.
What you do about it: Add structured data that suits this type of page. Most content management systems and SEO plugins generate it for you.
Source: Measured by us
A page we have not checked yet gets no score. Not a zero and not a fifty — none. And a finding we cannot establish reliably is left out rather than assumed: we only say anything about orphan pages once we have fetched at least 60% of your pages ourselves. One finding is explicitly a suspicion rather than an observation, and it is labelled as one.
Alongside our own checks we ask Google for the status of each page. That yields twelve outcomes — ten from Google itself, plus “not checked yet” and “other”, which are ours. Each one comes with what it means and whether resubmitting is worth it — because in half the cases it is not, and you should know that before you spend your daily quota on it.
These pages are in the index.
Our advice: Nothing to do.
Google fetched and read the page but decided not to include it. That is almost always a judgement about the content: too thin, too similar to other pages, or too little reason to show it.
Our advice: Make the page stronger or more distinct. Resubmitting rarely helps here — the content is the problem, not awareness of the page.
Google knows the address but has not fetched the page. This is usually a crawl budget signal: Google is not spending enough time on your site, or the server responded slowly.
Our advice: Submitting does help here, and we do it automatically. Also add internal links from pages that already rank.
Google treats this page as a variant of another one and shows that other page instead. This happens with duplicate content, with filters and variants in the URL, or with a canonical pointing elsewhere.
Our advice: Decide which version is the right one and point the canonical there. If that other page is already in Google, nothing is wrong and you can ignore this.
The page carries a noindex instruction. Someone explicitly stated this page must not appear in Google.
Our advice: Is that correct? Then all is well. If not, remove the noindex — this is the most common unintended block.
Your robots.txt forbids Google from fetching the page, so Google cannot even see what is on it.
Our advice: Check your robots.txt. If the page should be findable, remove the block.
This address redirects. The destination may well be indexed; this address itself is not meant to be.
Our advice: Remove this URL from your sitemap and link internally to the final address.
Google gets a 404. There is nothing at this address anymore.
Our advice: Remove the URL from your sitemap, or put a page back if this was a mistake.
Your server returned an error when Google came by. This is about availability, not content.
Our advice: Check your server logs around the time of the check. While this persists, no page gets through properly.
Google has never seen this address.
Our advice: Submitting helps most here, and we do it automatically.
We have not asked Google about these pages yet. That happens automatically on the next run.
Our advice: Nothing to do. Wait for the next check, or start one yourself.
Google returned a reason we do not name separately yet. The literal text is shown per page in the list below.
Our advice: Look at the text from Google on the page itself.
We fetch these outcomes, translate them and add advice, but the verdict itself comes from Google. If Google changes its categories, this list changes with it. Our own findings above sit beside it and never contradict it: everything we know from Google runs through a single place in the code.
Google caps how much we may request and submit per day. Those limits exist for a reason, and working around them is a reliable way to lose access.
So Indexly spreads the work: a daily maximum per website for pages we check and submit, with priority for the pages in your sitemap. When a limit is reached we stop for that day and continue the next — instead of hammering on until Google blocks us. On top of that we always submit your sitemap to Search Console: that is the route Google supports for every website.
Without owner or full access, Google gives us no information about your pages and accepts no requests on your behalf. With view-only rights we can see the website listed, but do nothing with it.
Because Google caps the Indexing API at 200 submissions per day per Cloud project, for the whole project and not per website. If every customer shared our project, that quota would be gone after a handful of websites. With a project of your own those 200 are yours. It costs you nothing, as Google's API is free, and the wizard in the dashboard walks you through the steps: create a project, upload a service account key, enable the Indexing API and add the service account as an owner in Search Console. Without this project we cannot submit anything; checking what is missing carries on as normal. We stay under that with a margin of our own: 180 submissions a day for your whole account, and at most 50 per website — so one site cannot drain the project at the expense of the others.
Then we can only work with the pages Google already knows, which means we cannot see what is missing. A sitemap is the only reliable source of 'what should be there'. Almost every CMS generates one automatically.
Google decides that, and it varies a lot per website and per page. We record when we submitted a page and when we first see it in the index, so you can follow it yourself rather than having to take our word for it.
No. After five attempts we stop with that URL. If a page still is not indexed after five tries, the cause is the page itself and not the submitting — continuing only burns capacity another page could use.
One text file holding a key, on your own domain, so the search engines can establish that a notification really comes from the owner. Nothing else: no Cloud project, no service account, no plugin. After that, changes travel to Bing, Yandex, Seznam, Naver and Yep automatically.
We fetch the page and compare a fingerprint of the parts that matter for indexing: the title, the meta description, the canonical, the robots directive, the first H1, the main content and the structured data. Navigation, footer and dates are deliberately left out, because they change on every visit and every page would always look changed.
Because that is not a new signal. Announcing the same page again while nothing changed spends daily quota that a genuinely new or updated page can put to better use. If something does change later, the comparison picks it up by itself.
They are our observations, not snapshots from Google. Apart from the crawl moment, which Google reports itself, the rule is the same everywhere: we know when we first saw something, not when it happened. The real event sits between the previous check and that observation, so accuracy depends on how often we check. Recently changed pages come up more often than the rest.
When the number of indexed pages has fallen by at least 5 pages and at least 2% compared with the previous measurement. Both conditions have to hold: a percentage on its own makes a small website cry wolf at every ordinary fluctuation, and a count on its own lets a real loss on a large website disappear into the noise. That is the threshold for what you see on your dashboard, per website and against the previous measurement. The email has its own, higher threshold: at least 3 pages and at least 5%, counted across your whole account and compared with a week earlier, at most one message of each kind a day. That keeps the dashboard fine-grained while your inbox stays quiet.
That we managed to check noticeably fewer pages than usual that day, for instance because a daily quota at Google ran out. The fall you are looking at may then come from our measurement rather than from the index. We do not measure Google's index itself; we measure what Google reported to us at our last check. That is why every alert states how many pages were actually checked: with that figure in hand you can judge for yourself whether the fall is worth investigating.
Because they are three breakdowns of the same loss, not three separate losses. Every page that disappeared sits in a folder, came from a sitemap file and carries a reason from Google, so it counts in all three. Adding them up gives you roughly three times the real number. Compare within one breakdown, never across them.
It is a number from 0 to 100 that we build ourselves, not a mark from Google. Every page starts at 100 and each finding takes a fixed number of points off: a canonical pointing at another URL costs 30 points, a page with no internal links pointing at it 25, missing structured data 5. Every page carries the full list of deductions with their weight, so you can check the number yourself. Pages with a noindex, a robots.txt block or a 404 work differently: those cannot get in at all and receive a ceiling of 5 rather than a deduction, so they can never end up above a healthy page.
No. That needs a headless browser and we do not have one; we read the HTML as it reaches us. What we do is flag the risk: little text in the HTML plus a lot of script produces the finding 'possibly built by JavaScript'. That is explicitly a suspicion — what stands on the page after that script has run is something we have not seen, and Google does render, so it may see more than we do. This is why the finding carries only 10 points and why the dashboard labels it a suspicion.
Because we have not checked them yet. A page we have never asked Google about and have never fetched ourselves gets no number: not zero, not fifty. Those pages are counted separately and stay out of the average, the median and the list of worst pages. Inventing a score would only say that we have not done anything yet, which is no statement about your page.
Most likely because we have not seen enough of your website yet. We only count incoming internal links once we have fetched the HTML of at least 60% of your pages. The content check works through a website over days, so after the first round we know the links of only a small part — and counting then would declare nearly your whole website orphaned. The dashboard shows what percentage we are at right now.
No. The signals come from the same fetch we already make to work out whether a page changed in substance; no second pass goes over it. We fetch with an identifiable user agent, honour your robots.txt and never approach your server faster than needed.
Connect your Search Console and see within minutes which pages Google does not have yet.