Search Engine Spider Simulator
A spider simulator shows what a search engine crawler reads in a page's HTML before any JavaScript runs. This free tool fetches your URL and lists the title, meta tags, heading outline, visible text in order, every link with its anchor text and nofollow state, and every image with its alt text. Use it to spot content a crawler cannot see.
Spider Simulator
Technical SEO
About this tool
Search engines do not see a page the way a visitor does. A crawler downloads the HTML, pulls out the text, the headings, the links and the image attributes, and decides from those what the page is about and where to go next. If an important sentence, menu or product list is added by JavaScript after the page loads, the first pass of a crawler may not contain it at all.
This simulator requests your page from our server, ignores scripts and styles, and prints what is left in the order a crawler meets it: the title and meta tags first, then the heading outline, the visible text, the links with their anchor text and whether they carry nofollow, and the images with their alt attributes. A short text block, empty outline or missing links is the signal to investigate.
Read the result as a first-pass view, not a copy of any single search engine. Google can render JavaScript in a later step, and other crawlers and AI bots often cannot. The text shown is capped at 6,000 characters, links at 100 and images at 60. If the page needs a login or blocks automated visitors, we can only report what the server returns to us.
Frequently asked questions
What does a search engine spider see on my page?
A spider downloads the raw HTML and reads the title, meta tags, headings, body text, links and image attributes. It does not see the visual layout or colours. Content that only appears after JavaScript runs may be missed on the first pass, which is why this tool prints the text found without scripts.
Why is some of my page text missing from the simulation?
The usual cause is content injected by JavaScript, such as tabs, sliders, product grids and infinite lists. Text inside images, canvas or iframes is also not part of the page text. Move important copy into the server-rendered HTML, or confirm Google has rendered it using the URL Inspection tool in Search Console.
Does a nofollow link matter to a crawler?
A nofollow, sponsored or ugc attribute tells search engines not to treat the link as a vote of trust. Google may still crawl it as a hint. Use them on paid, user-submitted and untrusted links, and keep normal internal links followed so ranking value flows through your site.
How is this different from the page source viewer?
The source viewer prints the raw HTML as sent. The spider simulator interprets it: it separates the title, headings, text, links and images and drops scripts and styles, so you can judge what a crawler can use. Check the outline and text here, then use the source viewer to find the cause.