crawler
In a technical context, this term refers to the automated software agents that power search engines. It carries a connotation of systematic, tireless exploration, often contrasted with manual browsing. Users should distinguish this from a "scraper," which focuses on extracting specific data rather than indexing a whole site.
In mechanical and biological contexts, the word emphasizes a specific mode of locomotion—slow, steady, and grounded. When applied to machinery, it specifically denotes the use of continuous tracks (crawlers) to navigate unstable terrain where wheels would sink or slip.
Meanings
A program or automated script that systematically browses the World Wide Web to index content for a search engine.
The search engine crawler indexed millions of pages in a single day.
An organism or animal that moves by crawling, typically one that stays close to the ground or surface.
The garden is full of slow-moving crawlers like slugs and snails.