A search engine spider, also known as a web crawler, is an Internet bot that crawls websites and stores information for the search engine to index.