What is Applebot? As Rob Attrell, writer for Mobile Syrup sums up its implications:
"Apple has been making plays on traditional Google turf for the last few years now, but today comes word of possibly the biggest so far"
Apple describes Applebot, its own web crawler, in brief technical detail on a support page. Website users will need to do little, if anything, to allow Applebot to index their sites. The support page states that
Applebot is the web crawler for Apple, used by products including Siri and Spotlight Suggestions. It respects customary robots.txt rules and robots meta tags.
If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions.
It's not clear how long Applebot has been operational. This support page first appeared in February 2015, and had content in April 2015.
It appears that Apple's motive is not entry into the advertising business, but rather to improve its spotlight search services on Mac and iOs, and allow Siri to provide better search results before sending the user to Safari to perform a Google search. Known for its ecosystem of products, Apple is now executing greater control over the search mechanisms that support its ecosystem.