About PicFastBot

Information for site owners and system administrators about the PicFast crawler.

What it is

PicFastBot is the image fetcher that powers the PicFast media offload service. When a WordPress site owner installs the PicFast plugin and connects their site, PicFastBot retrieves the images from that site so they can be optimized and hosted on the PicFast CDN. The plugin then rewrites image URLs in WordPress so visitors load images from the CDN instead of the origin server.

User-agent string

What it fetches, and what it doesn't

PicFastBot only fetches images for sites whose owners have explicitly connected their site to PicFast using the plugin. It follows the list of image URLs registered by that site's plugin installation. It does not crawl links, index pages, or visit any URL that wasn't specifically provided. If you are seeing PicFastBot requests on a site you did not connect to PicFast, please contact us immediately at the address below.

Behavior and limits

PicFastBot respects standard server timeouts and does not retry excessively on errors. Fetches are distributed over time so they do not spike load on origin servers. It does not respect robots.txt directives (it only fetches images it has been given explicit permission to retrieve by the site owner), but it will honor a plain HTTP 429 or 503 response and back off.

Blocking PicFastBot

If you want to block PicFastBot from your server for any reason, you can do so by user-agent string or by IP range. Contact us for our current IP ranges. Blocking the bot will prevent PicFast from optimizing images for any account connected to that server.