The SereneReader bot

What SereneReader is

SereneReader is a feed reader. When someone subscribes to your RSS or Atom feed, our servers fetch it on their behalf and show them new articles.

When a feed entry has no image, we fetch the article page once to read its og:image tag for a preview image. We also fetch your site's icon for the sidebar.

User-Agent

Every request from our servers identifies itself with this string:

SereneReader/1.0 (+https://serenereader.com/bot)

What we fetch

• Feed URLs that people have subscribed to.
• Article pages for entries without an image, usually once each, reading at most 1 MB of HTML to find the og:image tag.
• Your site's icon: the one your feed declares, /favicon.ico, /apple-touch-icon.png, or /apple-touch-icon-precomposed.png.

We don't crawl sites or follow links from article pages. Apart from subscribed feeds, we fetch a feed from our Discover catalog when a signed-in reader previews it and we have no articles stored for it, and when we check a feed before adding it to the catalog.

How often

Feeds are refreshed on a schedule. When your server sends an ETag or Last-Modified header, the next refresh sends If-None-Match or If-Modified-Since, so an unchanged feed costs you a 304 and no body.

We normally fetch each article page once, for its preview image. A permanent failure such as a 404 is remembered for 30 days. A temporary one, such as a timeout or a 500 error, is retried up to three more times, about 30 seconds, 1 minute, and 2 minutes apart, and then the page is left alone for at least an hour.

A 429 or 503 tells us to back off. For feeds, each failed refresh pushes the next one further out, starting at 15 minutes and doubling up to a day; on a 429 we also wait about as long as your Retry-After header asks, or longer. For article pages, a 429 or 503 counts as a temporary failure, so we may retry up to three times over about three and a half minutes. After that we wait as long as your Retry-After asks, up to seven days, or seven days if you don't send one.

robots.txt

Before our scheduled fetchers request an article page or your site's icon, they read your robots.txt and follow Disallow rules and crawl-delay for User-agent: SereneReader, falling back to the * group. The icon request made when someone first adds your feed doesn't check robots.txt.

Feed URLs are different. A feed is fetched on behalf of a reader who subscribed to it, so we treat it the way Google’s Feedfetcher does: the * group’s Disallow rules don't apply to it. A User-agent: SereneReader group that disallows the feed path is still honored, including when someone first adds the feed. Our scheduled fetchers honor crawl-delay for feeds too, up to 60 seconds between requests (a longer delay is treated as 60 seconds). The one-time fetch when someone first adds a feed, and our checks before adding a feed to the Discover catalog, don't wait for crawl-delay.

If your robots.txt is unreachable because of a server error, we treat it as allowing the fetch. A subscription is explicit consent from a reader, and a flaky robots file should not stop their feed.

How to block us

Block or rate-limit the User-Agent above at your server or firewall, or turn us away in robots.txt with a group that names us. A rule for every bot does not stop feed fetches; this one does:

User-agent: SereneReader
Disallow: /

Blocking your feed URL stops refreshes for everyone who subscribed to it, not just one reader.

Request signatures

Requests are signed with Web Bot Auth (HTTP message signatures, RFC 9421) so you can verify they came from us. Our public keys are published at:

https://serenereader.com/.well-known/http-message-signatures-directory

Outbound IP addresses

Scheduled feed refreshes, Discover previews, and preview-image fetches come from these static addresses. Some requests come from other addresses: the feed check and icon fetch made when someone first adds a feed come from our web servers, checks before we add a feed to the Discover catalog come from our admin tools, and our staging environment sends requests with the same User-Agent. An allowlist by IP alone will reject those requests.

162.220.234.241
152.55.180.242
152.55.180.243

Machine-readable list/bot/ips.json

Contact

Questions about our traffic, or something you want us to fix? Get in touch.