Parsebot/1.0

Hi, I'm Parsebot. 🤖

The data-collection agent behind parseAPI. If you found this page from your server logs, here's exactly what I was doing there.

Sleeping, last active 4 hours ago

Parsebot identifies itself with this User-Agent on every request:

Parsebot/1.0 (+https://parseapi.com/bot; hello@parseapi.com)

How I operate

  • On a schedule
    About once a day per source, plus whatever else a job needs to stay current.
  • I say who I am
    Every request carries my User-Agent, a link to this page, and hello@parseapi.com. No anonymous hits.
  • I go get what the APIs need
    Registry data, geofeeds, routing snapshots, postal and population feeds, email domain lists, cloud and CDN ranges, and whatever else keeps an endpoint honest. If it's public and we need it, I pull it.

What I fetch

Sources Parsebot pulls from today, across the APIs. The list grows when an endpoint needs something new.

  • Internet registry allocations
    Delegation files published by the regional registries that assign IP address blocks
  • Operator geofeeds
    Self-published IP location feeds that network operators ask the world to read
  • Routing snapshots
    Daily snapshots of which networks announce which IP address blocks
  • Cloud provider ranges
    IP range files published by cloud and CDN platforms
  • Postal and address files
    Official postal code files published by national postal, mapping, and statistics agencies
  • Population figures
    Population data published by national census bureaus and international statistics agencies
  • Email domain lists
    Public blocklists that flag disposable, throwaway email domains

The full job

Downloading files is one part of the job, not the whole thing. I also run live latency checks against our own API from 16 cities worldwide. I sync product data like postal codes and population counts so lookups stay ready.

Questions

Seeing traffic you don't want, or publishing a geofeed I should pick up? Email hello@parseapi.com and a human will sort it out.