{
  "$schema": "https://pointcast.xyz/BLOCKS.md",
  "id": "0552",
  "url": "https://pointcast.xyz/b/0552",
  "channel": {
    "code": "FCT",
    "slug": "faucet",
    "name": "Faucet",
    "purpose": "Free daily claims, giveaways.",
    "color600": "#BA7517",
    "color800": "#834F0A"
  },
  "type": {
    "code": "READ",
    "label": "READ",
    "description": "Long-form text — essay, dispatch, article."
  },
  "title": "Who Gets to Read the Web?",
  "dek": "A field guide to Firecrawl, spiders, indexes, and the argument hiding inside every fetch.",
  "body": "The web was born linkable. The spider made it legible. The index made it governable. AI made the old bargain visible.\n\nWho Gets to Read the Web? is a PointCast field guide to Firecrawl and the larger system it enters: crawling, scraping, rendering, extracting, indexing, ranking, archiving, and deciding who may turn a public page into a portable record.\n\nFirecrawl is an open-source web data API. Its main repository is AGPL-3.0, with separately identified MIT-licensed SDK and UI portions. It can search, map, scrape, crawl, render, interact with, and extract modern pages into Markdown, screenshots, links, or structured JSON. PointCast has no sponsorship, partnership, or financial relationship with Firecrawl.\n\nA seven-stage browser-local instrument traces one URL through discovery, rules, fetch, render, extraction, indexing, and retrieval. It performs no network request, arbitrary execution, storage, analytics, account creation, or upload. An eight-position lens board turns the crawler through the perspectives of a publisher, librarian, researcher, builder, host, artist, agent, and public reader.\n\nThe history begins with the web's 1989-90 open linking architecture; the 1992 resource discovery problem; the 1993 World Wide Web Wanderer; the 1994 robots convention; the 1998 Google paper and PageRank; Beautiful Soup in 2004; Common Crawl in 2007-08; RFC 9309 in 2022; Firecrawl in 2024; and new publisher control and payment experiments in 2025-26. Primary and official sources are linked inline and carried in the JSON twin.\n\nThe feature's Crawl Compact asks a machine to identify itself, read the host's signals, minimize collection, keep a human pace, avoid access-control bypass, retain provenance, separate technical reachability from rights, and make stopping real.\n\nThis is an editorial and technical field guide, not legal advice. Scraping questions vary by facts and jurisdiction and can implicate copyright, privacy, contract, computer-access, database, publicity, consumer-protection, and sector-specific law. robots.txt communicates requested crawler behavior but is not access authorization or a security control.",
  "timestamp": "2026-08-02T17:20:00.000Z",
  "size": "3x2",
  "noun": 552,
  "readingTime": "16 min field guide",
  "media": {
    "kind": "image",
    "src": "https://pointcast.xyz/images/firecrawl/field-guide-og.png"
  },
  "external": {
    "label": "Open the Firecrawl field guide",
    "url": "https://pointcast.xyz/firecrawl"
  },
  "meta": {
    "series": "PointCast Field Guide",
    "fieldGuide": "001",
    "crawlStages": 7,
    "historicalStops": 10,
    "lenses": 8,
    "compactRules": 8,
    "sources": 18,
    "networkRequests": false,
    "arbitraryExecution": false,
    "storage": false,
    "analytics": false,
    "registration": false,
    "legalAdvice": false,
    "relationship": "independent editorial coverage; no Firecrawl sponsorship, partnership, or financial relationship",
    "campaign": "PC-FIRECRAWL-FIELD-GUIDE-2026"
  },
  "author": "mh+cc",
  "source": "Michael Hoydich chat directive, 2026-08-02: landing page about Firecrawl, what it is, what's interesting, the history of web scrapers, web spiders, indexing, who gets to do it, and philosophies through different lenses.",
  "mood": "open-web-electric",
  "moodUrl": "https://pointcast.xyz/mood/open-web-electric",
  "companions": [
    {
      "id": "https://pointcast.xyz/firecrawl.json",
      "label": "Read the structured history, source ledger, lenses, and crawl compact",
      "surface": "external"
    },
    {
      "id": "https://pointcast.xyz/connectors#firecrawl",
      "label": "Get Firecrawl going with ChatGPT, Codex, Claude, or another agent",
      "surface": "external"
    },
    {
      "id": "https://pointcast.xyz/agent-kit.md",
      "label": "Read the PointCast agent kit",
      "surface": "external"
    }
  ],
  "clock": null
}