The buildout of our first data center environment is complete, which means we now own our entire infrastructure stack.¹
We started with residential proxies to make the internet more accessible again, expanded into large-scale web crawling to support leading AI labs, and have now completed the stack with custom storage and compute, cutting millions in cost and removing the limits on what the network can store, crawl, and serve.
Why Own Infrastructure?
The only way to achieve economies of scale with an internet scale web crawl is building your own infrastructure. Simply put, cloud providers were not built for this use case.
- Our high-speed storage operates at 10% the cost and delivers twice the throughput of traditional cloud object storage. That makes it possible to store a complete, always-updated index of the public web, and serve it without constraints.
- Our tape-based cold storage comes in at 4% the cost of archival cloud storage. Tape remains stable for up to 30 years at room temperature, making it possible to preserve a permanent historical record of the web.
- Our compute servers are 90% cheaper than cloud reserved instances on hardware alone, and 95% cheaper when you factor in bandwidth. We lease transit and peering infrastructure at the source, bypassing inflated egress fees.
Building our own infrastructure allows Grass to operate at a lower cost, with stronger margins, and with more flexibility. That lays the foundation for a durable, user-owned network where value flows back to its contributors.
What This Unlocks
Owning the stack gives us full control over how the network scales.
Every layer, from storage to compute to bandwidth, is optimized for throughput and cost, making the system not only sustainable, but infinitely scalable as demand grows. That means we can support more users, more data, and more use cases, without compromising performance or control.
If the public web is to truly serve as a public good, its foundational infrastructure cannot be rented. Opening up the public web through an internet scale crawl demands owning as much of the critical stack as possible.
¹ Comprised of custom storage, compute, and bandwidth systems.
