Builders v3 Opens Discussion About Challenges in Running Large Platforms
Greylock Partners recently hosted our third Enterprise Builders night, a small event at our offices in Menlo Park. We invited a group of product folks and technologists from leading public and private companies.
Builders v3 was themed around challenges in designing and running very large platforms — select windows into the plumbing of Facebook, Salesforce and Google.
Thanks again to our speakers for their lightning talks:
- Najam Ahmad, Director Technical Operations at Facebook, “Challenges of Network Infrastructure at Scale”
- Scott Hansma, Chief Architect at Salesforce.com, “Doing ACID with Cattle”
- Ben Lutch, VP Site Reliability Engineering at Google, “Control Systems at Massive Scale”
Discussions touched on two infrastructure themes we’re excited about at Greylock
1. Re-imagining Datacenter Networks
In an era where users can be located anywhere (increasingly mobile) and applications can be anywhere (including public cloud), the paths between users and applications are changing and becoming more dynamic. However, the network has not adapted, remaining one of the most static and inflexible components of the datacenter. Server virtualization, the decomposition of monolithic applications into distributed services, new distributed computation models like map-reduce, and even distributed storage, are all driving increasing east-west traffic. Datacenters today are limited not by individual compute resources, but by bandwidth and agility, as the economics and architecture of traditional networking gear remains stuck in the vertically-integrated mainframe era. We’re excited about companies that help “unstick” the network’s components, and deliver an application-driven, programmable network. This coming transition to SDN/NFV architectures will not only require novel implementations of core networking functions, but also require new and more open approaches to network management and visibility. Finally, network services must be elastic and distributed, to match the applications they serve.
2. Re-imagining Data Infrastructure
Over the last couple of decades, the industry standardized around relational databases as the model, and SQL as the interface. Over the past 5 years, businesses (especially young internet companies) began adopting schema-less NoSQL databases for their development speed and scalability. The database wars have already started, and we think this is just the beginning. A zoo of options has already emerged (with different design choices and interfaces), due to parallel sea changes in both requirements and components. On the requirements side, companies are dealing with unprecedented data scale and performance needs, a shifting mix of data types, and mounting pressure for development agility. At the same time, the growth in disk storage capacity continues to outpace the growth in disk I/O speed, and both disk and memory are getting further away from the CPU. The economics of flash, memory and other components are improving, and databases must be re-written to be aware of that shift in underlying hardware. The need for transactions has not disappeared. Additional significant waves of change are coming down the road, involving advancements in data-aware transaction execution, partitioning techniques for ACID-compliant databases, hybrid / switchable representation, query optimization, the use of Byte-Addressable Memory, and many others.
A few days after our event, the NYTimes came out with a very articulate piece from a CS graduate student on the divide between new and old school, on young engineers in Silicon Valley shunning “less sexy” infrastructure tech companies for consumer-facing ones. We believe one of the things that makes Silicon Valley great is the interchange of ideas between people and companies from different areas. Consumer internet companies learning about how to take advantage of the cost and scale of new infrastructure technologies. Enterprise SaaS companies being inspired by consumer mobile experiences.
We want to support entrepreneurs both new AND old school, and yes, we think infrastructure is sexy. If you’re excited about building in these problem areas, come talk to us.