The friends had already ordered GPUs, also known as images chips, for same-day supply from Amazon so they could start mining bitcoin, but their enthusiasm was high. The strong processors are prized for their capability to squeeze the math of bitcoin- mining at higher speeds. Over period, the interest evolved into a company called Atlantic Crypto, which stocked garages and afterward stores with GPUs. Some of the hefty chips were rented to shady cryptocurrency miners via the sky. As gold prices climbed, day was cash, so the company’s professionals grew skilled at hastily installing GPUs and pushing their effectiveness to the brink.
Venturo and the crew flipped when crypto prices sagged into the new year. They raised money to purchase cheap GPUs from struggling crypto workers, rebranded as CoreWeave, and offered up their hundreds of GPUs as a professional cloud services for businesses in need of the strong cards, such as AI startups and visual effects producers. GPUs are essential to machine learning projects like training algorithms to process images or text. CoreWeave was unknown in those fields, but it persuaded clients by giving up everything to deal with support issues. A manager from that time says,” We would resolve things in 15 minutes.” ” The dedication to customers was phenomenal”.
After OpenAI released ChatGPT late in 2022, everything changed. A shortage of GPUs, particularly Nvidia’s highly prized chips, resulted from a rush of interest and investment in all things generative AI. With its couple of data centers packed with GPUs and a long- standing relationship with Nvidia, CoreWeave quickly became vital to the accelerating AI hype train. According to former CoreWeave employees, the company has since acquired clients including Microsoft, OpenAI’s main sponsor, and image-generation startup Stability AI. Venuto’s biggest regret is that he did n’t order more Nvidia chips back when he could. ” If I had unlimited amounts, I could have it all sold in two weeks”, he says.
Things continued to work out. The company invested$ 2.5 million in purchasing more of the chips and building new data centers as demand for CoreWeave’s GPUs increased through the year 2023. A company that began as a beer- money side project now does n’t touch crypto, and is all in on cloud computing. The startup has received nearly$ 2.2 billion in investment funding, including$ 1.1 billion announced this week, from investors including Nvidia, and a$ 2.3 billion loan that uses GPUs as collateral over the past year. The startup’s valuation increased to$ 19 billion with its most recent cash injection.
By the end of this year, CoreWeave expects to have 28 data center campuses spread across a dozen or more US states, Europe, and potentially Asia—overall more than nine times its footprint at the beginning of 2023. Venturo, who is currently CoreWeave’s chief strategy officer, claims that long-term agreements to supply AI giants justify the construction. He claims that it has transformed this small cloud startup into one of the biggest juggernauts. ” We’re lucky enough to be in this position because of this crazy origin story where we had so much expertise building at this rapid, frenetic pace”.
Six former employees who spoke on the condition of anonymity claim that this pace has led to a culture of grueling hours and high-pressure management for some of CoreWeave’s 550 employees, claiming that this pace is unsustainable. However, CoreWeave does n’t appear to be slowing down as the AI boom is still going on.
Core Components
GPUs are required to train and operate machine learning systems like OpenAI’s GPT-4, which analyze large amounts of content, interpret user prompts, and return eloquent writing or astonishing imagery. The project occupied 3, 072 Nvidia H100 chips for three full months while Databricks, a startup that recently trained an open source AI model to compete with GPT-4, consuming a$ 10 million cloud tab. Similar bills are chewing through much of the$ 29.1 billion that generative AI companies raised last year from investors, according to market researcher PitchBook’s estimate.
The blazing 2023 of CoreWeave set a benchmark that both old and new are eager to follow. By delegating extra GPU space to software developers, other cryptocurrency miners like Applied Digital, Hut 8 Mining, and Iris Energy are attempting to become AI prospectors. With the financial reward for mining bitcoin falling, more GPUs are set to be freed up for AI.
And leading cloud players are spending a lot of money to expand their GPU and other AI-focused cloud offerings. Over the next 15 years, Amazon plans to invest almost$ 150 billion in data centers, and Google recently announced$ 3 billion in investments to build and expand two US data centers. Microsoft has spoken of a tripling its data center growth in the second half of this year.
But CoreWeave is one of only seven companies that Nvidia calls “elite” cloud partners in the US, cloud giants like Amazon are n’t on the list. Even with limited supplies, the head of Nvidia’s investment arm said last year that it does n’t help anyone” jump the queue,” so that gives CoreWeave a chance to get GPUs even when supplies are tight. The biggest benefit of CoreWeave may be its scrappiness as long as it can keep its supply of GPUs. So far it has n’t found too many hurdles it ca n’t get over—or slide past. According to Venturo,” We survived because of our hustle and creativity.”
One hundred and one of the incorrect cabinets was ordered as it scrambled to expand its facilities. Because supply chain backups have delayed new shipments by months, it was a costly error. ” You’re moving so fast, and somewhere along the line a process fails—and you do n’t realize until you have 17 tractor trailers with cabinets outside the door and you have to turn them all away”, Venturo says.
However, Venturo’s team in that crisis quickly put aside their frustration and made the decision to purchase used cabinets from what he called the gray market, an example of the shrewdness that’s been key to rapid expansion. Significant delays were prevented by the action. ” This was just one of the instances of challenges we faced and overcame to make sure we delivered for our partners”, he says.
CoreWeave has turned to the “gray market” for more than just cabinets to keep things moving. According to a former employee, it’s bought networking switches and routers from eBay to avoid waiting as long as two years for new equipment. The security and reliability of used parts can be questionable, but amid the urgency of the AI boom, some conventional practices had to be brushed aside, the person says.
In Plano, CoreWeave last year completed the installation of four one-megawatt data center halls in less than three days, a feat that typically takes weeks. We can start building really quickly with the gloves off, Venturo says.
CoreWeave was forced into another creative solution when an internet provider was slow to install a broadband connection at a new site—a problem familiar to many home internet users. One morning, three senior executives sat down to discuss ways to avoid postponing the project. All of the same solution was provided by the convenient but not inexpensive Starlink service of Elon Musk’s SpaceX until the fiber provider arrived. It eliminated weeks of potential delay. We must be incredibly flexible, Venturo asserts.
Helter Skelter
Lessons from early projects inform CoreWeave’s standard procedures today. Because of its unique design, installation only takes an hour rather than ten, CoreWeave has chosen to pay a premium for the custom manufacturing of tens of thousands of fiber-optic cables. CoreWeave began pushing orders through a number of alternate ports right away after US customs authorities stopped one significant shipment of important equipment. To avoid shortages, the company now orders far more parts than it needs, betting that the leftovers can be shipped to the next project that comes along.
Haste has occasionally had undesirable effects. According to one source, the data center at CoreWeave’s Las Vegas location still smells like burning plastic because it ran out of electrical components after the site was set up several years ago.
At the core of CoreWeave’s operations are its data center technicians. Instead of working full-time at one campus, the most skilled ones fly from site to site to start new data centers. Venturo declines to disclose how many miles his crack team of technicians has traveled, but he claims that they installed about 6, 000 miles of fiber-optic cabling last year. ” I probably have the most interaction with that team, more than any other team in the company, just because they are so incredibly important”, he says.
Some former CoreWeave employees claim that the work environment is excessively demanding. Employees are expected to be present at all times. Their personal phone numbers are visible on their company Slack profiles and ca n’t be removed. It was frustrating for a former employee because it allowed coworkers to text him about work on the weekends.
After recognizing his own back and neck pains at work, Venturo claims to have started weekly massages as a reward for employees at CoreWeave’s New Jersey headquarters a few years ago. ” If I feel like this, other people feel like this too”, he says. ( One of the former employees claims that executives were primarily the only ones who felt at ease receiving treatment at the office. )
According to Venturo, the high demands placed on workers have a justification. ” Everything that we do is about identifying and removing problems and blockers”, he says. We “push our teams to move quickly,” and” we lift them up by solving the problems together when things go wrong.”
The reliability of CoreWeave’s services as it has grown has occasionally been impacted by tight staffing and equipment supplies. The number of server outages started to increase as it shifted away from crypto because CoreWeave until recently had just one engineer focused on maintaining uptime, former employees say. According to two of the sources, the company occasionally did n’t have enough working GPUs to fulfill contract requirements. They “never had an adequate number of spares,” one claim. To compensate, the company skipped some testing of newly installed GPUs, the two sources say. In a separate internal Slack message seen by WIRED, a vice president said,” We do n’t have another choice, I believe.” According to Venturo, the company’s testing platform was created to ensure that clients adhere to their online order deadlines.
CoreWeave’s fast pace has forced some considerations—like accounting for the company’s environmental impact—down the priority list, according to former employees. Venturo claims that the majority of CoreWeave’s data centers are 100 % renewable and that the company has n’t attempted to locate data centers in floodplains that are n’t insurable. He claims that “it’s not so frenetic that we’re put in a position where we have to make irrational decisions.” Asked whether CoreWeave would release a sustainability report, as some other cloud companies do, to report its water usage and efforts to reduce supply- chain emissions, Venturo told WIRED in October that it would have more to share soon. There is no current release of any reports.
Behind Your Copilot
If you’ve ever used ChatGPT, Microsoft’s Copilot, or other generative AI offerings, or image creator Stable Diffusion, you may have reaped the fruits of CoreWeave’s frenetic labors.
According to sources, the company has constructed facilities for Microsoft with input from OpenAI, and it is also operating a supercomputer data center for Nvidia in Plano, Texas. The Plano campus of CoreWeave, which is about the size of the largest Ikea store in the US, cost about$ 1.6 billion and covers 450, 000 square feet. Another space dedicated to a single customer is in Oregon, where it has served business chatbot software startup Inflection AI. According to Venturo, these websites typically have between 16 000 and 32 000 GPUs.
Other CoreWeave locations give GPU access to a number of businesses, including Stability AI, a developer of image-generator software, and NovelAI, an online writing service. These sites tend to serve a broader range of uses by offering access to more affordable chips, such as Nvidia A100s, launched in 2020. To reduce the time it takes for customers to receive their AI-generated essays and art, they are typically located at the edge of major cities. Vincento refers to these locations as having National Football League stadiums. We’re really trying to make our cloud as dynamic and flexible as our customers demand, Venturo says.
CoreWeave’s locations have spanned new construction, empty buildings it filled in, and fully equipped data centers leased from others, including a bitcoin miner. With that swift strategy, CoreWeave is in good business shape. By avoiding the expenses of building their own facilities from the bottom up, according to Raul Martynek, CEO of data center provider DataBank, companies like CoreWeave that offer GPU access can have 70 % gross profit margins. Venturo declines to comment on gross margins.
If and when demand for GPU time is stabilize, big cloud providers like Amazon and Google could undercut costs to outstrip smaller rivals like CoreWeave. However, Tony Harvey, a senior director analyst tracking data centers for Gartner, believes a specialized vendor like CoreWeave could survive. ” It’s an interesting dance to play, and they’re going to have to work hard at it”, he says.
The fact that growth may be stymied by electricity shortages, including a lack of new power and transmission construction, is a more immediate worry for CoreWeave and the entire data center industry. According to real estate firm JLL, US campuses of cloud giants like Microsoft use about 11 gigawatts of power each day, while smaller players like CoreWeave use an additional 11 gigawatts. It expects that usage to grow about 20 percent on average annually through 2030 to an estimated 79 gigawatts combined.
Due to the popular region of Northern Virginia being such a “food fight to get power” there, CoreWeave for its part has n’t built a data center there, Venturo claims. However, the business might be just one innovative, original idea away from dominating the market.
