Beyond Pixels – NVIDIA’s Quest to Uplift Storytelling

Whether you like it or not, Nvidia is an absolute must-know for PC gamers. Among the heavyweights in the gaming tech world, Nvidia undoubtedly takes a top spot in terms of influence. Without a doubt, PC gaming would be in a much tougher spot without Nvidia’s contributions.

Nvidia has been around for over 30 years. It started out as a gaming company, but it has pivoted into then AI area. Since then, the giant has started to gain the upper hand to dominate the game of AI. This is not a strange thing about Nvidia to rock the stage, in fact, the business, under the lead of Jensen Wong, already the behemoth in the stage of GPUs.

That’s why in order to comprehend the abundance of Nvidia, we need to get sharp on its captain, Jensen Wong.

Immigrant Boy Turned Blind Alley into Open Gate

Jensen Wong was born in southern Taiwan in February of 1963. When he was nine years old, his family moved to America for a better future, but they faced the challenge that confronts almost every prospective migrant: they didn’t have the money.

Due to tight finances, Jensen ended up at Oneida Baptist College in Eastern Kentucky, a school known for its work in rehabilitating troubled youth. At just nine years old, his roommate was a seventeen-year-old who had recently been released from prison and was still recovering from stab wounds.

Though this situation could have taken a dark turn, Jensen displayed an early glimpse of the character trait that would define him, when others saw adversity, he saw opportunity. Jensen helped his roommate with math, and in turn, his roommate introduced Jensen to weightlifting.

He loved math, so it was only natural that he chose to study electrical engineering. After graduating, he moved to Silicon Valley together with Laurie, his wife and former Oregon State lab partner.

Joining AMD, he worked on chip design and learned the importance of innovation. He realized that chip design was challenging and not as glamorous as he had thought. To make it cooler and more cutting-edge, he saw the potential in specialization.

This belief was reinforced when a colleague introduced him to LSI Logic, a company focused on Application-Specific Integrated Circuits (ASICs). Jensen recognized the power of ASICs and their alignment with his ideas about chip design. When offered the chance to join LSI Logic, he seized it.

In his new role, Jensen played a crucial part in collaborating with Sun Microsystems to create chips for their SPARCstation 1 workstation. This marked the beginning of his journey in the world of chip design.

But it wasn’t until late 1992 that marked the beginning of his entry to the chip design industry

3 Bold Guy Take on the 3D Graphic Industry

Towards the end of 1992, Jensen got a call from two Sun Engineers, Chris Malikowski and Curtis Prime. Having a strong working rapport with Jensen after years of collaboration, they reached out, sharing their discontent with life at Sun and asking if he could find some time for a chat about an interesting concept.

“Demand for 3D Graphics is increasing,” Chris and Curtis began. “We are great chip engineers; you’re a great chip product manager. How about we make a chip that can enable a PC to handle 3D Graphics by taking some of the workload off the CPU?”

The case for the idea they were presenting was undeniably compelling. Early indicators strongly favored the appeal of Advanced Graphics among consumers. For instance, Wolfenstein 3D, released in May 1992, had already sold a quarter of a million copies by 1995. Doom, which hit the market in 1993, became a worldwide sensation. What’s more, Jurassic Park marked a significant milestone by extensively utilizing 3D Graphics, raking in a billion dollars at the box office.

However, venturing into this territory involved considerable risks. The technology was cutting-edge and came with a hefty price tag. It wasn’t merely a wager on consumers’ interest in 3D Graphics in games; it was also a bet on the gaming industry becoming a profitable one. This was a challenging prospect, especially if you were involved in chip manufacturing, where control over the industry’s direction was limited.

Taking the plunge was a risky move, but Jensen had already demonstrated his knack for turning risks into opportunities. So, they were all in. That memorable dinner at Denny’s would go on to be one of the most critical meals and the best investment of their lives.

Jensen Huang decided to resign from LSI Logic and seek venture capital for a new venture. His first encounter with legendary venture capitalist Don Valentine didn’t go well, as he was nervous and unprepared.

However, it was an eight-year process that paid off when Sequoia decided to invest in Jensen’s new company, valuing it at six million dollars.

Jensen saw that gaming had the potential to become the best way to tell stories because, unlike movies or TV, games weren’t pre-recorded; the outcomes weren’t predetermined, they could be different every time you played them.

But he also understood that technology was simply not mature enough to realize that potential. That led him to define Nvidia’s purpose: to enable graphics to be a new medium to tell stories.

Nvidia faced a challenge in convincing 2D game developers to transition to 3D game development and adapt to new APIs and frameworks for their graphics chips. This was a tough sell due to the limited number of developers ready for the shift and Nvidia’s lack of a track record. However, their backing by Sequoia carried weight and helped secure a deal with Sega, a leading gaming company of the mid-1990s.

Their collaboration with Sega started on a rocky note as Nvidia and Sega chose quadrilateral polygons as their geometric primitives for 3D graphics, which would later prove problematic.

As the gaming industry evolved, with more companies recognizing the demand for PC gaming, including Microsoft, which sought to promote 3D graphics development directly on Windows, potentially sidelining Nvidia.

Microsoft’s Direct3D became the industry standard with triangles as the geometric primitive. This prompted others to join Microsoft, posing challenges for Nvidia. Sega unexpectedly ended their collaboration, and falling memory costs allowed competitors to create more efficient, cheaper chips.

Nvidia was now hemorrhaging cash fast, with about nine months of runway left. Most normal founders would have admitted defeat and pulled the plug. But Jensen wasn’t ready for defeat.

Facing dire circumstances, Jensen made a radical decision: he laid off most of the staff, leaving just around 35 members with an impossible mission to design and deliver a new chip from scratch.

This seemed unachievable, given the conventional principles of chip design in the mid-’90s. Nvidia’s rivals had been outpacing them by producing chips with more memory at lower costs, leveraging Moore’s Law. However, Jensen was about to change the game.

Flip the Script – Unlock a New Way to Tell Story

If Nvidia couldn’t benefit from Moore’s Law, they would have to get around it instead. The basic idea was this: comprehensive verification of chip design is crucial to reducing development costs and timelines. The functional verification of a chip design is usually performed with logic simulation or physical prototyping on what’s called field-programmable gate arrays, or FPGAs.

Jensen knew that physical prototyping had a few drawbacks. Firstly, the process required to implement a large design into several of these gate arrays could be long and arduous. Secondly, it made fixing design flaws time-consuming. Therefore, Jensen recognized an inefficiency. Simulation was underutilized early in the verification process when bugs were more frequent. The next step was finding a way to exploit it.

He had caught wind of a new software that had the capability to emulate chips, though the startup behind it lacked a track record. Nevertheless, Jensen believed it had the potential to fulfill his needs. With no other viable options, he found himself at a critical juncture. It was a minute to midnight, and he made the decision to stake every last dollar Nvidia had in the bank on this gamble – an all-or-nothing move.

Fortunately, luck was on their side. Emulation provided Nvidia with a way to bypass the usual lengthy production and shipping processes. What had once taken 18 to 24 months could now be achieved in just six. The outcome was the Riva 128, one of the initial consumer graphics cards to seamlessly incorporate 3D acceleration alongside traditional 2D and video acceleration.

Almost overnight, it became much easier for Nvidia to persuade developers to use its chips. Why? Because Jensen had identified and optimized for a fundamental principle of customer behavior: the perception that performance was king.

Although his approach may have had its share of flaws, by drastically shortening their design cycle, Nvidia managed to take significant strides every six months.

Around 1997, Nvidia was achieving a performance doubling every six months, running three times faster than Moore’s Law. By 1999, they had launched the Riva TNT and Riva TNT2, moving beyond the Riva 128. The successor to the TNT2 was named through a contest, and “GeForce” was born.

The first GeForce, the GeForce 256, shipped in 1999 and marked a significant performance improvement. Jensen referred to it as the first GPU, which turned out to be accurate as it tackled the slowing CPU performance increase due to physical constraints faced by chip makers. Nvidia had once again surpassed Moore’s Law.

Nvidia’s success hinged on its adoption of parallel computing, which allowed for faster and independent task execution, starting with the G4 series.

Their 1999 IPO, valued at $12 per share and $600 million, provided capital and credibility, marking a milestone for its founders and investors.

A pivotal collaboration with Microsoft for the Xbox project, featuring Nvidia’s GeForce 3, revolutionized GPU architecture and led to substantial revenue growth, reaching about $1.4 billion by 2001, thanks to rapid chipset releases.

Yet, Jensen knew that if you don’t continue aggressively innovating, you’re moving backward. By the middle of the 2000s, some clouds were gathering on the horizon.

As gross margins started to shrink, the impending replacement of the original Xbox by the Xbox 360, which did not feature an Nvidia GPU, signaled the potential end of a significant revenue source. Meanwhile, Nvidia’s main competitors were becoming more formidable, with Intel’s 2006 announcement of Larrabee, its dedicated GPU.

While the situation wasn’t as dire as it had been a decade earlier, thanks to Nvidia’s chip emulation breakthrough, they were still reaping substantial revenues from the PC gaming market. However, Jensen wasn’t content with dominating a single market. He aimed to prepare for the moment when GPUs could find applications in diverse domains.

Ahead of Time – Shaping the Future Landscape

During the early 2000s, a quantum chemistry researcher at Stanford reached out to Jensen Huang to express his profound gratitude for helping to resolve a major issue. Previously, it would consume up to two weeks for the researcher’s machine to execute critical computations.

Then, a breakthrough occurred when the researcher’s son recommended purchasing several GeForce cards from a nearby electronics store and installing them on the lab computers. Following this advice, what used to be calculations taking days or weeks on his lab machine were suddenly accomplished in just a matter of hours.

He decided to verify the results by running the same computations on his old supercomputer setup, and the findings remained consistent. However, when he ran the calculations on the Nvidia GPU rig, they were completed ten times faster.

The precise details of Jensen’s transition into scientific computing are not the focal point. What truly matters is his drive to reshape Nvidia into a company that facilitates a fresh approach to storytelling through technological advancement.

Although gaming had been a thriving market for Nvidia, a grander vision existed: to enable innovation and storytelling across diverse fields. Jensen aspired to broaden the scope of scientific computing, igniting his mission to uncover the reachable market.

Even though the opportunity might not have been immediately apparent, he believed innovation would create new use cases. He fell in love with the idea of advancing human knowledge and expanding storytelling through hardware.

To pursue this vision, Nvidia had to develop a groundbreaking platform comparable to the transistor’s impact on computing. The introduction of the G4 Series marked the birth of the GPU concept, enabling parallel graphics processing for faster and more dynamic rendering. Nvidia extended the C programming language with CG (C for graphics) tools, but this was only the beginning.

In this game, Jensen knew a comprehensive API was needed, leading him to collaborate with researcher Ian Buck. Together with Microsoft, they created CUDA (Compute Unified Device Architecture), a pioneering parallel computing platform.

CUDA 1.0, launched in November 2006, empowered developers to harness Nvidia GPUs for scientific and AI workloads in various domains, from computational finance to medical imaging and more.

If we consider the invention of the transistor and microprocessor as the top innovations in computing, then CUDA isn’t far behind. What’s even more incredible is that the technical aspect wasn’t the most extraordinary part of CUDA. The truly remarkable aspect was its business model.

To begin with, Nvidia has never charged a single dollar for CUDA, but it can only be used on Nvidia’s hardware.

GPUs unlock the power of parallel computing, and both Apple and Nvidia devised business models that raked in billions of dollars through parallelism. However, the progress of this new development platform and the equally ingenious business model alone weren’t enough to pave the way for its current trajectory.

In fact, in 2011, Nvidia’s stock experienced a significant drop of around 50% when the company failed to meet its earnings targets. In a sense, CUDA was ahead of its time, invented before the rest of the world was truly prepared. It would require a stroke of luck for it to eventually gain the recognition it deserved.

Take the Leap – Pivoting into the AI Supremacy

In 2000, Computer Science Professor Fei-Fei Lee started a project called ImageNet, influenced by Princeton’s WordNet, aimed at creating a vast database of labeled images for AI image recognition.

This resulted in a competition in 2012 that was won by the Alex Krashevsky, Ilya Sutskever, and Jeffrey Hinton team from the University of Toronto using their algorithm, AlexNet. Neural network AlexNet fared better than rivals, identifying roughly 85% of the images that were shown.

Here’s where it gets interesting. The team from the University of Toronto used CUDA on Nvidia GPUs to train their neural network. This was a pivotal moment for artificial intelligence, as it brought together exceptional parallel computing capabilities with deep learning models that teach machines how to learn.

It was a perfect fit for CUDA, marking a significant turning point. According to Jensen, it wasn’t merely a stroke of luck. Nvidia’s extensive efforts in pioneering GPUs had positioned them to reap the most benefits from the fusion of deep neural networks with GPU technology.

Nvidia made itself the hardware supplier of choice for the machine learning revolution by being ready for the opportunity when it came. Machine learning transformed Nvidia’s business by enabling entirely new revenue streams.

One of them was expanding the customer base for its high-end GPUs like the RTX 3090. That involves gamers, researchers, and those who just want top-of-the-line processing power. But the real wealth was always going to be found among Enterprise customers. They have the deepest pockets and the greatest need.

Today, Nvidia’s top-notch GPUs power colossal data centers for tech giants like Google, Microsoft, and Amazon, along with those shaping the internet’s infrastructure. Tesla, despite announcing plans to develop its chips, continues to use Nvidia hardware for its Dojo AI Training Center, illustrating Nvidia’s broad market reach.

In April 2020, Nvidia acquired Mellanox, tripled data center revenue, and saw substantial growth. Its financials have evolved remarkably with $26.9 billion in revenue and a 66% gross margin. Nvidia has established an imposing market presence through its open development ecosystem, CUDA, and GPUs, serving as the foundation for various applications, notably deep learning.

Nvidia’s power lies in its ability to operate on the price curve and lead in technological innovation, making competitors react to its moves. It harnessed the emulation of graphics chips, creating a proprietary ecosystem and delivering unmatched performance.

Under Jensen’s lead, Nvidia continually innovates, with ventures into electric and autonomous vehicles, an addressable market worth $300 billion. Nvidia’s Omniverse, Earth 2, and simulation capabilities present groundbreaking potential, offering limitless applications in fields like climate modeling, space exploration, and pharmaceuticals, while maintaining continuity with its scientific discovery roots through CUDA.