Huang’s Law Is The Catalyst That Drove NVDIA’s 1000x Chip Speedup In Less Than A Decade
NVIDIA "Huang's Law" is the primary catalyst for driving chip performance and efficiency to over 1000x in less than a decade. For NVIDIA, Huang's Law is the fundamental approach that moves beyond traditional chip speedup fundamentals such as Moore's Law which had dominated the tech industry in the past.
Huang's Law To Dominate The Future For NVIDIA, Chip Shrinking In No Way Defines The Increment in Performance
NVIDIA's CEO Jensen Huang has expressed multiple times that Moore's Law is "slowing down," and the concept it is backed with is starting to get outdated. The argument became heated, especially after Jensen's GTC 2023 keynote. If we look at what Moore's Law is, it is related to the number of transistors on a microchip and how it "should" double every year.
Though NVIDIA migrated GPUs from 28nm to 5nm semiconductor nodes over the decade, that technology only accounted for 2.5x of the total gains, Dally noted.
In a blog post, NVIDIA's Chief Scientist Bill Dally clarified that Team Green's approach towards next-gen technology revolves around "Huang's Law." We will get into what this means, but NVIDIA themselves claim that the terminology originated through a report by IEEE Spectrum and was later familiarized amongst several media outlets. The concept NVIDIA has been implementing with its products recently is indeed interesting, and it probably holds the keys to opening the doors to the industry's future.
In a talk at Hot Chips 2023, Bill Dally expressed that NVIDIA witnessed an astonishing 1000-times performance increment in computing chips during the last decade. Going by the books, such gains wouldn't have been possible with Moore's Law, and the cherry on top, process shrinking, had no impact on this figure. Now, you might ask me how this became a possibility, and the answer I would give here is prioritizing innovation within individual "stacks" instead of chip development.
To back my statement, NVIDIA says in their blog post that introducing "Hopper architecture" was decisive in showing the huge performance figures since they utilize "eight- and 16-bit floating point and integer math". To step things up, unveiling "Ampere architecture" aided performance in statistical learning, bringing in a 2x performance bump in computing workloads. To chain the individual technologies together, NVIDIA's "NVLINK" came in handy, which, in total, led to the x1000 mark.
Now, coming to the crux, NVIDIA's blog post mentions that throughout the 10-year period, the company switched from a 28nm to a 5nm process, accounting for a 2.5x performance gain only. This negates Moore's Law, which expects a 2x year-over-year increase every time a chip "shrinks." Dally said that NVIDIA's future depends upon "Huang's Law," which brings several industry advancement opportunities.
"It's a fun time to be a computer engineer," expresses Dally, and the industry's situation indeed validates the fact. We can say that the computing industry is at a defining moment, and it all depends on how companies approach "chip and computing development" moving forward.
Post a Comment
0 Comments