According to Korean media reports, Professor Kim Joung-ho of the Department of Electrical Engineering at the Korea Advanced Institute of Science and Technology (KAIST), known in Korean media as the "Father of HBM," stated that High Bandwidth Flash (HBF) is expected to become a key memory technology in the next-generation AI era and will develop in parallel with HBM, jointly driving the performance growth of major chip manufacturers.
HBF's design concept is similar to HBM, both utilizing through-silicon vias (TSVs) to connect multiple layers of stacked chips. The difference is that HBM utilizes DRAM as its core, while HBF utilizes NAND flash memory for its stacking, offering "higher capacity and more cost-effective" advantages. Kim Joung-ho noted that while NAND is slower than DRAM, its capacity is often over 10 times greater. Stacking hundreds or even thousands of layers effectively can meet the massive storage requirements of AI models, potentially becoming a NAND version of HBM.
Generative AI models are rapidly expanding, with the input token count for a single model already reaching millions, and processing requires terabytes of data. Insufficient memory bandwidth during these thousands of read and write operations per second can create bottlenecks, significantly slowing the response of large language models (LLMs) like ChatGPT and Google Gemini.
Kim Joung-ho emphasized that this limitation comes from the current von Neumann architecture. Since the GPU and memory are designed separately, the data transmission bandwidth determines the performance ceiling. "Even if the GPU size is doubled, it is meaningless if the bandwidth is insufficient."
He predicts that future GPUs will incorporate both HBM and HBF, forming a complementary architecture: HBM will serve as a high-speed cache for real-time data processing, while HBF will provide high-capacity storage, directly storing complete AI models. This will help overcome memory bottlenecks and enable GPUs to handle larger, more complex generative AI, even encompassing feature-length films. Kim Joung-ho stated, "In the future, AI will move beyond text and images and will be capable of generating feature-length films. The memory capacity required will be over 1,000 times greater than currently available."