Why Build Foundational AI Models?
Foundational AI models are AI models that form the building blocks behind AI applications. Foundational AI models can be of various types, such as, large language models (LLMs), large multimodal models (LMMs), vision models, automatic speech recognition models, video, audio and image synthesis models, and others. Examples of these would be Mixtral 8x7B and the other variants of the Mistral AI ecosystem, Llama2 and AudioCraft by Meta, Stable Diffusion and Stable Video Diffusion, and others.
The current generation of foundational models can perform a wide range of tasks across different domains and modalities, such as natural language processing, computer vision, and speech recognition.
One of the most popular and powerful architectures for building such models, especially the Large Language Models (LLMs), is the transformer, which was introduced in 2017 by Vaswani et al in the paper ‘Attention Is All You Need’. The transformer is based on the idea of self-attention, which allows the model to learn the relationships between different parts of the input and output sequences. The transformer consists of two main components: an encoder and a decoder. The encoder processes the input sequence and generates a representation that captures its meaning and context. The decoder generates the output sequence by attending to both the encoder representation and the previous outputs. The transformer has been used to create state-of-the-art models for various tasks, such as machine translation, text summarization, image captioning, and natural language generation.
In the ongoing AI boom, foundation models have become pivotal, as they are used as the core intelligence behind a range of AI applications. Startups that have launched foundational AI models have gone on to become some of the fastest growing technology companies in history. Their inherent capabilities help build powerful moats for the innovators creating them. Building and launching foundational AI models have become the holy grail for many fledgling AI startups.
What Does It Take to Build a Foundational AI Model
Building a foundational AI model is a complex and challenging task that requires a combination of skills, knowledge and resources. A foundational AI model should also be able to learn from data, generalize to new situations, and adapt to changing environments.
Some of the steps involved in building a foundational AI model are:
Defining the Problem and the Objectives: The first step is to identify the problem that the AI model is supposed to solve, the scope of the solution, and the desired outcomes. This step also involves defining the metrics and criteria for evaluating the performance and quality of the model. This is also key because it will define the next steps. The process of building a general purpose AI model differs from those specific to a particular domain.
Collecting and Preparing the Data: The next step is to gather the data that would be used to train and test the model. The data should be relevant, representative, and diverse enough to cover the different aspects of the problem. The data should also be cleaned, annotated, and formatted according to the requirements of the model. Large numbers of open datasets are now available on Hugging Face, Kaggle, and other platforms. Furthermore, it is also possible to generate synthetic data, which can assist in filling gaps in existing data, or help in situations where data is hard to come by. This step may take from a few days to a few weeks, depending on the availability and quality of the data.
Choosing and Designing the Model Architecture: The third step is to select and design the model architecture that will best suit the problem and the data. The model architecture refers to the structure and components of the model, such as layers, nodes, activation functions, etc. The model architecture should be able to capture the features and patterns in the data, as well as handle the complexity and variability of the problem. Some of the most popular architectures are the transformer architecture and its variations, the Mixture of Experts or Ensemble of Experts approach, the GAN and diffuser architectures. This step may take from a few days to a few weeks, depending on the complexity and novelty of the model, and depends heavily on the expertise of the AI researchers working on the problem.
Training and Testing the Model: The fourth step is to train and test the model using the data. Training involves feeding the data to the model and adjusting its parameters to minimize errors between its predictions and the actual outcomes (a process known as fitting). Testing involves evaluating the model on new and unseen data to measure its accuracy, robustness, and generalization ability. Often, a section of the dataset is carved out for testing, and the rest is used for training. Training a model often can go into weeks or months of effort, and therefore access to advanced GPUs is key. This step varies widely, and may take from a few hours to several months, depending on the size of the data and the computational resources available. With high-end GPUs, the training time shortens drastically, therefore reducing cost and increasing efficiency.
Deploying and Maintaining the Model: The final step is to deploy and maintain the model in a production environment. This step involves integrating the model with other systems and applications, monitoring its performance and behavior, and updating it as needed to improve its functionality and reliability. This has emerged as an entire domain of specialization known as MLOps. Often the model would be set up in a way that the usage data can be harnessed for adapting the model’s future generations. For instance, the Llama2 model outperforms the Llama model on several benchmarks. This step may take from a few hours to a few days, depending on the deployment platform and the scalability requirements.
Choice of ML Training Infrastructure
In the entire lifecycle of foundational AI model building, the capability of the GPUs used for training is a major factor. The training process of a foundational AI model involves a large amount of data and computation, which can vary significantly depending on the hardware used.
For instance, a low-end GPU like V100 can perform around 14 teraflops of single-precision floating-point operations per second, while a high-end GPU like HGX H100 can achieve up to 160 teraflops. This means that the HGX H100 can process more than 10 times faster than the V100, reducing the training time from weeks or months to days or hours. The difference in performance is due to several factors, such as the number of cores, the memory bandwidth, the interconnect speed, and the cooling system. The HGX H100 is designed specifically for AI supercomputing, with 8-16 GPUs connected by NVLink and NVSwitch, allowing for high-speed data transfer and parallel processing. The V100, on the other hand, is a general-purpose GPU that can be used for various applications, but has lower efficiency and scalability for AI workloads.
Furthermore, to achieve highest performance during training, one should consider using multi node GPU clusters which use InfiniBand. These clusters are designed to handle large-scale image, language, and speech models, which require significant computational resources and high-speed interconnectivity. Essentially, InfiniBand adapters provide GPU Direct RDMA (Remote Direct Memory Access), allowing for direct memory access between GPUs on different nodes without the need for intermediate copies. This significantly reduces the time required for data transfer and model training. Also, by distributing the training workload across multiple nodes, AI model training can be accelerated, allowing for faster convergence and better performance. Finally, by partitioning the training data into manageable units and distributing the workload across multiple nodes, InfiniBand-powered multi-node GPU clusters can optimize resource utilization and minimize idle time.
Top Cloud GPUs for Building Foundational AI Models
As we saw above, the cloud GPU infrastructure available to a startup during its growth stage completely redefines the time and efficiency of the training process. Therefore, the biggest decision point for a startup building any foundational AI model is the accelerated cloud platform they are building on.
To understand this, let’s look at the two most capable cloud GPUs currently available for AI building, both of which are available on E2E Cloud at an incredible price-performance ratio: the A100 and the HGX H100.
The A100 is based on the NVIDIA Ampere architecture, which delivers up to 20 times the performance of its predecessor, the V100 GPU. The A100 GPU also features several innovations, such as the Multi-Instance GPU (MIG) technology that allows multiple workloads to run on a single GPU, and the third-generation Tensor Cores that accelerates AI and scientific computing. On E2E Cloud, the A100 cloud GPU is available on instant access at a competitive rate of ₹226/hr.
Several of the most well-known AI models have been trained on clusters of A100 GPUs, and it was the most powerful GPU technology available until the HGX H100 arrived. Let’s understand why.
First, the HGX H100 has a new Transformer Engine that supports FP8 precision, which enables up to 5x faster training for large language models. This feature allows the HGX H100 to handle trillion-parameter AI models with ease.
Second, the HGX H100 has a faster and larger NVLink domain which connects eight H100 GPUs with four third-generation NVSwitches. This topology provides 900 GB/s of bidirectional bandwidth between any pair of GPUs, which is more than 14x the bandwidth of PCIe Gen4 x16. Moreover, the NVSwitches support in-network compute with multicast and NVIDIA SHARP, which accelerate collective operations like all-reduce by 3x compared to the HGX A100. Third, the HGX H100 also integrates NVIDIA Quantum-2 InfiniBand and Spectrum-X Ethernet for high-speed networking.
The HGX 8xH100 is available on E2E Cloud for Rs 445.2 per GPU/hr*, a rate that is again very competitive in the current market, where demand for this GPU is leading to massive wait times.
Why E2E Cloud Is at the Forefront of Foundational AI Building
There are several reasons why startups building a foundational AI model could consider choosing E2E Networks and its platform, E2E Cloud.
First, the price. Currently, there are two plans for HGX H100: one with 200 vCPUs, 1800 GB RAM and 21000 GB SSD for Rs 342.46 per GPU/hr, and another with InfiniBand, 200 vCPUs, 1800 GB RAM and 30000 GB SSD for Rs 445.2 per GPU/hr. Compare this to the fact that no other hyperscaler in India offers HGX H100 at the time of writing this article. The closest plan to HGX H100 might be eight A100 GPUs, which is available at a pricing upwards of Rs 1557 per hour.
In other words, the pricing of HGX H100 on E2E Networks is much lower than the other cloud providers that offer similar GPUs.
Second, the performance. Unlike other hyperscalers, E2E Cloud offers cloud GPUs that are close to metal, meaning they have direct access to the hardware resources with very little virtualization overhead. This results in faster and more efficient GPU performance, often surpassing the benchmarks of other cloud providers. We are soon planning to release a benchmark around this.
Third, the security and stability that comes from an NSE listing. E2E Networks, being an Indian NSE-listed hyperscaler, is 100% compliant with Indian IT laws, and ensures that the companies building on its platform aren't affected by interventions by foreign actors. This is a key fact to keep in mind, as the AI building process can involve training on sensitive data that forms part of the company’s core IP.
Furthermore, E2E Networks has also recently launched TIR, a platform that is designed ground up for simplifying the AI development process. TIR uses highly optimized GPU containers (NGC), pre-configured environments (PyTorch, TensorFlow, Triton), automated API generation for model serving, shared notebook storage, and much more. The purpose behind building TIR has been to simplify the life of AI researchers, who need to focus on the training process rather than deal with infrastructure.
Finally, the ecosystem. E2E Networks has been playing a central role in the AI ecosystem in India, organizing hackathons, workshops, offering credits, and enabling startups building AI in India. This supportive ecosystem is extremely valuable for early-stage AI startups.
Final Words
Foundational AI models are going to become the biggest moat that technology startups can create. Increasingly, as AI will touch upon every domain over the next decade, these models would form the building block of intelligence that would power a range of AI applications. To build a foundational AI model, startups need access to advanced cloud GPU servers, and that is not easy to access. E2E Cloud is simplifying the process by offering advanced cloud GPU servers at incredible prices to startups, helping them scale the training process efficiently and in a cost-effective manner. Get in touch with us at sales@e2enetworks.com and discuss more with our sales team.
