Introduction
In the realm of high-stakes technology, where major international companies typically dominate the AI field, India is carving out a notable niche for itself. The country is witnessing the emergence of its own Large Language Models (LLMs) and AI innovations, each boasting unique characteristics and capabilities. It's time to highlight these Indian innovations, celebrating their impressive features and significant advancements in the AI landscape.
OpenHathi
OpenHathi, deriving its name from the Hindi word for ‘elephant’, represents more than just a sizable language model; it stands as a testament to the increasing influence of Indian languages in the field of AI. Developed by Sarvam AI, this model with 7 billion parameters is the inaugural offering in the OpenHathi series. It's tailored to cater to a variety of applications within the Indian market. As the first Hindi Large Language Model (LLM) available to the public, OpenHathi signifies a crucial milestone in the advancement of AI in India.
OpenHathi is trained on Hindi, English, and Hinglish data, enhancing its ability to understand and generate content in these languages. Its unique tokenization approach combines a 16K Hindi vocabulary sentence-piece tokenizer with the Llama2 tokenizer, improving efficiency in processing Hindi text. The training involves three phases: first, bilingual text translation using low-rank adapters for cross-lingual comprehension; second, bilingual next-token prediction to aid in context-aware language generation; and third, supervised fine-tuning on specific tasks for versatility in applications. The base model, post-second phase, is openly accessible on Hugging Face, offering opportunities for further fine-tuning by developers and researchers. With its bilingual training, OpenHathi is well-suited for tasks involving Hindi-English translation and information retrieval, demonstrating strong cross-lingual capabilities.
Bhashini
Bhashini, an ambitious project by the Government of India, aims to revolutionize digital inclusivity by democratizing internet and digital services in various Indian languages. More than just developing Large Language Models (LLMs), Bhashini is a comprehensive program designed to integrate India's linguistic diversity with advanced technology, thereby overcoming language barriers. Its goal is to ensure digital empowerment for all citizens, regardless of their linguistic background. Currently in beta, the Bhashini app, available on Apple and Google Play stores, exemplifies the initiative's potential to transform sectors like education, healthcare, governance, and economic development. While challenges remain, including accessibility, ongoing technology development, and government support, Bhashini's vision is a promising step towards a future where linguistic diversity is a cornerstone of digital empowerment.
Tamil-LLAMA
Tamil-LLAMA, crafted by Abhinand Balachandran, is a specialized large language model for Tamil, building on the LLaMA model to enhance Tamil text processing. It features an expanded vocabulary with an extra 16,000 Tamil-specific tokens atop the original 32,000, ensuring nuanced language handling. The model employs the LoRA methodology for efficient, robust training and offers four variations (7B, 13B, 7B Instruct, 14B Instruct) to suit diverse requirements and computational resources. Further fine-tuning is achieved using a Tamil-translated Alpaca dataset and selected OpenOrca dataset portions, optimizing it for Tamil language tasks. Additionally, Tamil-LLAMA's open-source availability of its code, models, and datasets fosters research and development in Tamil language AI, making it a pivotal tool in this domain.
Kan-Llama
KanLlama, developed by Tensoic AI, is a groundbreaking NLP model tailored for the Kannada language, addressing the gap left by popular LLMs in handling Kannada. It's built on the 7 billion parameter Llama-2 model, with specific pre-training and fine-tuning for Kannada tokens. The model incorporates a specialized phrase fragment tokenizer, trained on Kannada texts, integrated with the Llama-2 tokenizer, enhancing its understanding and processing of Kannada. Employing LoRA techniques for efficient pretraining, KanLlama optimizes training while conserving weights of previous models. It demonstrates improved scalability and conversational abilities in Kannada, thanks to its optimized data structures. Pre-trained on 600 million Kannada tokens from the CulturaX Dataset using Nvidia A100 80GB instances, the model achieved a cost-effective and efficient training process, taking about 50 hours and costing around $170. KanLlama's advancement in Kannada language processing holds significant potential for enhancing public service accessibility and allowing users to engage with AI in their native language.
Krutrim
Krutrim, an ambitious project by the Ola group, aims to transform the AI arena in India and beyond by developing a comprehensive AI computing stack for individuals, businesses, and researchers. Central to its vision is the creation of advanced AI computing infrastructure, encompassing high-performance computing resources, AI accelerators, and cloud infrastructure. Krutrim's AI Cloud will offer accessible AI tools and resources, facilitating the development and deployment of AI applications without substantial hardware investment. The initiative is also focused on developing foundational models, including large language models, speech recognition systems, and computer vision models, tailored to India's linguistic and cultural diversity. These models will underpin a range of AI applications targeted at sectors like healthcare, education, agriculture, and finance, addressing the unique needs of India's diverse populace. Krutrim's overarching goal is to generate practical and impactful AI-powered solutions across various domains.
Project Indus
Tech Mahindra's Project Indus represents a significant leap in language technology, focusing on developing a pure Hindi Large Language Model (LLM) powered by AI, with an impressive 539 million parameters and a vast dataset of 10 billion tokens from Hindi and its dialects. Aimed at building an Open Source LLM, Project Indus targets revolutionizing language technology to serve a significant portion of the global population. It holds immense potential for sectors like rural finance, retail, and logistics in India. Starting with Hindi and its 37 dialects, the project lays a foundation for future expansion into more languages and dialects, transcending technological boundaries and fostering global inclusivity. This initiative is a testament to Tech Mahindra's commitment to bridging language divides and enhancing growth through innovative AI applications.
CoRover.ai
CoRover, a pioneering force in the AI industry, is recognized as the world's first human-centric Conversational and Generative AI platform, offering high ROI with its secure and scalable patent-pending technology that integrates AI, ML, NLP, AR, and VR. Its versatility is highlighted by Multi-Format capabilities, including AI VideoBots, VoiceBots, and ChatBots, and it supports over 100 languages, serving a vast user base of over 1 billion. A notable feature is its Video-Voice Commerce Virtual Assistants, which facilitate complete transactional processes. CoRover has enhanced its offerings with BharatGPT, a proprietary Generative AI for text, voice, and video, compatible with ChatGPT. Aimed at transforming user-system interactions to be as intuitive as human conversation, CoRover has gained recognition from global leaders like Microsoft’s Satya Nadella and India’s Prime Minister Narendra Modi, and collaborates with numerous Fortune 100 companies, marking a significant presence in the AI landscape.
Conclusion
As we conclude our exploration of India's AI advancements, it's important to commend the exceptional teams and minds behind these initiatives, from Sarvam AI's OpenHathi to Tamil-LLAMA, Ola group's Krutrim, Tech Mahindra's Project Indus, and the government-led Bhashini. Each project, while not all traditional LLMs, contributes significantly to India's diverse linguistic and technological landscape with unique features like multilingual support and domain-specific applications. This evolving story of India's AI journey invites further contributions and insights, celebrating every innovation that shapes an inclusive, technologically advanced, and culturally vibrant future.