RBI-regulated entities have four realistic places to run AI workloads: public hyperscaler cloud, sovereign cloud hosted in India, a private or hybrid GPU cluster, and on-premise infrastructure. Three things decide the choice: whether the workload touches personal or payment data, whether you fine-tune models on proprietary data, and your inference volume. Most entities end up running a mix.
The question nobody is answering
There is no shortage of writing on what India's AI regulation says. RBI's FREE-AI framework, released on 13 August 2025, has been summarised to death. The Digital Personal Data Protection Rules, notified on 14 November 2025 with an eighteen-month phased compliance runway, have been explained by every law firm in the country.
What nobody has written is the next question, the one that actually lands on an engineering leader's desk. Fine. Given all of that, where do you run the thing?
This guide compares the four options on the criteria that decide it in practice.
What regulated entities are actually doing
Before comparing options, it helps to know where the sector really is. RBI surveyed regulated entities before publishing the FREE-AI report, and the numbers are more sobering than the discourse suggests.
The Department of Supervision surveyed 612 supervised entities between February and May 2025. Of those, 20.8% (127 entities) were either using or developing AI systems. Not deploying at scale. Using or developing. Among the 171 NBFCs in that sample, only 27% were using AI in any manner at all. Tier 1 urban co-operative banks reported none; Tier 2 and Tier 3 UCBs came in below 10%.
Those 127 entities reported 583 AI applications in production and under development, concentrated in customer support (15.6%), credit underwriting (13.7%), sales and marketing (11.8%) and cybersecurity (10.6%). Most of it was simple rule-based logic and moderately complex machine learning. Advanced models were rare in production.
A separate survey by the FinTech Department covered 76 entities representing over 90% of banking and NBFC assets. There, 67% were exploring at least one generative AI use case, which tells you the pipeline is full even though production is thin.
On hosting, one number is worth quoting precisely, because it gets misused constantly: the report found "a preference towards cloud-based deployments for lower cost, scalable solutions and expansion of digital services, with 35% respondents using the public cloud."
That is all it says. It does not break down what the other 65% are doing, and it does not attribute anybody's hosting choice to data-control anxiety. Plenty of vendor content has read that 35% as proof that regulated entities are quietly fleeing public cloud. That is an inference dressed up as a finding, and we're not going to make it. What the number supports is narrower and still useful: public cloud is common but nowhere near universal for AI model hosting in Indian financial services, and there is no default everyone has converged on.
The governance numbers in the same report are the ones that should actually worry you. Among entities running AI: 15% used interpretability tools like SHAP or LIME, 18% maintained audit logs, 35% validated for bias and fairness, and only during development, not in deployment, 21% monitored for data or model drift, and 14% did real-time performance monitoring.
Read that alongside the FREE-AI framework's direction of travel and the implication is uncomfortable. The audit burden is coming, and most of the sector has not built the infrastructure to carry it. Where you run a workload determines how much of that observability you get to control.
The three criteria that decide it
Most comparison content scores cloud options on a dozen dimensions. In regulated financial services, three do the deciding.
1. Does the workload touch personal or payment data?
If not internal document search over public filings, code assistance, marketing copy, this is mostly an economics question and you have wide latitude. Optimise for whatever makes your developers fast.
If it does, the analysis changes shape. Under Section 8(1) of the DPDP Act, a Data Fiduciary is responsible for compliance "irrespective of any agreement to the contrary" for any processing undertaken by it or on its behalf by a Data Processor. Section 8(2) permits engaging a processor only under a valid contract. Outsourcing the processing does not outsource the liability, and thin vendor paperwork is where that bites.
RBI's framework points the same way. It notes that where a regulated entity has outsourced a service and the provider uses AI to deliver it, that constitutes outsourcing — and that existing outsourcing agreements do not explicitly cover AI-specific governance, risk mitigation, accountability and data confidentiality. The report is blunt about the layer below, too: there are risks from "the vendor's subcontractors over which financial institutions have even more limited visibility and control." With a managed model API sitting on a hyperscaler, that chain can run three parties deep before anyone notices.
2. Are you fine-tuning on proprietary data?
Running inference against a general-purpose model is one risk profile. Fine-tuning on your bureau data, repayment history, claims corpus or transaction records is something else. You are now manufacturing an artefact that encodes customer data in model weights.
Your auditor will ask four questions. Where do the training data, the intermediate checkpoints and the final weights physically sit? Who else can reach them? Can the weights be deleted? Can you evidence the deletion?
That last pair matters more than teams expect. Section 8(7) of the DPDP Act requires a Data Fiduciary to cause its processors to erase personal data made available to them. Under a managed fine-tuning API, several of those answers are some version of "the vendor's environment, per their published policy." On dedicated infrastructure, all four are "our tenancy, our controls, here is the log."
3. What is your inference volume?
This is the criterion everyone ignores at pilot stage and nobody survives ignoring at production stage. A fraud model scoring 200 transactions a day costs nothing to run anywhere. The same model scoring 10 million transactions a month has unit economics that decide whether it ships.
Two line items dominate, and a proof of concept hides both: cost per million tokens at sustained volume, and data transfer charges. Egress is the quiet one. It barely registers in a pilot, because a pilot moves a trickle of data. Production means moving data between your core systems and the inference layer continuously, forever, and the bill scales with it.
The four options compared
| Criterion | Public hyperscaler | Sovereign cloud (India) | Private / hybrid GPU cluster | On-premise |
|---|---|---|---|---|
| Data custody | Residency available; foreign-parent jurisdiction remains | Indian entity, Indian law, Indian DCs | Dedicated tenancy, contractual isolation | Total control |
| Cost at low volume | Low | Low | Poor (committed spend) | Very poor |
| Cost at high volume | High (egress + on-demand) | Moderate | Low | Low if utilisation is high |
| Egress charges | Yes, structural | Often none | Typically none | N/A |
| Managed services | Deepest catalogue | Narrower; often self-service | Varies by provider | You build it |
| Fit for fine-tuning | Weakest custody answer | Good | Strongest | Strongest |
| Time to first workload | Hours | Hours | Days | Quarters |
| Capex | None | None | None | Substantial |
Public hyperscaler cloud
Where it wins. Deep managed-service catalogues, mature tooling, large partner ecosystems, and Indian regions that satisfy basic data-residency requirements.
Where it hurts. Three things, in ascending order of seriousness. Egress becomes a structural cost line at production volume. Foreign-parent providers carry jurisdictional exposure — whether lawful access requests under a home country's legislation could reach your data is now a standard question from Indian regulators, and picking an Indian region does not answer it. And sector-wide dependence on a handful of foreign providers is a concentration risk RBI's own report names: it warns of "concentration risk that arises on account of a limited number of dominant vendors," and its recommendations lean toward shared financial-sector data and compute infrastructure as a partial counterweight. The report also raises, without settling, whether India needs indigenous foundation models for the financial sector, noting that reliance on foreign models could expose systemic vulnerabilities.
Best fit: workloads that never touch regulated data; teams that need the full managed catalogue now; organisations where shipping speed genuinely outweighs custody.
Sovereign cloud in India
Cloud infrastructure operated by an Indian-domiciled provider, data resident in Indian data centres, contracted under Indian law.
Where it wins. It removes the jurisdictional question instead of mitigating it — there is no foreign statute under which an access request could reach the infrastructure. It speaks directly to the concentration-risk concern. It is usually cheaper than a hyperscaler, sometimes by a lot, and often without egress charges, which changes production economics rather than pilot economics. And billing in INR with GST-compliant invoicing removes a procurement headache that sounds trivial right up until your finance team raises it in month three.
Where it hurts. Managed-service catalogues are narrower. Some providers hand you infrastructure and a self-service operating model, which is a problem if your team has no appetite for running Kubernetes. Third-party integration depth is thinner. Before you assume parity, ask specific questions: managed operations or self-service? DR architecture? Contractual RTO and RPO? A named technical contact, or a ticket queue?
Best fit: regulated workloads at meaningful volume, fine-tuning on proprietary data, and organisations with a capable platform team or an SI partner.
Private or hybrid GPU cluster
Dedicated GPU capacity in a provider's data centre, reserved to you, network-isolated.
Where it wins. Dedicated tenancy makes the custody answer unambiguous. Your data, your models, your weights, on hardware allocated to you. No noisy-neighbour variance, which matters when you are scoring transactions in real time and have a latency budget in milliseconds. Reserved capacity prices well below on-demand once utilisation is sustained. And you get the custody story without the capital expenditure (the usual argument for on-premise, minus the balance sheet consequence).
Where it hurts. Committed spend means a capacity forecast, and forecasting is uncomfortable when the workload is nine months old. Under-utilisation is expensive and entirely your problem. Scaling past the committed cluster takes a conversation, not an API call.
Best fit: production workloads with predictable volume; ongoing fine-tuning programmes; anything where sub-100ms latency is a functional requirement rather than a nice-to-have.
On-premise
Where it wins. Maximum control, and no third-party processor in the chain at all. It is the simplest possible answer to a custody question.
Where it hurts. Capital expenditure against a 3–5 year refresh cycle, in a hardware generation that turns over roughly annually. Power and cooling most enterprise facilities were never designed for. And the need to hire and keep infrastructure engineers who can actually run GPU clusters, which is a scarce and expensive skill set that you will be bidding for against every AI company in the country. For most institutions the total cost of ownership does not clear the bar unless utilisation is high and stays high.
Best fit: the largest institutions, with existing data centre operations and workloads at a scale that amortises the investment.
A practical way to decide
Sort your workloads before you shortlist providers.
- No regulated data, any volume → public cloud is fine. Optimise for developer velocity and stop thinking about it.
- Regulated data, low volume, no fine-tuning → sovereign cloud, on demand. Custody resolved, nothing committed.
- Regulated data, high volume, or any fine-tuning → private or hybrid cluster. This is where custody and unit economics finally point the same direction.
- Extreme scale, existing DC operations → model on-premise properly, on total cost of ownership. Include power, cooling and salaries, and be honest about your utilisation assumption.
Most institutions land on a mix. That is a sound outcome, not a failure to decide. What is hard to defend in front of a regulator is running everything on whichever platform your first pilot happened to use.
