India’s AI future depends on Edge AI to deliver real-time, localised, and privacy-aware intelligence at scale, but achieving this requires stronger infrastructure, governance, interoperability, and indigenous innovation
India’s artificial intelligence ambitions are increasingly tied to semiconductor manufacturing and high-performance computing. The current approach prioritises compute capacity but overlooks a critical layer: how intelligence is deployed at scale. While India has around 1.8 lakh AI startups, most remain stuck at the pilot stage, and only a few domain-specific cases have successfully scaled.
Edge AI, a type of distributed intelligence, can close this gap and bring intelligence to the last mile, delivering local solutions even amid fragmented connectivity. This enables real-time decision-making, reduces latency, and allows data to be collected directly at the source without being sent to the cloud. Further, edge devices are increasingly viewed as a more sustainable alternative to cloud-heavy AI systems. This is due to AI workloads being processed directly on devices, minimising continuous data transfer, and the improved power efficiency of edge-specific chips.
However, these sustainability gains are not automatic. Edge systems also face challenges related to energy efficiency at scale, hardware density, and resource orchestration, underscoring the importance of careful system design in delivering their full environmental benefits. Another key advantage is reducing network latency: in high-stakes systems, latency itself becomes a safety boundary. For instance, in mobility, India’s Kavach, deployed across thousands of kilometres of railway track, relies on on-board sensors and edge computing to help ensure zero signal-passed-at-danger incidents.
In healthcare, solutions such as Niramai’s Thermalytix demonstrate how edge-enabled AI can bring diagnostics closer to patients, enabling real-time screening in low-resource settings. This reduces reliance on centralised infrastructure while allowing for on-site image processing. The solution also eliminates the need for radiation-based screening and physical contact. Results are generated within minutes, enabling immediate risk assessment, which is particularly important in resource-constrained rural environments.
In healthcare, solutions such as Niramai’s Thermalytix demonstrate how edge-enabled AI can bring diagnostics closer to patients, enabling real-time screening in low-resource settings.
These examples demonstrate that in critical systems, intelligence should increasingly reside where decisions are made. More importantly, they underscore the value of privacy and data sovereignty as outlined in India’s Digital Personal Data Protection (DPDP) Act. Further, distributed intelligence is central to the next generation of systems. Moreover, India is well-positioned to build on its digital public infrastructure to develop a hybrid edge–cloud ecosystem. This can deliver real-time services at scale while maintaining low costs and resilience.
Edge AI implementation also presents several challenges:
Edge deployments in India face several unconventional challenges. These span environmental, behavioural, and usability constraints, directly affecting their performance and adoption. High ambient temperatures, often exceeding 40°C, can cause thermal throttling and hardware degradation in uncooled devices. Intense monsoon conditions can also physically displace or damage sensors.
Beyond technical factors, trust deficits and low digital literacy—especially in rural contexts—limit adoption, as users are often hesitant to rely on automated systems due to previous negative experiences or poor UI/UX design. This is compounded by poor transparency and explainability, as AI outputs are often not easily interpretable or actionable for end users. Further, vague accountability and liability frameworks reinforce the existing trust deficit. Together, these challenges make edge AI a complex technology to implement.
Edge AI also introduces a security and governance paradox. Thousands of distributed devices deployed in remote, often unsecured environments are harder to monitor, update, and protect, making them vulnerable to tampering, theft, and physical damage. At the same time, managing these systems at scale remains a challenge. Robust edge MLOps (Machine Learning Operations) capabilities, such as reliable over-the-air (OTA) updates and remote monitoring across multimodal inputs, remain underdeveloped in India. This limits the ability to maintain performance and keep devices updated and secure across large deployments spanning vast regions.
This challenge is compounded by model bias embedded in foreign models or arising from the saturation of particular data types. Today, many edge applications rely on highly localised datasets that are incomplete or difficult to integrate, risking reduced accuracy and reinforcing existing biases. Vulnerable communities, ethnic groups, and socio-economic classes are often excluded from these datasets. Together, these factors create distinct vulnerabilities in edge AI systems, requiring new approaches to distributed system management and data governance.
The current data centre landscape remains heavily concentrated in metros such as Mumbai, Bengaluru, Chennai, and Visakhapatnam, largely due to their proximity to submarine cable landings, industrial hubs, and established digital infrastructure. India currently has fewer than 10 micro data centres (MDCs), although initiatives such as BSNL’s local MDCs and the Open Cloud Compute (OCC) project aim to build 10,000 such edge facilities to support AI and local data processing. However, inconsistent power supply and limited underground fibre connectivity remain significant hurdles to edge deployments. These regions also often lack the fibre density required to support low-latency 5G and AI edge applications.
Expanding local micro data centres (MDCs) is critical to addressing key infrastructure gaps: these facilities enable faster processing closer to the user, reduce bandwidth and energy costs associated with long-distance data transfer, improve system resilience in low-connectivity environments, and support distributed workloads across sectors such as healthcare, agriculture, defence, and mobility.
Without interoperability, edge systems cannot aggregate or act on data collectively, undermining the premise of real-time, system-wide decision-making and limiting the effectiveness of distributed AI.
High upfront capital costs for specialised edge hardware remain prohibitive for small and mid-sized enterprises, particularly in sectors such as logistics, even with enabling platforms like the Unified Logistics Interface Platform (ULIP). At the same time, the lack of standardisation across devices and platforms creates significant interoperability challenges. Vendors often rely on proprietary protocols, resulting in fragmented data silos in which systems cannot communicate seamlessly. This prevents the integration of distributed nodes—such as drones, warehouses, sensors, and vehicles—into a unified intelligence layer. Without interoperability, edge systems cannot aggregate or act on data collectively, undermining the premise of real-time, system-wide decision-making and limiting the effectiveness of distributed AI.
Currently, India faces a shortage of indigenously designed Analog-Mixed Signal Integrated Circuits (AMICs). These combine digital and analog signals and are especially important in audio and video equipment, automotive systems (such as radar and sensor fusion), and IoT sensors. However, several leading international startups have established a presence in India in recent years, attracting young Indian talent and exposing them to highly specialised work. Teams at IIT Madras are also researching analog, RF, and mixed-signal ICs and their real-world applications.
However, dependence on electronic design automation (EDA) tools persists, with most chip designs relying on proprietary tools from Synopsys, Siemens, and Cadence Design Systems. India is, however, actively exploring open-source EDA initiatives such as eSim at IIT Bombay. Further, startups such as Simyog Technologies are pioneering testing and validation solutions for early-stage hardware designs.
Lastly, post-silicon reliability infrastructure—a set of tools, hardware, and software used to test physical microchips after manufacturing and before deployment—remains underdeveloped. Currently, two fabs and eight ATMPs/OSATs have been approved; however, more domain-specific testing infrastructure is needed. Further, under the Semiconductor Mission 2.0, Qualcomm has completed the tape-out of a 2nm chip design, which is important for edge AI applications.
India faces a middleware and compiler deficit. Middleware serves as a layer that connects fragmented data, models, and workflows into a scalable system. The India AIKosh dataset is a crucial middleware platform, alongside enterprise solutions offered by Infosys and Wipro, among others. However, 45 percent of Indian enterprises continue to operate with low-maturity, highly fragmented data foundations due to hard-to-integrate data silos.
Compilers, such as Apache TVM and NVIDIA TensorRT, translate source code from complex machine learning models and enable users to adapt it to existing hardware, including GPUs.
There is still no unified, open, indigenous edge compiler stack capable of efficiently translating frameworks such as PyTorch or TensorFlow into optimised code for edge devices. However, multiple approaches are being pursued, particularly Multi-Level Intermediate Representation (MLIR), a novel open-source approach developed by PolyMage Labs that aims to reduce fragmentation across different hardware systems.
Global evidence shows that scaling AI depends as much on governance transformation as on technological capability. Advanced edge AI systems risk remaining confined to pilot projects rather than becoming embedded infrastructure due to systemic limitations. A key constraint in India is the limited in-house technical capacity within implementing agencies at the local level. This has contributed to an over-reliance on external vendors, while the state continues to face challenges in improving interoperability across its various databases.
Global evidence shows that scaling AI depends as much on governance transformation as on technological capability.
To address this, it is essential to build dedicated digital teams across ministries, states, and urban local bodies. Equally important is making digital literacy a core administrative capability. Officials at the district and state levels must be equipped to evaluate and deploy AI systems in increasingly data-driven governance environments. This would also help identify and nurture technical talent early, enabling a more inclusive and distributed innovation ecosystem.
India’s AI strategy must also extend to the district level, where deployment challenges are most acute. Local innovation ecosystems that incentivise student and graduate participation can generate context-specific solutions, with successful models scaled through national platforms. Finally, deployment must be rooted in local realities: systems should use vernacular interfaces, integrate with existing government data workflows, and align with DPI. Without these reforms, India risks building AI capacity without achieving real-world impact at scale.
Ishita Deshmukh is a Research Intern at the Observer Research Foundation.
The views expressed above belong to the author(s). ORF research and analyses now available on Telegram! Click here to access our curated content — blogs, longforms and interviews.
Ishita Deshmukh is a Research Assistant with ORF’s Centre for Security, Strategy & Technology. Her work focuses on how artificial intelligence is reshaping national security, economic ...
Read More +
PREV
