India is deploying facial recognition and algorithmic policing tools at scale without a governing framework, where unaddressed opacity and discriminatory error risk making efficiency come at the cost of justice
Artificial intelligence and biometric tools have moved from proposal to practice in Indian law enforcement, a shift that has been underway since at least 2017-2018 and has only gained urgency over time. After the 2020 Northeast Delhi riots, investigators turned to facial recognition technology to identify those involved. The results were uneven. In one case, a man spent four and a half years in custody before securing bail. His detention rested substantially on an 80 percent similarity score generated by Facial Recognition Technology (FRT) from a CCTV frame. Delhi Police deployed FRT in more than 750 riot investigations, yet over 80 percent of those that reached a verdict ended in acquittal or discharge.
The state has structural reasons to lean on automation. The India Justice Report records a police-to-population ratio stalled at 155 per 100,000, well short of the United Nations benchmark of 222. Bihar manages roughly 81 per 100,000, and 22 percent of sanctioned posts remain vacant nationally. In an environment of chronic under-resourcing, algorithmic assistance is easily presented as an administrative remedy.
These methods are far from foolproof; over-dependence risks blind policing. Algorithmic policing in India has expanded without an anchoring statute. The Project Panoptic tracker documents 170 facial recognition systems commissioned across agencies, though only around 20 are operational, at a cumulative outlay of ₹1,513 crore. Punjab’s PAIS searches over 390,000 records and 84,000 voice samples, Uttar Pradesh’s Trinetr holds more than 900,000 records, and Telangana issues TSCOP units to officers for real-time biometric matching. In 2018, under the Delhi High Court’s direction in Sadhan Haldar v. NCT of Delhi, the police acquired FRT for a single benign purpose, tracing missing children, with no other usage sanctioned. Affidavits put the system’s accuracy at 2 percent in 2018, falling below 1 percent the next year. The technology was nevertheless deployed for broader identification work.
The Project Panoptic tracker documents 170 facial recognition systems commissioned across agencies, though only around 20 are operational, at a cumulative outlay of ₹1,513 crore.
The burden of this expansion is not evenly distributed. Spatial studies of CCTV density show heavier concentrations in neighbourhoods with significant minority populations. Models trained on digitally rich populations, meanwhile, perform poorly against the markers of disadvantage most salient in India, particularly caste and religion.
An algorithmic system converts data inputs into a probability score or ranked output. The inputs and outputs are visible; what happens between them is not. The opacity is structural, designed into how these systems are built and commercially protected. Vendors routinely invoke trade secret protections to shield model architecture, feature weighting, and decision thresholds from external scrutiny. The system performs the most consequential act, classifying a person as high-risk, suitable, or suspicious, behind a wall that neither the affected individual nor the reviewing authority can examine. The problem deepens with training data. Machine learning models do not classify from first principles; they identify patterns in historical data and apply them to new cases. A risk assessment tool trained on past arrest records does not learn who poses a risk. It learns who has historically been arrested, from a dataset shaped by where police were deployed, which communities were monitored, and which offences were prosecuted most aggressively. The output looks statistically neutral while replicating the logic of the data that produced it. The ability to examine the reasoning behind a finding is foundational, and opacity defeats it. A person cannot contest a classification they cannot interrogate.
The output looks statistically neutral while replicating the logic of the data that produced it. The ability to examine the reasoning behind a finding is foundational, and opacity defeats it. A person cannot contest a classification they cannot interrogate.
The deploying organisation is often no better placed: it cannot explain the model’s reasoning in terms a court would accept, and the developer may invoke trade secrecy to withhold the parameters. No one in the chain can be held responsible for explaining why. Nor are the resulting errors random. They fall on the groups long targeted by disproportionate policing, who carry that history forward into the scores now used to judge them. The architecture conceals these patterns.
Machine learning generates probabilistic similarity scores rather than definitive identifications. Treating an 80 percent match as substantive proof overlooks the unequal distribution of errors across demographic groups. The American Civil Liberties Union’s 2018 audit of Amazon Rekognition is instructive. At an 80 percent confidence threshold, the system falsely matched 28 members of the United States (US) Congress with arrest photographs. Nearly 40 percent of those false matches involved people of colour. A threshold that produces demographically skewed false positives cannot satisfy the burden of proof beyond a reasonable doubt.
Trade secret protection, unlike a patent, requires no disclosure and can run in perpetuity. Architecture, training data, feature weights, and thresholds are precisely what a court would need to assess a contested match. Yet the only route to inspection, reverse engineering, remains closed to a defendant or a reviewing magistrate. The Organisation for Economic Co-operation and Development (OECD)’s guidance points in the opposite direction, recommending that true and false positive rates be recorded and made open to scrutiny. A privacy intrusion under K.S. Puttaswamy v. Union of India must satisfy legality, legitimate aim, necessity, and proportionality. A system whose accuracy cannot be inspected cannot credibly establish the latter two.
A threshold that produces demographically skewed false positives cannot satisfy the burden of proof beyond a reasonable doubt.
That uneven distribution of error is not merely technical; Indian constitutional law has a developed answer. Articles 14 and 15 now reach beyond intentional discrimination to indirect discrimination, facially neutral measures that disproportionately burden disadvantaged groups, recognised by the Supreme Court in Lt. Col. Nitisha v. Union of India. Adopting the two-step test from Fraser v. Canada, the Court asks whether a measure disproportionately affects a protected group and whether it reinforces that group’s historical disadvantage. Where the disproportionate effect is apparent, claimants may rely on robust judicial common sense rather than internal data they cannot obtain. That doctrine is well suited to challenging an opaque model whose outcomes reveal the discrimination its weights conceal.
The OECD classifies law-enforcement bodies among the highest-risk AI users and notes that, of 61 cancelled public-sector automation projects across regions, half concerned policing, many cancelled following the withdrawal of facial recognition systems. Its caution about opaque variables is concrete: a predictive tool used by the French Gendarmerie draws on 15 socio-demographic variables said to correlate with crime, with no transparency and no demonstrated causal basis.
The European Union (EU)’s AI Act, in force since 2025, treats real-time remote biometric identification in public spaces as an unacceptable-risk practice, permitting it only under narrow, judicially pre-authorised exceptions. Argentina went further. A court suspended Buenos Aires’s facial recognition system in 2022 after finding more than 9.3 million unauthorised biometric requests and barred its return until oversight and impact-assessment mechanisms were established; the decision was upheld on appeal in 2023. The United Kingdom offers a quieter lesson: Durham Constabulary retired its Harm Assessment Risk Tool in 2020, citing the cost of meeting the required oversight standard. The doctrine of disparate impact, from Griggs v. Duke Power Co., targets neutral practices that disadvantage a protected group regardless of intent. The capabilities approach adds the next question. Even where a tool serves a genuine public purpose, it should not foreclose the affected community’s capability to move, work, and participate without presumptive suspicion. Concentrated facial recognition in minority wards struggles on both counts.
Moving to genuine data governance requires a few binding measures.
Reorient procurement toward administrative assistance: The Ministry of Electronics and Information Technology and the Ministry of Home Affairs could prioritise low-risk applications, such as case-file management and the redaction of bystander faces in evidence footage, over public identification at scale.
Governed with care, AI can ease the operational burden on India’s police. Two conditions are essential: its risks must be acknowledged and addressed before deployment, not after, and the systems must be designed to be transparent.
Legislate evidentiary thresholds: Parliament, or High Courts exercising their rule-making powers, could amend the Bharatiya Nagarik Suraksha Sanhita to prohibit arrests or extensions of custody based solely on a facial recognition match.
Establish statutory audit rights for regulators: Disparate (Algorithmic) Advantage demonstrates that the obstacle is informational rather than legal. Equality law can already address algorithmic discrimination; regulators and claimants simply lack visibility into it. The remedy is institutional infrastructure. An audit right would allow a designated regulator to access aggregate outcome data disaggregated by protected group, alongside mandatory outcome reporting by deploying agencies. Such a right would disclose outcomes, not source code or trade secrets, and reveal disparate impact without undermining legitimate commercial protections.
Require transparency registers and sunset clauses: Every force could publish a register of deployed AI systems, listing the vendor, the location, and error rates disaggregated by demographic group. No monitoring deployment should hold a perpetual licence. Each should expire within a defined period unless recertified for fairness.
Governed with care, AI can ease the operational burden on India’s police. Two conditions are essential: its risks must be acknowledged and addressed before deployment, not after, and the systems must be designed to be transparent. At present, India’s algorithmic policing is advancing faster than its safeguards. Used strictly for administrative support and anchored in transparency registers, disaggregated error reporting, and independent audit rights, AI can serve justice rather than quietly undermine it.
Purushraj Patnaik is a Research Assistant with the Centre for Digital Societies at the Observer Research Foundation.
The views expressed above belong to the author(s). ORF research and analyses now available on Telegram! Click here to access our curated content — blogs, longforms and interviews.
Purushraj Patnaik is a Research Assistant with the Centre for Digital Societies at Observer Research Foundation (ORF). His research focuses on the governance of emerging ...
Read More +
PREV
