Agentic AI Raises New Security Challenges for Federal Agencies

Federal agencies are strengthening AI and agentic system security through risk-based governance, continuous monitoring and safeguards for autonomous systems, officials said at GovCIO Media & Research’s Federal Cloud & Data Forum last week.

The Centers for Medicare & Medicaid Services (CMS) is the single largest payer for health care in the U.S. as it oversees Medicare, Medicaid, the Children’s Health Insurance Program and the Affordable Care Act Marketplace. Together, these programs hold the personal data of more than 160 million Americans at any given time, said Leslie Nettles, CMS’ acting deputy CISO.

“All of us will eventually be customers of CMS as we age. That’s our main mission: protecting that information and making sure our systems are secure for our beneficiaries and customers to use,” she said.

To protect these systems, CMS is shifting its focus from compliance to risk management, particularly as it deploys and secures AI systems. Nettles emphasized the importance of maintaining human oversight as AI adoption expands.

“AI is a force multiplier. It is definitely not a replacement for any individuals. We have to make sure that we have humans in the loop,” Nettles said.

One challenge is ensuring users critically evaluate AI-generated outputs, which can contain inaccurate information. To mitigate these risks, CMS is working to secure its AI systems while strengthening its cybersecurity workforce.

The agency is hiring and training personnel and building institutional knowledge to support its long-term cybersecurity efforts, Nettles said.

Continuous Monitoring at FedRAMP

The General Services Administration’s is also shifting toward a risk-based approach to cloud security. The agency’s FedRAMP 20x initiative aims to modernize cloud security authorizations by moving toward continuous, real-time monitoring, said FedRAMP Security Director Nicole Thompson.

The approach would give agencies faster access to relevant security metrics rather than relying primarily on traditional compliance documentation.

“Not just narrative updates about what a could service provider intends to do, but actual security metrics from their environment, because we live in the year 2026, and all those [resources] are plentiful,” Thompson said.

Cloud service providers already collect and continuously monitor large volumes of security data. FedRAMP aims to leverage those existing metrics rather than requiring separate compliance teams to generate additional reports. Thompson said FedRAMP 20x is designed to streamline that process, giving agencies more timely visibility into cloud security risks.

Setting Boundaries for Agentic AI

As agencies adopt more sophisticated AI systems, distinguishing legitimate automated activity from malicious traffic is becoming increasingly challenging, said Anish Patel, head of federal at Cloudflare.

This can occur when automated traffic from an agency or authorized users encounters a firewall or system that does not recognize the query. Unless it is ordered not to do so, the AI system might seek another way into a network to access the information it was ordered to retrieve.

“These are the challenges that government is dealing with. How do you protect that front door? How do you make sure that as it’s going through the side doors, you’re ahead of that?” he said.

One way to avoid these issues is by setting clear boundaries, said Justin Ubert, speaking in his capacity as professor of cybersecurity and cyber policy at Marymount University.

As agencies deploy AI agents, they must ensure that automated actions can be traced to their owners. This requires auditable records of agent activity and guardrails capable of stopping actions that exceed authorized parameters.

“This makes the constraints that we have on these [AI agents], whether its guardrails, various hooks in the environment, layer defenses — all of these things need to come together to ensure that these identities, as they’re operating through this context we give them, are able to be audited and then stopped when it’s appropriate,” Ubert said.