When Tackling AI Risk, Feds Say It’s All About Data, Governance

Artificial intelligence’s potential impact on federal systems is generating both enthusiasm and caution, with agency officials balancing the efficiency gains the technology can deliver with the governance needed to ensure intended outcomes.

With federal officials increasingly interested in the benefits of agentic AI, agency and industry leaders at GovCIO Media & Research’s AI Summit discussed how they are adjusting their risk tolerance when applying the technology to their missions.

“I’ll repeat this over and over again, trust in AI is our number one factor in considering whether to use it or not,” said Matheus Passos, senior advisor of technology and strategy at the Commerce Department, at the Nov. 6 event. “What we found on our user base, our business folks is that if they don’t trust that their data is safe and secure and private and protected, they won’t touch it. It does not matter what you tell them. It does not matter what you make available. It does not matter if this could make them be able to go golf at 3:00 on a Wednesday, they won’t touch it.”

Start with the Data

Before deciding where to deploy agentic AI, officials must understand how the technology will use the data sources it draws from, said Defense Health Agency Chief Data and Analytics Officer Jesus Caban.

“For us, everything starts with the data,” said Caban. “We can have the best IT capabilities out there, the best agentic models and so on, but the trustworthiness of those agentic models relies on the underlying data and the underlying governance of that data.”

Caban said DHA officials initially focused their data strategy on properly indexing and publishing authoritative datasets so AI systems know which datasets to use and trust.

“Are we heading into agentic space? Absolutely. Does that mean we will take open Claude and deploy it across the enterprise? No,” he said. “Building blocks. Start with the data, data quality, data strategy and so on and then all the way to the governance.”

When determining whether to apply agentic AI and its decision-making capabilities to a use case, Caban said DHA weighs three criteria: whether an agent’s decision can be undone, whether there is a single source of auditable truth for the agent’s actions and whether its use could cause patient harm.

“Those are the cases that are going through rigor, through the governance process before we start doing any kind of pilot in that space.”

Know What Success Looks Like

Before deploying agentic AI, agency officials must understand what they want to accomplish, said Alexis Bonnell, head of AI adoption and deployment at OpenAI for Government.

“One of the things that slows us down, if we are trying to be helpful, is there’s not a definition of success,” she said. So we love it when we meet a customer that’s like, ‘This is the golden example, this is success.’ … We can either say, ‘Please don’t use agents,’ or we can say, ‘That’s perfect, we can absolutely help you.’”

Bonnell said leaders should take an intentional approach to AI adoption before deploying agents. They should first focus on using AI to improve workflow efficiency, then determine what data they want AI to work with and identify where processes need to change before deploying agents.

“Really what AI/SI is is a relationship with knowledge at speed and scale. So really what you want to do is be quite intentional about that knowledge,” she said.