GAO’s Take on AI Costs, Cyber Risks in Federal Tech Strategies

Government Accountability Office officials said federal agencies must address growing challenges around cost, governance and cybersecurity as AI adoption grows.

Speaking at GovCIO Media & Research’s AI Summit on Nov. 6, GAO Chief Scientist Sterling Thomas and Managing Director for IT and Cybersecurity Issues Nick Marinos outlined some of those challenges.

Federal agencies, particularly those in the executive branch, have largely relied on trial periods and promotional pricing offered by AI companies. Those arrangements are beginning to expire, exposing agencies to the true costs of AI use, Thomas said.

He noted that token-based pricing, which charges organizations based on the amount of data processed by AI models, can become tremendously expensive at scale.

This cost challenge has already emerged in the private sector, where companies have begun imposing limits on AI use.

“They went from ‘everybody use these tools’ and advertising how many tokens you use, to ‘get this many tokens a day and that’s it.’ So that is becoming a reality within the federal government,” Thomas said.

As agencies transition to paid AI services, Thomas emphasized the importance of evaluating deployment strategies to manage token consumption and control costs.

It’s a similar sentiment from the vendor side.

“Having access to things faster does not necessarily mean better,” noted Cognition Federal Civilian Director Justin Herman at an earlier session at the event. “If your contracts themselves are structured merely on consumption, guess what? You’re guaranteed to consume more. If you’re working with AI companies and setting firm fixed contracts that are actually performance and outcome-based, forget how many tokens are consumed, are you getting the problems you have solved, are you reaching the goals that you have and doing it in such a way that’s agnostic and not getting tied into any particular model or vendor?”

Strategies for Managing AI Costs

Thomas outlined several approaches agencies can take to reduce AI expenses, including optimizing how AI agents operate and ensuring they maintain context throughout their tasks. When agents lose context, he recommended stopping their operations and rebuilding that context before redeployment to avoid unnecessary processing and costs.

Thomas also encouraged agencies to consider open-weight models, which can offer a less expensive alternative to proprietary AI systems. Open-weight models make their underlying model parameters publicly available, allowing organizations to download and operate them on their own infrastructure.

However, Thomas noted that many existing open-weight models are developed outside the United States, limiting their availability to federal agencies. He added that U.S. companies are beginning to offer more of these models.

As agencies develop and deploy their own agentic AI systems, they could gain greater flexibility to experiment with different models and deployment strategies.

Still, Thomas warned that the financial realities of AI adoption could slow federal implementation over the next eight to 10 months as agencies adjust to the technology’s ongoing costs.

“You’re going to see a significant slowdown of use of AI until we deal with that policy,” he said.

People in the Loop

Beyond costs, GAO officials highlighted the importance of maintaining human oversight as agencies increasingly automate operations, particularly in cybersecurity.

As agencies implement guidance from the National Institute of Standards and Technology, many are integrating AI capabilities into existing cybersecurity architectures.

“I think the challenge continues to be that many of the risks we see existing within cybersecurity programs are ones that you need some kind of human intervention to help correct,” Marinos said.

Marinos warned that automated systems could make decisions based on inaccurate or incomplete data, potentially introducing new security risks. Additionally, agencies may struggle to protect their networks if AI capabilities are deployed without proper documentation or visibility into how they operate.

While many of these challenges predate AI, Marinos expects the technology to exacerbate existing cybersecurity risks while increasing the speed and sophistication of cyberattacks.

Recommendations for Government

GAO issued a series of recommendations for federal AI governance in 2023 after examining how agencies were adopting and managing the technology. Marinos said many of those recommendations remain relevant today.

One persistent challenge is that agencies lack comprehensive inventories of their AI use cases, making it difficult to determine where AI capabilities have been integrated into existing systems.

Marinos emphasized that maintaining accurate inventories is essential not only for cybersecurity and risk management but also for developing effective governmentwide AI policies.

Congress also has a role to play in modernizing federal cybersecurity legislation to account for emerging technologies, he said.

One potential area for reform is the Federal Information Security Modernization Act (FISMA), which was originally enacted in 2002 and updated in 2014. Marinos suggested that lawmakers could revisit the law in 2027 to better reflect the evolving cybersecurity landscape and growing use of AI.

He also emphasized the need for greater coordination between Congress and the executive branch, particularly as federal cybersecurity responsibilities continue to evolve.

“We’re still waiting to see that legislation [can] be updated, for example, to reflect the creation of the Office of National Cyber Director. … And I think that’s going to be really important because I think the way that we solve this problem is going to be through collaboration and coordination across the federal government,” Marinos said.