Skip to Main Content

How AI and Emerging Tech Can Clean Up Federal Security Clearance Mess

This backlog costs government and supporting vendors millions of dollars in lost productivity and overhead costs.
7m read
Photo Credit: fandijki/iStock & animation by Jerome Wiley/GovernmentCIO Media

Meagan Metzger is founder and CEO of Dcode. She will be speaking at GovernmentCIO Media’s CXO Tech Forum: Uncle Sam Meets Silicon Valley on March 15. 

The federal government faces a massive backlog of processing security clearances, which wastes money and severely restricts government innovation. In short, this backlog costs the government, and the vendors that support the government, millions of dollars in lost productivity and overhead costs.

Even more critically, the backlog of over 700,000 investigations has left countless of critical national security positions unfilled, so agencies can’t to do their jobs. This doesn’t just impact our men and women in uniform — the government spends billions of dollars on tech products and services that require personnel with security clearances to deliver. The security clearance backlog is killing innovation across government.

In a previous article, I highlighted action the government can take around policy and process. The pain caused by inefficiencies in the security clearance process is felt across government in wasted hours and critical jobs left unfilled, and some concrete steps are being taken to try to address these issues. The SECRET Act, which passed the House last year and is now pending before the Senate, would require more transparency in the process and new plans to cut the backlog through the simplification of forms and the digitization of investigative work still generally done in person with pen and paper.

While these are critical steps, the irony is that the same innovation kept out of government by the security clearance backlog can actually be the solution. The security clearance process itself faces many of the same problems tech has solved for large commercial firms. Federal agencies each manage their own security clearance process, leading to failures to share data and clearances across government. Investigators across the country and the world conduct research and investigations on separate systems that do not communicate.

In fact, the data collection itself is almost entirely manual: For each person going through the process, federal investigators interview his or her colleagues, friends, neighbors in-person with pen and paper. Each person who applies for a security clearance must fill out a 127-page form. Public online records like social media and state-by-state criminal records aren’t automatically pulled into the process. All of this leads to mountains of paperwork, and the inefficiency of the process puts lives at risk.

When Aaron Alexis killed 12 people at the Washington Navy Yard in 2013, he had held a security clearance since 2008. That clearance was granted even though he failed to report an arrest in 2004 for shooting out the tires of a car in a “blackout fueled by anger.” He then continued to maintain his clearance through several arrests involving violent incidents.

In 2015, the Office of Personnel Management database containing all the sensitive information found in the security clearance investigations for over 21 million people was hacked. All that information is now thought to be in the hands of the Chinese government, posing an unprecedented risk to national security.

The government has taken steps since these incidents to address some specific issues, but new and existing private sector technologies can do so much more to make this process more efficient and effective.

Through our work as a government-focused tech accelerator, Dcode has witnessed firsthand the capabilities that exist in today across the technology industry. And while government may be known for lagging the private sector in adopting modern technology, we work with many partners across the government working hard to leverage emerging technology solutions. Dcode has helped dozens of companies bring their products to market with the federal government in areas that could play a huge role in reducing the clearance backlog and the burden it causes.

Here are four areas that could leverage emerging technology and the companies that could do the work:

Facial Recognition and Document Validation

There are two obvious issues with ensuring someone is who they say they are (a critical piece of the security clearance process), (1) matching a picture ID (like a passport) with that individual in real-time and (2) ensuring the ID is not fraudulent. The more difficult question is how do you meet a certain standard at scale when in-person checks are too costly and time consuming. Onfido uses real-time facial recognition with a “liveness test.” That verification is coupled with a machine learning algorithm that identifies fraud, digital tampering, or knows if a document is lost or stolen. In 60 seconds, Onfido can provide that first step to anyone in a seamless interface without an individual present.

Natural Language Processing

Many of the federal government’s backlogs are because of humans evaluating completeness and accuracy of forms. This is massively inefficient at scale and technology is at the point where for simple tasks, like evaluating completeness, machines are as good if not better than humans. Skymind is able to tailor its open source neural-net to many different types of problems, including a task to ensure humans are evaluating complete documents and those documents match the identification provided by the applicant. This could save the government and vendors hundreds of hours of back and forth between applicants and the evaluators.

Handoffs Within and Between Agencies

The security clearance process involves at least three agencies each time a clearance is processed, if not more. When an information exchange is handled by people, there are waiting times, individuals who get busy, and information that slips through the cracks. Catalytic Software has built a machine learning bot called PushBot that could seamlessly handle all handoffs and even push information to applicants without a human involved. The bot learns and provides suggested improvements to the process as well base on the data it collects throughout.

Understanding Anomalies in Data

Each security clearance application contains important data on the possible contractors working in the federal government. Over time, this data has become a massive data set with more noise than signal. To make the best use of this data set, the government needs to apply modern data science techniques to it.

Unfortunately, the number of data scientists across government is limited. The government could leverage a company like DataRobot to put data science tools in the hands of analysts. DataRobot’s product allows for drag and drop data science and could allow for less sophisticated individuals to make the best use of the data set created by the processing of these clearances.

These are just four of the many areas where technology could support or replace entirely costly and inefficient human involvement in the process. Unfortunately, the government usually tries to solve these problems through more people or buying a new “platform” when parts of the problem could easily be solved through existing products currently on the market.

Using these products could mean the difference between wasting millions of dollars while the 700,000 person-backlog languishes, and national security threats slip through the cracks, and having more efficient system that saves taxpayer money and fixes the problem.

Related Content
Woman typing at computer

Stay in the Know

Subscribe now to receive our newsletters.