Prototype to Enhance Vetting for Security Threat Assessments is Key Goal of Rutgers Researchers

The CCICADA Research Group Stresses the Need for a Robust Human Partnership to Help Computers to Optimize Throughput of Applications

The United States Congress authorized and established the Transportation Security Administration (TSA) following the September 11, 2001, terrorist attacks, during which 19 individuals hijacked four commercial aircraft in a catastrophic act of terrorism. Since that time, TSA has strengthened transportation security, but persistent threats remain. The U.S. Government, private sector, and research universities are coordinating to strengthen national security and prevent future attacks on the nation’s transportation system. The Command, Control and Interoperability Center for Advanced Data Analysis (CCICADA) at Rutgers University aims to help in these endeavors.

Securing the nation’s complex transportation system, which includes surface, maritime, and aviation components, presents major challenges for TSA; state, local, and Tribal law enforcement; and other government agencies. TSA conducts security threat assessments (STAs) under federal law¹ to assess eligibility for access to secure transportation facilities or information, or membership in transportation security-related programs. These STAs are critical to improving the security of transportation systems and preventing insider threats.

On a continuing basis, TSA reviews the criminal history records of millions of individuals in vetted populations with access to transportation sectors. This large workload necessitates extensive investment in time and resources to complete manual application and adjudications. Automation of some of the manual processes would allow staff to perform more efficiently without compromising thorough and effective review of STAs.

The Record of Arrests and Prosecution, also known as a “rap sheet,” is a key component of the vetting review process for any criminal history record. The rap sheet is a listing of any criminal history associated with an individual. Adjudication of events found within this document is a  requirement for gaining employment or access to benefits within populations that TSA vets through STAs, such as TSA PreCheck® members, certain Aviation Worker employees, and Transportation Worker Identification Credential (TWIC®) cardholders.

The TWIC program provides an example of how TSA identifies potential threats to national and transportation security. The Maritime Transportation Security Act requires that workers seeking unescorted access to secure areas of the nation’s ports, vessels, and other maritime facilities must pass a STA to receive a TWIC.

TSA has implemented vetting processes and solutions that expedite and strengthen the screening process while adapting to new threats to the transportation system. For example, the implementation of recurrent vetting via the Federal Bureau of Investigation’s Rap Back service and the DHS Automated Biometric Identification System (IDENT) is key for  mitigating insider threats, which represent a grave security risk. In 2018, TSA began to issue TWIC – Next Generation (NexGen), a measure that provides an improved physical TWIC® card, designed to deter counterfeit and fraudulent credentials.

An applicant may be denied a TWIC, temporarily or permanently, for having criminal arrests or convictions involving any of the 27 offenses identified in Title 49 of the Code of Federal Regulations (CFR), section 1572.103. Also, an applicant who is wanted or under indictment is disqualified until the want or warrant is released or the indictment is dismissed. For example, persons convicted of felony assault, extortion, rape, or bribery, among other offenses, are disqualified from obtaining a TWIC for seven years following conviction and five years following release from incarceration. Persons convicted of murder, espionage, treason, terrorism, and other high crimes are permanently disqualified from obtaining a TWIC. The variation of naming and coding the 27 federal disqualifying offenses in states’ law enforcement agencies further complicates the task of identifying applicants disqualified on the basis of their criminal history.

Calling in the CCICADA Research Team

TSA collaborated with Professor Fred Roberts of the Rutgers-based CCICADA research center, leveraging its team to address these distinct challenges. CICCADA’s project drew upon a multidisciplinary mix of technical specialists and policy experts ranging from security professionals, law students, software developers, computer scientists, data analysts, and mathematicians. This group’s combined knowledge and utilization of resources produced a prototype that demonstrates possible enhancements to the STA process for individuals seeking access to transportation sector employment or benefits.

Launching the project with face-to-face discussions between the technical team and other subject matter experts boosted the chances of saving significant staff time—and vastly increased the efficiency of a key security agency.

One of the interesting partnerships that emerged joined the data analytics talent of CCICADA with the research skills of criminal justice lawyers and Rutgers student interns. With the assistance of Rutgers undergraduate and graduate students, the legal team reconciled the 27 disqualifying federal crimes with comparable statutes in states throughout the country. The team’s ongoing efforts played a vital role in  helping their technical colleagues craft a more efficient screening process for pre-empting serious security  threats. Elie Honig, Executive Director of the Rutgers Institute for Secure Communities (RISC), and Associate Director Ava Majlesi, who directs the University’s Intelligence Community Center for Academic Excellence (ICCAE) and Critical Intelligence Studies, believe the creation of this “unlikely” team directly led to their success.

Building a Better Machine (With Humans)

TSA and CCICADA’s team of experts embarked by setting measureable and specific goals for their joint endeavor:

• Construct an automated system that identifies and matches the 27 disqualifying crimes listed in the CFR with both state and federal statutes and criminal offense keywords.
• Create custom TSA technologies that recognize machine-readable rap sheets.
•  Optimize TSA adjudicators’ time so that they can focus on the most high-risk applications, rather than treating all applications as high-priority reviews.
• Maintain TSA’s final authority in revising or rejecting the machine-generated conclusions or recommendations for individual applicants.

Vladimir Menkov, Ph.D., project programmer and software architect at Rutgers University, constructed a system designed to screen records from 12 states whose criminal code definitions, abbreviations, data formats, and legal language differ from the way disqualifying offenses are presented in the federal regulations. These states were Alaska, California, Florida, Hawaii, Illinois, Louisiana, Maryland, New Jersey, New York, Ohio, Texas, and Washington State. The gathered data assisted the team in further developing an automated background check prototype named the Criminal History Enhanced Evaluation Tool for Adjudication (CHEETA).

This rules-based, artificial intelligence system uses factors, such as the severity of the crime, the sentence imposed, and the time since a sentence was served, to make a recommendation to a TSA adjudicator whether an applicant record includes a disqualifying crime.

In a recent interview, Paul Kantor, one of the principal investigators of the CCICADA project and professor emeritus at Rutgers, The State University of New Jersey, said: “One of the key features of CHEETA is that it’s rule-based. Every decision it makes is computed based on a rule that can be understood, summarized, and presented to the [human] decision maker, along with the recommended decision.”

CHEETA rapidly scans for key words and potential violations in large numbers of rap sheets to provide an automated adjudication assessment. This assessment is displayed in a preliminary “Overall Opinion” for each record:

• Records that indicate an applicant’s high chance of possessing criminal history information pertaining to one or more of the 27 disqualifying crimes are marked RED.
• Records that indicate an applicant possesses no apparent criminal history information pertaining to the 27 disqualifying crimes are marked GREEN.
• Records that do not clearly indicate an applicant’s chance of possessing criminal history information are rated RED or GREEN based upon available information, but are labeled with a low quality or confidence score.

All preliminary opinions are subject to further review by a human adjudicator. The adjudicator’s priorities will prompt review of RED files, additional review of low-confidence files before making a final determination, and timely approval of GREEN files. CHEETA provides automated decision factors that may be tailored to meet any level of risk mitigation strategy if elected to be employed by TSA.

“The most serious concern of the United States [still] is to not let the bad guys get through the screening process,” said Professor Kantor. “Flagging an application ‘Yellow,’ meaning an applicant might need further investigation, doesn’t help because it doesn’t tell you what to do. Red says, ‘YOU MUST LOOK AT THIS!’ and that’s obviously good.”

“With that in mind, it becomes really important to understand how accurate we are if we flag  something Green [applicant OK],” Kantor continued. “The aim of the project is to get Green to be so reliable that it requires nothing more than a quick check by an experienced analyst.”

CHEETA’s ultimate goal is to assess the feasibility and impact of adopting its rule-based system across TSA’s populations in all U.S. states and jurisdictions. The project also seeks to optimize adjudicators’ time and concentrate on each case’s potential threat level comparatively.

The background-check model developed by the Rutgers CCICADA research team could be adapted for use by other agencies or groups seeking reliable background checks.

Implications for the Future

To date, 50,000 historical applications from the 12 participating states have been processed through CHEETA as a training exercise. CHEETA’s judgments were compared with the judgments of human adjudicators. Results give the team optimism that the system has the long-term potential to reduce the number of applications requiring high-level scrutiny and the time required to review each case. Additional studies are planned to further analyze CHEETA’s accuracy.

If the CHEETA model proves successful, Kantor believes it might be adapted for use by other agencies or groups seeking reliable background checks:

“There are obviously many settings where people need to read and screen these rap sheets. One example is all the people who work with children. Another example is all the several million people who hold security clearances. There’s a big world out there of organizations that need to check whether there’s a criminal record and, if so, to screen it.”

The directors and students from the legal team share Kantor’s hope that some version of an accurate, efficient system for criminal history background checks could benefit both private and public sector employers. Elie Honig and Ava Majlesi of the Rutgers Institute for Secure Communities are “confident that we could someday replicate a similar process involving data analysis for other sectors, such as health care.”

Emelin Flores, a Rutgers undergraduate studying Information Technology and Critical  Intelligence, called the project a “game-changer” and sees her work reconciling the state and federal statutes as part of an exciting movement combining digital technology, security, and public policy. Dana Domenick, a Rutgers law student, agreed that the CHEETA model could “absolutely” impact a variety of industries. Karl Zenowich, now a graduate student in Security & Resilience (postattack strategies for physical and economic recovery), said development of systems that recognize the critical value of coordinating technology and human input are an “absolute necessity” if the U.S. is to keep pace with the fast-changing threat environment.

For now, a CHEETA-style system cannot be simply “dropped into” a different sector without adjusting to that sector’s unique challenges and security needs. That is why an enormous amount of work went into designing CHEETA’s specific logic and rules for the TSA project, and why it will incorporate a framework that security assessment experts can use to customize future rules. “Results are only as good as the data coming in,” Dr. Menkov said. “Discrepancy in the data, missing information, out-of-date information … are still important issues.” And some errors, he said, “just happen.”

TSA adjudicators remain the ultimate decision makers in assessing the nation’s transportation security threats when using the CHEETA model. State-of-the-art technology can help expedite the process, but even the fastest equipment may miss critical information and patterns outside of the system that may be picked up by astute adjudicators. Ultimately, the “human factor,” with its nuanced abilities and instincts, is still a crucial resource when making judgments.

The CCICADA-led research team has not yet recommended that CHEETA be used to screen current applications. They prefer at this stage to collect feedback from TSA adjudicators on how well the system works to process existing data. These human decision makers continue to provide insights into how well the prototype system helps them to reach good decisions.

“It is very much a machine and human ‘co-partnership’ relationship,” Professor Kantor said. “The adjudicators’ feedback is critical. Feedback will let every user record a change in any conclusion or opinion that CHEETA has reached and provide information on why that change is being made. Using extensive conversations with experts at TSA in a back-and-forth process, we find out what works. Frankly, I believe that will prove to be the successful model for increased automation of a great range of government tasks.”

Ultimately, the “human factor” with its nuanced abilities and instincts is still a crucial resource in making judgments. The machine should not have the last word.

The CCICADA team remains focused for now on perfecting the CHEETA algorithms. “Even if CHEETA’s accuracy proves to be high,” Dr. Menkov said, “there are still data problems to be solved in the future when data from participating states does not reflect the applicant’s complete record.” Continued improvements to vetting processes and capabilities, such as CHEETA, will help TSA and other government agencies bolster national security in the transportation sector and beyond.

¹See 49 CFR §§ 1540.205, 1542.209, 1544.228, 1544.229, 1544.230, 1572.5.

Download the pdf

Related Posts via Taxonomies

• CCICADA Organizes COE COVID-19 Supply Chain Meeting
• Cyber Attacks on Ports and Ships Could be Catastrophic, Symposium Speakers Say
• DHS Awards New Center to a Team Involving CCICADA
• Missed a Flight Due to Long Security Lines? TSA Official Unveils ‘Intelligent’ Solution at CCICADA Research Retreat
• CCICADA Addresses the Growing Threat of Maritime Cyber-Attacks
• Cyber Attacks Could Paralyze US Ports, Captain Moskoff Tells CCICADA
• New CINA-CCICADA Project: Early Warning about Criminal Supply Chain Disruption
• Rutgers Researchers Seek Root Causes, Root Cures, for Disruptions to the Technology Supply Chain
• Stadium Security Guide: Best Anti-Terrorism Practices for Large Venues
• Rutgers Researchers Making Sure It’s Safe to ‘Take Me Out to the Ballgame’