Creating Cybersecurity Training Exercises with CERT

December 09, 2009

Throughout the first day of competition, Poland and Australia were jockeying for the lead, but at the end of the two-day challenge, it was Australia in first place among the 29 competing teams from 20 countries. No, it wasn't the Olympics or World Cup; the five-person Australia 1 team took first place in the Tactical Response and Analysis Challenge (TRAC) conducted by the SEI's CERT Program as part of the weeklong International Cyber Defense Workshop (ICDW), which concluded November 13.

The ICDW is sponsored by the Office of the Assistant Secretary of Defense for Networks and Information Integration (NII) and hosted by the University of Nebraska Omaha. CERT is one of several organizations providing training as part of the virtual workshop. CERT's TRAC is the only portion with competitive scoring.  The top five finishers in the TRAC event were Australia, Poland, the Republic of Korea, NATO, and a combined (but geographically separated) team from Germany and France.

TRAC tested the teams' responses to cyber-attack scenarios, or war games for cyber-security, said Chris May, technical manager of the CERT Workforce Development team. May draws upon his experience as an Air Force captain to explain: "Going to a training course doesn't mean you can actually do something in the fog of war; you need experience with situations that test your abilities to do live problem solving. That's why the military uses live-fire exercises that embody the 'train as you fight' goal. You can have the best bombing or shooting range in the world, but the range is only as good as the training scenario that goes with it."

"The TRAC exercises were designed around realistic scenarios to serve as experiential training," said Jeff Mattson, a member of the CERT Workforce Development team. "Participants generally walk away with more benefit because they are hands on. Participants approach the activities in a free-play environment where they have to bring their problem solving skills to bear on the situation. It's more than just doing something they know how to do. They’re in a new environment where they have to apply principles they know in that new environment."

The teams are guided through the sessions by quizzes that direct the players toward the information that they must find. "We don't tell them how to find it," said Mattson, "and we've actually seen a lot of different approaches to getting to the same answer."

In the TRAC scenario, internet-based attackers find and exploit vulnerable web-application and database servers of a fictional shipping company and wreak havoc throughout its network. "This is all accomplished in real time while participants are in the environment," said Mattson. "It's hacked while they're watching. The intruders are able to get a root kit, download it, and then they start with some botnet activity trying to infect other machines in that network. So there's a lot of network activity that the participants can find—there's a lot of log file artifacts they can find—so it's a robust exercise."

The first day of the two-day TRAC exercise concentrated on detection, monitoring, and mitigation activities, and the second day focused on computer forensic analysis. The scenarios were created by May and Mattson, CERT staff members Rob Floodeen and Josh Hammerstein, and graduate students in information security from the Carnegie Mellon University Heinz College and Information Networking Institute.

Excerpt from article "CERT Tactical Response and Analysis Challenge Tests Cybersecurity Skills" from CERT.

Pictured above: Jeff Mattson (left) and Chris May