Despite cyber-prowess and a staggering and secret budget, US intelligence agencies appeared unable to halt a deep and sustained invasion of sensitive federal information.  In May, an attack hit the IT system of the Office of Personnel Management (OPM) and its data stored at the Department of the Interior’s data center.

Information for about 4 million federal employees was likely copied by the hackers.  It appears the security clearance data may have been the high value target, since OPM holds the data for about 90 percent of security clearances.  The latest attack’s signature is similar to earlier Chinese Army attacks on technical industries, on health care databases (Anthem and Premera Blue Cross) and a 2014 attack on OPM.

The OPM intrusion should have been blocked by security arrangements at the OPM and Department of Interior data complex.  An intrusion monitoring system called EINSTEIN evidently failed to detect the attack quickly enough, but we cannot blame software — the responsibility for securing personnel data lies with the data’s owners.

Because the attack on OPM has roots in the Chinese government, it is precisely what we expect NSA to detect and thwart.  Trading stern language is not enough.  In response to Chinese Army invasions in high tech US industries, the US Department of Justice issued a subpoena for five Chinese military hackers.  No one is surprised that China ignored that limp gesture.

In 2012, the Justice Department allowed NSA “to monitor only addresses and cybersignatures — patterns associated with computer intrusions — that it could tie to foreign governments.” However, the NSA hoped to also get more permission to target other hackers not necessarily linked to foreign states.  Indeed only narcissist criminals would advertise their true identity.  It makes sense to chase those who commit criminal acts and who are suspected of association with an overseas power.  In some countries, the distinction between criminal and government official is very blurred indeed.

Timely new revelations about NSA surveillance assets suggest that NSA theoretically could have spotted both the malware headed toward OPM and the data copied from OPM’s database on its way to the perpetrators.  But “could have” is a stretch.

NSA surveillance assets include an intrusion detection system (IDS) that sits on an internet backbone at “chokepoints operated by U.S. providers through which international communications enter and leave the United States.”  For catching suspicious source or destination addresses, NSA would need an advanced version of “deep packet inspection” (DPI) that can compare the packet’s source and destination those against a database of known suspicious addresses.  For catching inbound malware, the DPI would also need to probe the packet’s content for snippets of computer code that match a database of known malicious code – similar to what Norton Security does when it scans your hard drive.  Of course, if the packet content is encrypted, the technical challenge becomes nearly impossible.  The extremely high volume of international Internet traffic suggests an outrageous scale of computing power would be needed to probe every packet all of the time.

Despite those challenges, IDS on a backbone is possible under special instances: Level 3, an Internet backbone operator cut off data from reaching a group of servers in China that it believed was involved in a hacking attack in the US.

In France, public opinion in the wake of Charlie Hebdo attacks and the technical track record of NSA have led to passage of surveillance authority similar to that available to NSA.  This seems a major reversal of posture since France had been a loud critic of NSA’s treatment of individual’s privacy.  Evidently, the slaughter of citizens by terrorists changes the acceptable balance between privacy and security.

While some may disagree, it is clear that aggressive pursuit by the NSA seems necessary to protect government assets and Americans personally identifiable information.  Some cynics suggest that NSA’s failure to detect the OPM breach was a tactic to collect information on millions of employee victims – making them fair game for future investigation.  The NSA safeguard that Americans really need is a trustworthy monitor who reports to the public, not just to US security agencies.