Ignorance and indifference: Delving deep into the Clinton e-mail saga

Hillary Clinton, former secretary of state and presumptive Democratic nominee for the presidency, is facing a massive backlash after an FBI investigation found her to have been “extremely careless” in the handling of classified information. The scandal surrounding her use of a private e-mail server has only grown since the Justice Department’s decision not to pursue criminal charges. Polls show that a majority of Americans believe she should have been indicted, and more recent polls place Clinton in a dead heat with the presumptive Republican nominee Donald Trump. Clinton led by a significant margin just weeks ago.

Regardless of the political games being played, the facts of Clinton’s use of a private e-mail server and the related potential exposure of Top Secret information—including the names of covert intelligence personnel overseas and at home—are worth knowing and nailing down. At the core, these details raise a much broader question surrounding how national secrets are kept and shared and how broken the information infrastructure of the United States government really is.

In order to have an intelligent conversation about Clinton’s e-mails, here is a technical analysis of the evidence as it has been presented (think of it like a print version of Congressional hearings, minus screaming, finger-pointing, and grandstanding). A clearer picture has started emerging based on the testimony given by FBI Director James Comey and the Inspectors General of the State Department and the Intelligence Community (OIG), plus a portion of the 30,000-plus e-mails released thus far through FOIA requests by the State Department and other agencies. That picture, based on our assessment, is not a very pretty one.

Plenty of blame to go around

The evidence reviewed by Ars, including a portion of the 30,000 e-mails sent or received by Clinton, other e-mails obtained by the conservative action group Judicial Watch, and the information cited in the State Department OIG report, appears to support Comey’s statement that Clinton lacked the “sophistication” to understand the impact of her own actions. The report and e-mails give the impression that Clinton simply did not know she was mishandling sensitive information because she did not recognize it to be classified, and she assumed what she was doing was within her purview as Secretary of State based on precedent set by previous occupants of her office.

Additionally, the OIG report and the e-mails themselves show not just carelessness by Clinton but a general indifference and even willful ignorance toward information security and document retention laws and rules at the State Department by both political appointees and career staffers. There was simply a culture where no one said “no” to the Secretary—no evidence exists that anyone ever directly contradicted her view or told Clinton not to use her private e-mail for reasons other than getting past State’s e-mail security filters. (The only documented time someone said “no” to her was when she asked for a Blackberry like President Barack Obama’s, and the NSA refused her.) Clinton’s appointed staff, career professionals at State, and others failed to tell her what she was doing was wrong. In some cases, staffers actively told IT people at State who did question Clinton’s private e-mail use to shut up.

This kind of culture absolutely pre-dates Clinton’s tenure at State. However, it failed during her watch on an entirely new level.

There are mitigating circumstances that clearly influenced investigators’ decision not to pursue a case against Clinton. First, the vast majority of Clinton’s more than 30,000 emails during her four years at State were unclassified. At worst, these contained sensitive-but-unclassified (SBU) content that would normally be restricted to State’s internal e-mail system. Clinton could have been granted permission to get and send this information under State Department regulations at the time. The small amount of e-mail traffic that was later to be determined to be of a classified nature was almost entirely not marked as such.

For the most part, such messages were sent to Clinton by people within the State Department from State’s unclassified mail system. There are also significant questions about the classification applied to many of the e-mails after the fact—some of the messages redacted by the State Department and marked as having contained “Secret” information were from sources outside government. Clinton would have reasonably assumed communications with such individuals were unclassified.

Still, a small fraction of the messages sent or received by Clinton containing classified information during her four years at State—especially eight message threads containing extremely sensitive information classified above Top Secret by reviewers—were major breaches of both State Department regulations and federal laws regarding handling of classified data. Some of the data was even beyond the security clearance levels of Intelligence Community OIG investigators.

These messages should have never been on State’s internal unclassified e-mail system, let alone on a server sitting in the basement of the Clintons’ home in New York. And by sending that information in the clear over the Internet between the State Department’s e-mail gateway and Clinton’s home mail server, vital national security secrets were exposed to potential espionage. Additionally, the information was potentially exposed through attacks on both Clinton’s and the State Department’s mail systems.

It’s true that highly classified information may have similarly been passed over the Internet by previous Secretaries of State, their staff, and political appointees and career executives in the Foreign Service at State. And it’s true that such breaches of protocol have likely happened before, during, and after Clinton’s time at State. None of that changes the facts—it only magnifies how poorly the United States’ diplomatic service handles information systems and security. And due to inadequate resources at State and outright resistance from the NSA to provide a solution, the State Department and the National Security Agency failed to provide the kind of support for Clinton early on that would have prevented such a situation from continuing.

Stretching the precedent

As detailed in the State Department’s Inspector General report, Hillary Clinton is not the first Secretary of State to use a private e-mail address for work. For instance, former Secretary Colin Powell brought a private phone line into the State Department so that he could send e-mail through his AOL account. The report says:

Secretary Powell has publicly stated that, during his tenure as Secretary, he installed a laptop computer on a ‘private line’ and that he used the laptop to send e-mails via his personal e-mail account to his ‘principal assistants, individual ambassadors, and foreign minister colleagues.’ Secretary Powell’s representative advised the Department in 2015 that he did not retain those e-mails or make printed copies. Secretary Powell has also publicly stated that he generally sent e-mails to his staff via their State Department e-mail addresses but that he personally does not know whether the Department captured those e-mails on its servers.

While Powell was at State, the Office of the Secretary/Executive Secretariat (S/ES) issued a memo reminding departing officials to forward printed copies of all of their correspondence—including e-mail—to the State Department for retention. Powell failed to do so. At a minimum, Powell’s oversight was possibly a violation of federal records retention law at the time. But there’s no way to know whether there was classified information in the e-mails, because they have not been recovered.

Powell’s e-mail usage was a workaround for a very fundamental problem at State: prior to his arrival, there was no effective enterprise e-mail system with Internet access. Additionally, Powell would not have been permitted to access personal e-mail from a State Department owned computer, so he needed a private line to access his account. Powell explained to State Department OIG investigators that upon his arrival at State, the official e-mail system only connected to others at State. “He therefore requested that information technology staff install the private line so that he could use his personal account to communicate with people outside the Department,” State OIG investigators reported. (Ironically, Scott Gration, an ambassador to Kenya, was forced to resign for doing essentially the same thing in 2012, while Clinton was using her own personal server.)

By the time Condoleezza Rice became Secretary of State, the OpenNet system was in place at the department to provide Internet mail access. Rice’s staff used State-issued Blackberry devices with access to State’s secure ClassNet e-mail servers, and Rice used only her State.gov e-mail. But Rice herself never had a State.gov e-mail address, and she “did not use either personal or Department e-mail accounts for official business,” Margaret P. Grafeld, Deputy Assistant Secretary for Global Information Systems at the State Department told State OIG investigators.

That aligns with the practice of most senior government officials over the past decade. E-mail was correspondence typically handled by staff, and it was generally only handled and marked up by senior executives in printed form, according to numerous people within government who spoke to Ars.

Clinton did most of her work in this way—having staff send e-mails or printed memoranda, then managing through face-to-face meetings and phone calls. But she was not an e-mail avoider. Clinton had developed an affinity for her Blackberry and e-mail during her term-and-a-half in the US Senate and on the campaign trail during the 2008 presidential primaries. At some point following the campaign as she was losing access to her Senate e-mail, Clinton became frustrated with technical issues related to her AT&T Blackberry e-mail and device, according to aide Huma Abedin. Clinton had a new e-mail account configured for her on a personal mail server running in the Clinton home at Chappaqua, New York. The server was set up for former President Bill Clinton by his own staff.

On January 19, 2009—the day before President Obama’s inauguration—someone working for the Clintons registered ClintonEmail.com on her behalf and configured the domain on the Exchange mail server. Clinton gave Abedin and others accounts. On her first day at State, Clinton became the first Secretary of State (and likely the last) to use a private e-mail server. She would routinely use it for official business communications with her core staff at State as well as for personal communications with friends and relatives. Over the next four years, Clinton sent and received over 30,000 e-mails.

Based on the e-mails released under FOIA request by the State Department and State’s OIG report, no one in State’s legal department ever authorized Clinton to use the private server. However, no one told her not to. “A March 17, 2009 memorandum prepared by S/ES-IRM staff regarding communications equipment in the Secretary’s New York residence identified a server located in the basement,” the OIG report notes. So from the very beginning, State officials were aware Clinton was using the server. According to State OIG testimony to the House Oversight Committee, questions were raised about Clinton’s e-mail by State IT team members. But State OIG was then told by the head of the Office of Information Resources Management for the Executive Secretariat (S/ES-IRM) that the system had been cleared by State attorneys and to never bring it up again.

Everyone’s favorite meme definitely took on a different meaning recently. Credit: Kevin Lamarque / Getty Images

Left to her own devices

The first roadblock to Clinton’s use of personal e-mail came up almost immediately upon her arrival at the State Department. Under State Department regulations adopted in 2009, “personal digital assistants” not owned by the State Department could only be turned on in areas designated as “strictly unclassified” (things like cafeterias and bathrooms). They were not permitted to be connected to internal State Department networks except via State’s Global OpenNet Web gateway to its intranet or some other department-approved remote access program. (The rule was changed after Clinton’s departure to allow personal mobile devices in work areas as long as they are at least 10 feet from equipment processing sensitive data.)

But the rules—and physics—of using a Blackberry in Clinton’s office were even more demanding. “Mahogany Row,” the State Department’s executive suite, is a Sensitive Compartmented Information Facility (SCIF). SCIFs are locked down areas where no wireless devices are allowed to operate, and in fact usually all radio frequency signals are blocked out by electromagnetic shielding, too. Immediately, Clinton started letting her displeasure show about not being able to use her Blackberry in a “SCIF’d space.” Clinton’s staff had to put up a satellite office for her outside of Mahogany Row so she could use her Blackberry periodically during the day.

Eric Boswell, then Assistant Secretary of State for Diplomatic Security, turned to his coordinator for security infrastructure, Donald Reid, to find an answer. Reid asked the NSA for support, inquiring about the measures taken to accommodate another Blackberry addict—President Obama. Each time the office asked the question ‘What was the solution for POTUS?,” they “were politely asked to shut up and color,” Reid reported in an e-mail on February 13. Instead, the NSA suggested that Clinton use their approved solution—the Secret-level SME-PED secure PDA. “The current state of the art is not too user friendly, has no infrastructure at State, and is very expensive,” Reid noted.

Clinton put her chief of staff, Cheryl Mills, on the case. In a meeting with representatives of the State Department’s Diplomatic Security office, members of the State IT team, and the NSA, Mills said that Clinton wanted all of her staff to be able to use Blackberry devices in the SCIF space and beyond just like members of Rice’s staff had done. “Ms. Mills described the requirement as chiefly driven by Secretary Clinton, who does not use standard computer equipment but relies exclusively on her Blackberry for e-mailing and remaining in contact on her schedule,” the NSA’s senior liaison to the State Department reported in an e-mail dated February 17, 2009. “Ideally, all members of her suite would be allowed to use Blackberries for communication in the SCIF.”

But the NSA would not provide Clinton’s staff with the special version of the Blackberry that had been created for President Obama because of the support costs involved. Mills was briefed on the vulnerabilities of the Blackberry and given some options for allowing its use in the SCIF areas—but those “mitigations” essentially disabled receiving of e-mail and phone calls. A technical solution was in the works, but it was years away from being ready. Ironically, at about the same time that the NSA’s security organization was denying Clinton a secure device, their colleagues on the signals intelligence side were allegedly spying on German Chancellor Angela Merkel’s personal cell phone.

As a result, Clinton continued to use her personal Blackberry outside the SCIF. There was no further discussion about her using an official device until November 13, 2010. At that point, deputy chief of staff Huma Abedin asked Clinton to consider either using a State.gov e-mail and Blackberry to communicate with internal staff or to get her private e-mail listed in the State directory to make sure her e-mails were not blocked by spam filters. Clinton responded, “Let’s get [a] separate address or device but I don’t want any risk of the personal [e-mail] being accessible.” However, Clinton and her staff never acted on the suggestion. Her mail server started working again and the issue was soon forgotten.

State of confusion

Clinton was not the only high-level official at State using a personal e-mail address during her tenure. State OIG investigators “reviewed an S/ES-IRM report prepared in 2010 showing that more than 9,200 e-mails were sent within one week from S/ES servers to 16 Web-based e-mail domains, including gmail.com, hotmail.com, and att.net,” the State report noted. “S/ES-IRM told OIG that it no longer has access to the tool used to generate this particular report. In another instance, in a June 3, 2011 e-mail message to Secretary Clinton with the subject line ‘Google email hacking and woeful state of civilian technology,’ a former Director of Policy Planning wrote: ‘State’s technology is so antiquated that NO ONE uses a State-issued laptop and even high officials routinely end up using their home e-mail accounts to be able to get their work done quickly and effectively.’”

This is, and was, in direct violation of State Department regulations. “Since 2002, Department employees have been prohibited from auto-forwarding their e-mail to a personal e-mail address ‘to preclude inadvertent transmission of SBU [sensitive but unclassified] e-mail on the Internet’,” the State OIG report noted. A State Department manual warns employees that ” transmissions from the Department’s OpenNet to and from non-U.S. Government Internet addresses, and other .gov or .mil addresses, unless specifically directed through an approved secure means, traverse the Internet unencrypted.”

State regulations allowed for “employees with a valid business need” to send sensitive but unclassified information unencrypted over the Internet “so long as they carefully consider that unencrypted e-mails can pass through foreign and domestic controlled ISPs, placing the confidentiality and integrity of the information at risk.” But State’s policy on “remote processing” of even unclassified information has placed restrictions on the use of personal or non-Department-owned systems within the State Department, or even State-owned hardware outside of departmental facilities (such as at home or in a hotel room) since 2008. The protocol requires “particular care and judgment” and that there are “appropriate administrative, technical and physical safeguards.”

There is no evidence to suggest that any of these measures were taken to protect Clinton’s Blackberry devices or her server. Bryan Pagliano served as a politically appointed State Department IT specialist (a truly unique arrangement), and he managed Clinton’s server as a sort of side-duty, In this regard, the server was in a very tangential way under State Department management. The server was also managed by Justin Cooper, a former White House aide to President Bill Clinton and an advisor to the former president’s Clinton Foundation and Clinton Global Initiative. But the system was certainly never certified to be in compliance with State Department regulations or Federal Information Security Management Act (FISMA) guidelines.

Like other high-ranking officials in the Obama administration, Clinton’s e-mail was a target for attack. As Ars reported previously, Justin Cooper informed Abedin on January 9, 2011 that he had shut down the server at the Clinton’s home because “someone was trying to hack us and while they did not get in, I didn’t want to let them have the chance to.” The same day, he e-mailed Abedin again, saying, “We were attacked again so I shut [the server] down for a few min.” The next day, Abedin told other members of Clinton’s staff not to e-mail Clinton “anything sensitive,” saying she could “explain more in person.” The incident was never reported to the State Department’s IT security team.

On May 13, 2011, the State Department OIG reported:

Two of Secretary Clinton’s immediate staff discussed via e-mail the Secretary’s concern that someone was “hacking into her e-mail” after she received an e-mail with a suspicious link. Several hours later, Secretary Clinton received an e-mail from the personal account of [then-Under Secretary of State for Political Affairs William Joseph Burns] that also had a link to a suspect website. The next morning, Secretary Clinton replied to the e-mail with the following message to the Under Secretary: “Is this really from you? I was worried about opening it!”

Again, the incident was not reported by Clinton, by her staff, or by Burns—all in violation of State Department regulations.

(In)security

Even with all of these issues, Clinton’s use of a private server would likely have been defensible if her e-mails had been backed up to a State Department archive regularly for review and if the messages she received contained only information appropriate for the State Department OpenNet e-mail system. The fact that she failed to provide her e-mails for retention and violated other regulations were essentially administrative matters—matters she would have been responsible as Secretary of State for adjudicating. The potentially criminal part of Clinton’s e-mail use came from classified information being sent and received by her e-mail account.

There is ample reason to believe that in the majority of cases, Clinton did not understand the information should not have been in unclassified e-mail. Of the more than 31,000 e-mails examined by State, Intelligence Community, and FBI investigators, a very small fraction—110 messages in 52 e-mail chains—contained classified information. The majority of that was sent to Clinton by her staff or other Foreign Service officials from within State using OpenNet’s Internet mail gateway, and classification markings were not used properly on any of those e-mails.

Only eight of those messages contained information carrying classification markings—none had headers indicating they contained classified data, but there were paragraph “content markers” indicating classified information was present. According to information provided to Congressman Elijah Cummings (D-Maryland) by the State Department that was cited during Comey’s testimony before the House Oversight Committee, those content markers were only “preliminary marks” that should have been deleted.

Among the 52 e-mail message threads found to contain information later determined to be classified, seven were found to “discuss matters that were classified at the Top Secret/Special Access Program level,” Comey said in his testimony. Intelligence Community Inspector General I. Charles McCullough III and State Department Inspector General Steve Linick told members of the House Government Oversight committee that they could not respond to questions about the nature of the contents of those threads publicly because even discussing the type of information in them would require a classified, closed briefing.

Among the remainder of the 52, much of the content was later classified as Secret—“information, the unauthorized disclosure of which reasonably could be expected to cause serious damage to the national security that the original classification authority is able to identify or describe”—based on Department of State classification guidelines. There were e-mails sent to Clinton that contained names of CIA employees, including members of former CIA Director General David Petraeus’ staff. The messages also included background on “protocols” being discussed between State and CIA forwarded by Clinton’s chief of staff Cheryl Mills. A backgrounder e-mail for a Clinton trip to Malta, labeled “Time Sensitive and Confidential” in the subject line, included information about overseas CIA and Defense personnel. Another e-mail from deputy chief Jacob Sullivan provided background for a call with Anders Fogh Rasmussen, then secretary-general of NATO.

Some of the e-mails that were later classified by the State Department came from outside the State Department e-mail system. On September 3, 2012, Ambassador Dennis Ross—who left the State Department in 2009 to join President Obama’s National Security Council staff as an adviser on the Mideast and South Asia—sent Clinton an e-mail from his iPhone referencing a discussion with Clinton staffer Jacob Sullivan “about some of the folks you will be meeting.” The State Department redacted the e-mail, classifying it as Secret because of “foreign government information” and “foreign relations activities.”

There was also some material that came completely from outside the government that got retroactively classified as Secret—information that Clinton and her aides could have reasonably assumed to be unclassified. For example, an e-mail thread forwarded to Clinton by Mills including attorney Martin Edelman (a real estate lawyer and a trustee of the Center for Strategic and International Studies) and Michael Kandarakis (an executive in a real estate equity firm working in Japan at the time) contained information from a third party about radiation leakage from the Fukushima nuclear power plant. It was classified after review by the State Department as Secret for its contents about “Intelligence Activities, Sources, Methods, and Cryptology.” The intelligence source, “M. Murata,” was described by Kandarakis as “a guy we know here in Japan who does background searches for us.”

It can’t happen again, can it?

The full extent of potentially classified information that was sent or received by Clinton will likely never be determined. There were many more e-mails on Clinton’s server that were judged by Clinton’s legal team (based on a search of headers and subject lines) not to be work-related, and therefore these were not forwarded to the State Department. Of those, a few thousand were recovered by the FBI from the physical drives of the decommissioned e-mail server from Clinton’s residence, but there were many more that could not be recovered to be checked for classified data. Those e-mails were destroyed “beyond the ability to be recovered by forensic tools,” Comey told Congress. Attorneys took that action, though Comey said Clinton had not instructed them to do so.

As far as Clinton is concerned, if she is elected president, she certainly won’t be using her own technology day-to-day. And if she isn’t elected, she’ll likely never have access to confidential information again—nor, potentially, will her former staff, who now face administrative reviews by the State Department.

Future officials won’t have nearly as much latitude as Clinton did with her e-mail, either. This year, new rules created by amendments to the Federal Records Act made in 2014 come into effect, requiring all government e-mail to be managed electronically (instead of being collected in physical printouts). E-mails of key personnel must now be retained under what is referred to as the “Capstone” approach to records preservation. “Capstone” officials—top agency executives—are required to have their e-mail directly managed by their agencies. The law now also prohibits federal employees from using a “non-official or personal account unless they copy an official electronic messaging account during the creation, receipt, or transmission of the record; or forward a complete copy of the record to an official electronic messaging account not later than 20 days after creation, receipt, or transmission of the record.”

Current Secretary of State John Kerry briefly used a personal e-mail account to do State Department business during his transition period, and he has periodically used personal e-mail to respond to work-related messages sent by others. But according to State Department OIG investigators, Kerry’s e-mails are being retained in accordance with the Capstone approach. Plus, Kerry has been an early adopter of a secure mobile device developed by the Defense Information Systems Agency. Called the Defense Mobile Classified Capability-Secret (DMCC-S), it’s an NSA-approved Android device based on the Samsung Galaxy S4.

A new phone won’t fix the bigger problems, however. The State Department’s e-mail system and network have been repeatedly hacked in the last four years, including intrusions by attackers from Russia, China, and Iran. “Longstanding, systemic weaknesses related to electronic records and communications have existed within the Office of the Secretary that go well beyond the tenure of any one Secretary of State,” the State Department OIG report concluded. “The Department generally and the Office of the Secretary in particular have been slow to recognize and to manage effectively the legal requirements and cybersecurity risks associated with electronic data communications, particularly as those risks pertain to its most senior leadership.”