The NSA's non-invasive ThinThread link analysis system

[tongodeon]

This article presents an interesting solution to the wiretapping problem that existed in the late 1990s, code named "ThinThread". The system "identified U.S. phone numbers and other communications data and encrypted them to ensure caller privacy." Presumably this means that every phone number's MD5 checksum would be stored in the database. They could search the database to see how many calls '6f174b684f102dc8a5cae267c2480351' made to 'b9ef938cf09c9ee77868dad8b4f72c37', but without a court order you couldn't determine that '6f174b684f102dc8a5cae267c2480351' was the md5 checksum of my phone number. Using hashed phone numbers you could discover that '6f' made a lot of phone calls to Kabul, and on that basis get a court order to reveal the identity not only of '6f' but 6f's associate 'b9'. At the same time an official couldn't abuse the database by scanning it to see who called certain reporters, because they couldn't get a court order to reveal the hash of the phone number of a reporter because that reporter has not committed a crime. ThinThread let us identify our nation's enemies while protecting our officials' personal enemies. A perfectly good system was already in place in the late 1990s, and that the NSA intentionally abandoned it for an invasive, abuse-prone system which has been subsequently abused.

(The system I describe using MD5 isn't foolproof, because you could still hash the phone number 8005551212 to 'b83687c03d25d120c069fe3750ca3697' and search the database for that hash. Presumably the NSA's system encrypts each number to a unique hash value using a secret key rather than a well-known hash function. MD5 is used for illustrative purposes only.)