Wired reports that the Senate could enable, as part of it’s National Intelligence Reform Act, work on a system “that would let government counter-terrorist investigators instantly query a massive system of interconnected commercial and government databases that hold billions of records on Americans.”

The proposed network is based on the Markle Foundation Task Force’s December 2003 report, which envisioned a system that would allow FBI and CIA agents, as well as police officers and some companies, to quickly search intelligence, criminal and commercial databases. The proposal is so radical, the bill allocates $50 million just to fund the system’s specifications and privacy policies.

In contrast to the PR battle surrounding a similar previous effort — DARPA’s Terrorism Information Awareness project — privacy and civil liberties protections are being touted prominently in advance. CMU Distinguished Professor of Computer Science Dave Farber, a member of the Markle Task Force, has posted an open letter (which he authored, along with Esther Dyson and Tara Lemmey) on his influential Interesting People e-mail list endorsing the proposed system provided the recommendations of the Task Force were implemented (“as looks likely”).

During the course of the debate in Congress over the implementation of the 9/11 Commission recommendations, valid questions have been raised over civil liberty concerns and role of such an information sharing network. We grappled with these same questions as we worked through our recommendations for the Task Force. We also learned important lessons from the problems of other efforts like the Total Information Awareness program (TIA) and MATRIX, both of which have raised serious privacy concerns. We eventually determined that you can achieve a balance between security and privacy if you ensure that strong guidelines, transparency, accountability and oversight are built into the network from the start.
In addition to the approach of building policy into the design of the network, the Task Force also designed the network not as a centralized database, but as a set of pointers and directories that allow only authorized users to gain access to information. The system also calls for regular and robust internal audits of how information is collected and stored and used. Privacy technologies such as anonymization, permission controls, and audit trails are built into the design of the network to prevent abuse. In addition, the Task Force also calls for a phased implementation to allow for appropriate public comment and a strong civil liberties board to oversee the system and ensure that privacy
The SHARE network capability, if implemented properly, would give us the ability to overcome the systematic barriers to information sharing that so seriously constrained our intelligence agencies prior to the 9/11 terrorist attacks, and that unfortunately still exist today. It would also provide us with the best opportunity not only to balance security and privacy, but to enhance them both as well.

CRA has argued in the past of the need to move forward with this sort of research and has faulted Congress for taking a heavy-handed approach in prohibiting similar work. Perhaps this new approach will allow some real progress in developing the technologies valuable in the war on terror while at the same time enabling the critical research needed to ensure that privacy and civil liberties concerns are met.