Our nation is drowning in data. At any given time, federal agencies use more electronic storage units than could fill every NFL stadium from Oakland to Foxboro. At last count, the US government owns or leases at least 2100 data centers, and spends about half of its multi-billion dollar IT budget on digital storage. The United States Census Bureau alone maintains about 2560 terabytes of information — more data than is contained in all the academic libraries in America, and the equivalent of about 50 million four-door filing cabinets of text documents. In addition to the federal deluge, tens of thousands of municipal and state facilities maintain data ranging from driver's-license pics to administrative e-mails — or at least they're required to.
"Going digital" was supposed to be an environmentally conscious way for governments to cut costs while improving efficiency. But it hasn't quite worked out that way yet. Storage capacity is increasing, but the volume of data is also increasing, perhaps just as quickly. Over the next decade, the world will produce the informational equivalent of nearly 100 million Libraries of Congress per year, according to Cisco's Internet Business Solutions Group.
Most of what the government stores are public records, which means that by law they must be made available to anyone who requests them. But while there are ambitious efforts underway to improve storage methods, the sheer bulk of information is alarming.
These quandaries are turning into an increasingly expensive problem, and one that now has its own advisory bureau: the Office of Government Information Services (OGIS) was opened in 2009, solely to serve as an intermediary service between requesters of information and agency gatekeepers. Furthermore, Barack Obama was the first president to appoint a chief information officer, the operational-IT wunderkind Vivek Kundra, who has made it a priority to streamline the government's inefficient, energy-wasteful information infrastructure under a newly enacted Federal Data Center Consolidation Initiative. Kundra claims the US has already reduced its overall IT budget by $3 billion, while increasing functionality in some agencies by more than four times. Still, dire problems persist.
Bogged down
Public data remains, by and large, a disorganized mess. A recent report by the Citizens for Responsibility and Ethics in Washington (CREW) found that despite the administration's policy adjustments, Freedom of Information Act requests are still not being processed much differently than they used to be.
Indeed, from Washington DC to small-town America, governments have quietly begun to acknowledge that the information age has gotten them in way over their heads. Why?
There is too much data. Digital storage is not a natural resource. The amount of information that government agencies may be required to keep — from tweets and e-mails to tax histories — is growing faster than the capacity for storage.
There's not enough manpower to manage all this data. The Obama administration hopes to make more information freely available online. But in the meantime, the old method of requesting data from the government — filing a FOIA request — is bogged down due to an insufficient workforce and long request backlogs.
Private companies are storing public data. This trend in outsourcing, largely the result of too much data and too little manpower, is a potential threat to both access and security, as resources that belong to the people are entrusted to outside vendors, raising new privacy concerns.
