An Idiot’s Guide to Metadata Retention

Words By: Trilokesh Chanmugam

Metadata is a term as nebulous in meaning as it is ominous in implication.

If not completely one-sided, debate about data-retention often involves one party loudly conjecturing about state totalitarianism (nineteen-eighty-four!), and the other party making sensationalist claims about the threat of islamic terror or internet child pornography. The resulting discussion is one wrought with assumptions, misunderstandings, and slippery slope concerns.

If you’re late to the fray and just want to get yourself informed, you face an overwhelming mass of information, from which separating fact from opinion, and relevant from irrelevant, is immensely time consuming.

Never fear, Rotunda is here to help.

1) What is metadata?

“A set of data that describes and gives information about other data.”

The set of information as it appears on your phone bill is metadata; the duration of the call, the ‘to’ and ‘from’ phone numbers, and possibly the location details of the caller and receiver, but not the actual content of the call.

Seems straight forward, right?

Except that it’s much hazier than that, especially when it comes to metadata for online communications. Even the politicians responsible for getting the bill through parliament don’t understand the nuts and bolts of this.

Tony Abbott publicly stated that metadata includes “the sites you visit”, but his office later clarified that this was not the case. Web browsing history falls under the category of content, which government agencies require a warrant to access. Chat room discussions, the content of email exchanges and social media posts also fall into the ‘data’ category, safe from prying eyes. However, the IP addresses of computers from which messages are received or sent will be accessible, as will the names and duration of active online applications, the status of chat sites, the ‘to’ and ‘from’ addresses of sent e-mails, and any chat aliases used. (Find the full list of included data here.

Smartphones are very “chatty” devices; constantly relaying information between your pocket and the nearest base station, providing your location to the nearest few kilometres. Metadata is extremely revealing. You can’t find out what a person is talking about (at least not directly), but you can find out almost everything else.

2) Why is metadata so topical in Australia?

The “Telecommunications (Interception and Access) Act” of 1979 protected the privacy of Australians who use the telecommunications system. It prohibited the interception of private communications, and prohibited access to stored communications such as SMS, e-mail, and voice messages. There was one exception: law enforcement agencies could access the information, pursuant to a warrant. Telecommunication companies were not obligated to retain their client’s metadata, but often did so for billing purposes.

In October 2014, an amendment to this act was introduced, based upon the recommendations of the Parliamentary Joint Committee on Intelligence and Security (PJCIS). The amendment successfully passed both houses of parliament on March 26th 2015, and required that communications providers retain their client’s metadata for a period of two years. Access to this metadata is available to a large number of government agencies without warrant. 

That means that you might be placed under investigation based on vague suspicions, without ever even knowing about it.

Despite the controversial nature of this bill, it was passed with very little parliamentary opposition. Labour jumped on board quickly, only negotiating over a minor aspect of the bill which failed to provide protections for journalists. Scott Ludlam and a few independent senators were the only voices making any serious efforts to resist the passage of this bill through parliament.


Bi-partisan support for this legislation results from an unquestioned consensus that it will assist the government in thwarting terrorism and other serious crime. The bill received royal assent and stands to come into force at the beginning of 2017, but a vocal minority continues to resist the metadata retention scheme, asserting that the security benefits do not outweigh the consequential privacy shortfalls.

3) Who will have access to your metadata?

Under existing laws, access to stored metadata is regulated by the 1979 TIA Act, which permits “enforcement agencies” to authorise telecommunications carriers to disclose data where that information is reasonably necessary for the enforcement of the criminal law, a law imposing a pecuniary penalty, or the protection of the public revenue. This resulted in approximately 80 enforcement agencies being granted “stored communications warrants” in 2012-13.

Attorney-General George Brandis has submitted to reason in recognizing that the above definition of an “enforcement agency” is dangerously broad. As a result, in the final reading of the amendment, the list of agencies that can access stored metadata without warrant has been specifically limited to “criminal law-enforcement agencies”:

(1) Each of the following is a criminal law-enforcement agency :(a)   the Australian Federal Police;(b) a Police Force of a State;

(c) the Australian Commission for Law Enforcement Integrity;

(d) the ACC;

(e) the Australian Customs and Border Protection Service;

(ea) the Australian Securities and Investments Commission;

(eb) the Australian Competition and Consumer Commission;

(f) the Crime Commission;

(g) the Independent Commission Against Corruption;

(h) the Police Integrity Commission;

(i) the IBAC;

(j) the Crime and Corruption Commission of Queensland;

(k) the Corruption and Crime Commission;

(l) the Independent Commissioner Against Corruption;

(m) subject to subsection (7), an authority or body for which a declaration under subsection (3) is in force.


Eighty agencies accessed stored metadata between 2012 and 2013. These notoriously included the RSPCA, and various local councils who would have been hard-pressed to make a strong case that their right to private data was in the public interest. So it’s a relief that the government has decided to specify which agencies will be granted access in the future. However, there are a few concerning features of this amendment.

Firstly, you may notice the glaring absence of ASIO in the above list. That’s not because ASIO won’t have access to industry stored data, but because ASIO is governed by separate provisions which enable access for “purposes relevant to security”. ASIO has been the main agency pushing for mandatory retention of metadata, and we can be sure that its access to the enlarged pool of private data will not be governed by due process.

Secondly, subsection (m) references another subsection, which states:

(3) The Minister may, by legislative instrument, declare:(a) an authority or body to be a criminal law-enforcement agency; and(b) persons specified, or of a kind specified, in the declaration to be officers of the criminal law-enforcement agency for the purposes of this Act.



Essentially, the Minister can declare that any agency, government or otherwise, can be considered a “law enforcement agency” for all purposes related to collecting stored metadata. That’s a lot of power at the fingertips of Mr. George Brandis. Excluding the RSPCA and local councils from the privileged group of “criminal law-enforcement agencies” suddenly seems like a pretty symbolic gesture.

4) Can we avoid metadata retention?

Yes, we can avoid our data being collected, and with relative ease.

Greens Senator Scott Ludlam has suggested that there is a vast array of methods available to those who want to circumvent the metadata retention scheme. Online activity can be masked using Tor or VPN encryption, people can switch to overseas hosted service providers which are exempt from surveillance, or far more simply, would be offenders can avoid the data retention scheme by using public internet: Cafe’ hot-spots, public libraries, or even the free wi-fi available at parliament house.

Malcolm Turnbull, one of those people personally responsible for the formation of the bill, has also publicly drawn attention to how easy it is to avoid the scheme. ISP’s will not have to store records about third party communications services running over the top of their network, so even services like gmail and hotmail are safe. Skype, Whatsapp, Wickr, and other similar applications fall into this category as well.

Basically, we can escape the effects of the metadata retention scheme with only the most basic of precautions. However, this raises a very important question – if it’s so easy to avoid, how will it catch those who are trying not to be caught?

Apparently, the added protection against terrorism and other serious criminal activity is close to nil, as the actual culprits can take steps to avoid detection, whilst the average, unwitting internet user bears the brunt of the surveillance. What good is metadata retention if the targets of the scheme can avoid it with ease?

5)  What are the financial costs of the scheme?

Critics of the scheme have drawn attention to the cost of retaining metadata for the two year period. The total cost of implementing the scheme is estimated at up to $300 million per year, though it’s uncertain whether the cost will be carried by the end user of internet services, or the service providers. It’s been speculated that the amount will be covered by a “surveillance tax” payable on telecommunication bills, raising the cost of broadband and phone use for all consumers.

6)  Why should we be worried?

I have opinions on this issue, and I’ve tried up until now, with only limited success, to keep them under wraps. But it’s time to let loose.

This amendment is concerning because it mandates telecommunication companies to retain their clients metadata, which is information that they would normally discard. This vastly increases the pool of personal data about you and me that government agencies have access to. The details about which agencies have access to this data and the process by which these agencies get access are largely irrelevant, because the data exists, and it can be used without our consent and without our knowledge.

It would be erroneous to view this as a tension between private freedom and government control because at the moment, the government seems unable to comprehend the implications of what it is unleashing on the public, let alone harness that power for oppressive purposes.


Jon Davidson puts it aptly: “Perhaps we prefer to cling to the idea of an organised attack against the privacy of the Australian people, because it’s scarier to contemplate that nobody is in control at all.”

Furthermore, it’s not a matter of national security versus civil liberties because, as we’ve seen, those people who are a genuine threat to national security can avoid surveillance with ease.

Instead, and this remains a matter of conjecture, it seems inevitable that the mandatory data retention scheme will be misused; taken advantage of for primarily unintended purposes. Examples of potential misuse are detailed fairly thoroughly in this article, but there are also some non-hypothetical examples  of controversial metadata access. These include Queensland Police officials accessing the metadata of cadets last July to determine whether they were sleeping with one another or faking sick days, and the Bankstown council in NSW using metadata to fine litterbugs.

In talking to a friend about government surveillance, I argued that Australia would never actually be as bad as the Oceania described by George Orwell in “nineteen-eighty-four”. His response was bleakly humorous; “No! It’s already worse! Winston only had to deal with one telescreen, we’ve got countless cameras and devices monitoring our movements…”

The difference is that while Winston Smith had the oppressive spectre of INGSOC to contend with, Australia has a largely innocuous variety of liberal democracy. That doesn’t change the fact that should things turn bad, the infrastructure is already in place for mass invasions of privacy on a scale that George Orwell could not even comprehend.

Slippery slope arguments are a poor form of criticism in any situation, but it’s still worth considering that, as a result of the metadata retention scheme, Australia will need to be far more vigilant about holding the government to account. It’s imperative that the insidious decline of privacy rights are not permitted to creep further, and that the metadata retention scheme is only used for its initially intended purposes of preventing violent crime.