Cuban government support of the weekly "packages"

Cuban blogger Isbel Diaz Torres has written a two part post on the information "packages" that are distributed each week in Cuba.

In part one, he lists 12 factors that lead him to believe that the Cuban government may be behind these weekly packages. I will not list the 12 factors here -- you can see them in his post -- but he makes a compelling case that the package service could not run as smoothly as it does without government participation or approval.

In part 2 he discusses the motive the government might have for supporting the service. Since the weekly packages include current television program episodes, movies, magazines, etc., they supply weekly entertainment, eliminating what may be the key factor behind people's desire for Internet access. He also lists other, very limited, services that the government argues substitute for Internet services. He speculates that the government wants to be able to claim that Internet access is not needed because Cubans have everything they want without it.

One cannot know whether Torres' hypothesis is true. The weekly packages are surrounded in mystery. I have asked many people who distributes them and how they get the material into Cuba and no one seems to know.

If the government is behind the weekly packages, I would suggest a simpler motive than trying to rationalize a lack of Internet access -- money. The packages are a going business with an established curating and distribution organization. Someone is making money and it might be the government or a friend of the government.

The Cuban government says information technology is now a priority, but they are limited in what it can afford.

They could surely afford to institutionalize and upgrade the weekly "sneaker net" if they were sincere. The people curating and distributing the material could be recognized as small businesses and new types of material -- like news and education -- could be included.

The big stumbling block would be copyright. The government might not want to acknowledge copyright violation. If they chose to worry about copyright, they could negotiate block licenses with the owners of the material. Since they not getting any royalties for Cuban distribution today, low royalties, perhaps with a promise of increases over time, could be negotiated.

I've made a couple of other low cost proposals the Cuban government could implement in the short run -- a satellite pilot trial leading, if successful, to a broader roll out.

If they are sincere in the desire to prioritize information technology, they could also get behind and extend the weekly packages.

Interviews of ETECSA officials

Juventud Rebelde published a summary of what seems to have been a three hour online question and answer session between their readers (at least the ones with Internet access :-) and ETECSA officials.

They talked about ETECSA's data center, which has over 100 terabytes storage capacity today and will be expanded by five times next year. While large and a good start, this is not comparable to the data centers operated by companies like Google.

The data center went online in February 2014 to host data and services of state agencies and Nauta email. It currently hosts over 160 websites and what sounds like colocation for "over 15" organizations. (The following image, which accompanied the Juventud Rebelde post, is a general sketch of a Savvis data center, not a representation of the actual ETECSA data center).

The data center also hosts the Orion search engine developed at the University of Information Science. I tried a few Orion searches and, as far as I could discover, there are no images and it is only crawling .cu web sites. I could not even turn up a picture of Fidel Castro:

When asked about cloud storage for individuals and home Internet service, the officials made no commitments saying were focusing on shared capabilities due to limited funds and said they expected to improve the quality of service in more than 230 Internet access rooms at third party sites. A significant number of access points will be in Joven Clubs. WiFi access will also be available at these sites, but evidently prices will remain $4.50 per hour.

The officials promised to continue modernizing and expanding mobile networks, adding 800,000 "lines," allowing over 3 million users. Nothing was said about providing 3 or 4G capability.

The article leaves me with the impression that this was more like a press conference for ETECSA than hard-hitting question and answer session.

If I could ask questions of ETECSA, I would be interested in learning about their management and relationship to the Ministry of Communication.

-----
Update 12/30/2014

Juventude Rebelde published more of the online Q and A with ETECSA officials.

ETECSA said they plan to create an exchange point for the networks of Infomed, the universities, the Joven Clubs and the Ministry of Education and that there are points of presence in all Cuban municipalities.

The short article is accompanied by many questions and answers.

Can there be a uniquely Cuban Internet?

I don't criticize to knock the system down. On the contrary, I criticize to perfect the system.
Cuban blogger Carlos Alberto Pérez

The first instance of citizen journalism on the Internet was during the Soviet Coup attempt of 1991, where it was used to coordinate dissent and share news during protests. We tend to think of citizen journalism as anti-regime -- the Twitter Revolution, the Arab Spring.

But there are important counter examples, like that provided by blogger Carlos Alberto Pérez, who was recently profiled in a New York Times article on Cuban bloggers. Pérez is not a revolutionary seeking to overthrow the Cuban government -- he works for the Ministry of Communication and has government-provided access to the Internet at work and at home.

He criticizes the government in his blog La Chiringa de Cuba, but does not advocate revolution -- he "criticizes to perfect the system."

Carlos Alberto Pérez -- taken from a New York Times article and video on Cuban bloggers

I hope his point of view prevails -- rejecting both the far left and far right and finding Cuban solutions to Cuban problems.

How does this general principle apply to the Internet? Today, the Internet is under the control of an opaque monopoly, ETECSA. Neither I nor Cubans paying $4.50 per hour for slow DSL access or using their 2G cell phones like the current situation.

But, I would not like to see Cuba go to the other extreme -- ceding control over the Internet to a foreign investor in return for infrastructure. I have seen that approach in the US, and it is far from optimal and in developing nations like Cuba, it has produced poor results.

Hopefully, Cuba will find a uniquely Cuban way to a modern Internet. The goal should be eventually providing universal, affordable (free in some cases) access to the people of Cuba -- not profiting ETECSA, the Cuban government or foreign investors.

I would not bet on that rosy outcome, but, if it is to be achieved, it will take many years -- involving both short and long-term programs).

I do not know Carlos Alberto Pérez or what his job is in the Ministry of Communication, but I hope the Minister is more inclined to read his blog than to cut it off.

-----
Update 2/4/20114

A post in the Havana Times asking "Where are the US-Cuba talks headed" is accompanied by the illustration shown below. This gets at the reservation I have over the possibility (probability?) that the Cuban Internet will be turned over to foreign investors from the US or elsewhere. (It's not just the US, the French telecommunication company Orange has already made a deal with Cuba).

Who owns ETECSA and who runs the show?

The move toward normalization of relations between the US and Cuba has generated speculation that Internet access will improve markedly. I agree that that is a possibility, but it is far from assured. As a virtual Internet "greenfield," they have the possibility of building a uniquely Cuban Internet using current and future technologies.

But Internet policy and goals are a bigger question mark than technology and that brings us to Cuba's monopoly telecommunication service provider ETECSA (Empresa de Telecomunicaciones de Cuba S.A.).

The ITU describes ETECSA as "one of the last state telecommunication-sector monopolies" and Wikipedia says that 27% of ETECSA is owned by Rafin SA and the remainder is owned by the Ministry of Information and Communication (MIC).

Who owns ETECSA

But, is ETECSA state-owned? In 2011, Telecom Italia sold its 27% share of ETECSA to a company called Rafin, SA. The Central Bank of Cuba describes Rafin as a non-banking financial institution and lists the operations it is authorized to perform on their Web site.

If Rafin owns 27% of ETECSA, what about the other 73%. Wikipedia and the ITU report that that belongs to the the Cuban Government, but the Official Gazette of the Justice Minister cites the following equity shares: Telefónica Antillana SA, 51%, Universal Trade & Management Corporation SA (Utisa), 11%, Banco Financiero Internacional, 6.15%, Negocios en Telecomunicaciones, 3.8% and Banco Internacional de Comercio, 0.9%.

Are these the owners of ETECSA?

Who manages and determines ETECSA policy?

Rafin and several of the other organizations listed above are "anonymous societies," which I take to be something similar to "corporations" in the US. The others are banks and a corporation.

I am not an economist, but this leaves me wondering what the meaning of an SA or corporation is in a communist nation -- aren't these capitalist organizations? That leads to other questions like -- who put up the money for the purchase of Rafin's 27% share of ETECSA? (There is an unsubstantiated rumor that Rafin is owned by the Castro brothers).

What happens to ETECSA profits? Do the organizations that own it receive dividends? Are they re-invested? Who covers losses?

Who sets ETECSA policy? Is there the equivalent of a board of directors? Does the MCI have a voice?

Who makes operational decisions -- which services to offer, where to invest? Who sets prices for services?

This post asks several questions and provides no answers, but the answers to those questions will determine the future of the Internet in Cuba. I hope they do not squander the opportunity to create a uniquely Cuban internet (as they did in 1997) -- for the people of Cuba and as an example for the rest of us.

-----
Update 4/13/2015

I remain confused about the ownership of ETECSA and their relationship to the Cuban Government. LinkedIn classifies ETECSA as "privately held" and 481 of more than 10,000 employees have accounts.

-----
Update 11/29/2015

For some inexplicable reason, Google Alerts just alerted me to a two year old post on ETECSA the Growing Monopoly. It is clearly out of date, but it discusses a Havana Consulting Group study estimating ETECSA revenue from prepaid wireless services as $562 million for 2013. That has doubtless gone up and now they are getting revenue for very expensive WiFi and Internet access rooms. A lot of this is prepaid from outside of Cuba using services like Ding.com.

This leaves me wondering what ETECSA's income statement and balance sheet look like, but I doubt that anyone outside a few ETECSA executives and Cuban government officials has those statements. (Where is Wikileaks when you need them)?

Cuba has the opportunity to create a unique Internet -- using the policy experience of other nations and tomorrow's technology, but if the goal is to maximize ETECSA profit or government revenue, the Cuban Internet will remain in the 1990s.

A Cheltenham Carol

On the Twelfth Day of Christmas my true love sent to me:

Twelve Zettabytes

Eleven Encryption Layers

Ten Coders Coding

Nine Hackers Hacking

Eight Routers Routing

Seven Inspected Packets

Six Spies-a-Spying

Five Back Doors

Four Fishing Warrants

Three Haystacks

Two Secret Laws

And a Paean to Proportionality

A joke and some cool images on US-Cuba relations

This post is off topic -- not about the Internet per se -- but the joke cracked me up and the image gallery accompanying this NY Times article are a terrific recapitulation of US v Cuba since 1959.

Click here for image gallery

Is the Internet a priority for Cuba? The ball is in their court.

Yesterday, a reporter asked me to comment on two quotes following the release of Alan Gross:

1. President Obama: "Unfortunately, our sanctions on Cuba have denied Cubans access to technology that has empowered individuals around the globe."

2. Senator Rubio: "The reason why they don't have access to 21st century telecommunications -- like smart phones, like access to the internet -- is because it is illegal in Cuba."

The reporter asked which statement I thought was closer to the truth.

While there is something to be said for both, I had to side with Senator Rubio. There are three primary causes for the sad state of the Cuban Internet:

1. Fear of an open Internet by the Cuban government: When the Internet first came to Cuba, there was high level debate over how to deal with it. Raúl Castro led the anti-Internet faction and they decided to restrict access. (Around the same time, the Chinese decided to encourage the growth of the Internet, but tightly control content and monitor users).

2. Financial constraints: The Cuban economy was in terrible shape at that time due to the collapse of the Soviet Union and they remain poor, but today they are better off than many Latin American and Caribbean nations with better Internet infrastructure.

3. The US trade embargo: The embargo raised the cost of computers and communication equipment in Cuba, which has had a dampening effect. This effect has been diminished with the emergence of China as a major manufacturer of communication equipment, but it is still a factor. The 2009 US decision authorizing the provision of communication services in Cuba could have enabled Cuban satellite connectivity -- the sort of thing Alan Gross was imprisoned for.

Senator Rubio could also point to the ironic facts that Cuba's first connection to the global Internet was over a Sprint link funded by the US National Science Foundation and that nearly all Cuban traffic flows through the United States today.

But, that is history. Cuba now says they want to give the Internet priority. I hope they mean what they say -- the ball is in their court.

-----
Update 12/20/2014

A number of politicians and Cuba watchers discussed these quotes. Their consensus was:

The U.S. sanctions have played a role in limited availability of technology. However, Rubio is right that the Cuban government has nearly complete control over the Internet. That isn’t a result of sanctions on telecommunication business activity in Cuba. Even if the United States fully repeals its embargo, government control over Internet access could continue.

We rate Rubio’s statement Mostly True.

-----
Update 12/29/2014

Diario de Cuba asked Cuba experts José Remón, Iván Darias Alfonso, Ted Henken and me what we thought about the future of the Cuban Interent. Each reply is worth reading, but they seem to agree that the ball is in Cuba's court now -- that the growth of the Internet will not be constrained by the US.

-----
Update
December 31, 2014

I should have posted these earlier, but here are the Cuba policy changes the President has announced "in order to increase Cubans’ access to communications and their ability to communicate freely:"

• The commercial export of certain items that will contribute to the ability of the Cuban people to communicate with people in the United States and the rest of the world will be authorized. This will include the commercial sale of certain consumer communications devices, related software, applications, hardware, and services, and items for the establishment and update of communications-related systems.
• Telecommunications providers will be allowed to establish the necessary mechanisms, including infrastructure, in Cuba to provide commercial telecommunications and internet services, which will improve telecommunications between the United States and Cuba.

The US will license the export of any Internet-related goods and services Cuba will allow -- what will they allow?

Welcome home Alan Gross!

I've been quite busy since the release of Alan Gross, so have not taken the time to comment on it on this blog.

I've been following his story on this blog -- both the technology and the politics -- for several years, and I am very happy to end that thread!

I'm also happy that his release has removed an obstacle to the normalization of relations between the US and Cuba -- that will benefit the Cuban people and the Cuban Internet.

This may or may not be the end of Alan Gross's involvement with the Internet in Cuba -- he is clearly an advocate of Internet freedom and a friend of the Cuban people.

I hope he adjusts quickly to his freedom and am looking forward to hearing more from him if he cares to share his experience.

The Cuban Internet in context

The Cuban Internet is minimal and unfree -- far worse than one would expect in a nation with a relatively high UNDP Human development index.

In the last week or so I've seen a spate of articles (for example this one) pointing out that only a few, relatively rich Cubans can access the Internet and that the Cuban Internet is not free. This is not exactly news. (It may be news that Cuba was an early networking leader).

These articles were triggered by the publication of the 2014 editions of the Freedom House Freedom on the Net report and the International Telecommunications Union (ITU) Measuring information society report. I will highlight a few of the reported findings on Cuba and put them in context by looking at some Cuban data from the United Nations Development Program (UNDP) Human development report.

Freedom on the Net, 2014

Freedom House ranked Cuba 62nd among the 65 nations they surveyed. As you see below, the overall freedom index is composed of three sub-indices: obstacles to access, limits on content and violations of user rights:

Net freedom and sub-component ranks out of 65 nations

These scores put Cuba in the group of nations that includes China, Syria and Iran -- not exactly august company.

Freedom House puts Cuba in the context of all nations with the following plot:

Internet freedom versus penetration -- purple indicates not free, green free

(Too bad we cannot combine Iceland's Internet with Cuba's climate).

The report includes a well documented essay on the state of Cuban Internet freedom and you can see summaries of Cuba's rating in the 2012 and 2013 Freedom on the Net reports here.

Measuring information society, 2014

This report is a compilation of data and analysis of the state of information and communication technology (ICT) in 166 countries. The key summary statistic is the ICT development index (IDI), which is based upon three sub-indices as shown here:

The ITU framework and index structure

The ITU model says infrastructure access plus capability and skills lead to ICT use which impacts individuals, organizations and society in a nation.

Cuba ranks 125th on the IDI -- they are doing well on skills, but access pulls them down:

IDI and sub-index ranks out of 166 nations

The IDI and sub-indices are a function of many variables and they include telephone, mobile and Internet indicators -- so, for example, Cuban access is pulled up by low-cost fixed telephones and pulled down by fixed broadband prices, as shown here:

Latin American fixed broadband price

Composite indicators like these offer a very rough characterization of the Internet in a nation and there is much more Internet-related data on Cuba in the report. For example, broadband is limited to 2 mbps DSL and even that is not available in private homes; the Cuban household connectivity rate is only 3.4%; Cuban IDI is 32nd out of 32 ranked nations in Latin America and the Caribbean (Haitii was not included in the IDI rankings for some reason); in spite of the ALBA cable,Cuba has the lowest international bandwidth per user in the Americas; Cuba is one of four nations in the Americas without wireless broadband and ETECSA is one of the last state telecommunication-sector monopolies in the world.

Human development report, 2014

Like the others, the UNDP human development report compiles an overall index, the human development index (HDI). The HDI is computed for 187 countries and territories and is a composite of sub-indices for health, education and income (up to a cutoff point).

Cuban fares better on the HDI than the other indices -- it is ranked 44th in the world and second only to Chile in Latin America and the Caribbean:

Cuba is ranked 44th in the world on the HDI.

The Cuban HDI is second to Chile in Latin America and the Caribbean.

As we see below, Cuba has made steady progress with the exception of the "special period" after the fall of the Soviet Union and more recently, in education. (What's up with education)?

Cuban HDI and constituent indices over time

The report includes profiles of the state of human progress in each nation -- you can see Cuba's here.

Cuba's HDI rank is laudable and it was achieved essentially without the Internet -- think of what they could have achieved with a robust Internet (even if it were controlled as in China). That is a sad opportunity loss. The Cuban government denies fear of the Internet, but they have restricted it since its inception.

The tip of the iceberg

The above is only a quick look at these three reports -- each is extensive, well researched and contains significant analysis. They also publish their data and provide interactive analysis tools so you can play with the data yourself. For example, the UNDP makes their data available in Google's Public Data Explorer (PDE), which makes dynamic plots of time series.

I used the PDE to plot the relationship between the number of users and the HDI in 2008, the year Cuban education began to drop off:

The graph changes dynamically as the year slider at the bottom is moved. Check it out for yourself, here -- you will like the dynamic presentation.

(This sort of analysis was introduced by Hans Rosling -- check out these great presentations if you are not familiar with his work).

The UNDP data is also available as a Stat Planet world map, which I used to create the two HDI charts shown above.

If, like me, you like the global perspective, you will want to look at these three reports and the accompanying data.

Granma says IT is now a priority -- is this true change?

An article in Granma says the "informatización" of society is a priority for Cuba.

They say the 154 public "navigation" rooms were a trial balloon -- it is not clear whether they were referring to the technology or public acceptance -- but evidently they have been pleased with the results.

At one point, the article refers to the "information superhighway" -- an expression that seems as dated as the Cuban Internet.

They say Cubans will not only "drink" from the Internet, but will contribute content -- putting the best of Cuban of culture, education, knowledge and humanism on line. As I have suggested, online education and medical information would be a good place to start.

The article speaks of the ongoing, gradual implementation of 26 projects, but it is not clear what sorts of projects they refer to. One concrete promise is the expansion of public access rooms in libraries and post offices. They will also be rolling out digital television.

If they are sincere, they should consider satellite connectivity as a low-cost, interim option and follow up on that friendly visit from Google a while ago. I would also keep an eye on Elon Musk's satellite plans.

If we are going to see a new era in Cuba's attitude toward the Internet, policy considerations are more important than technology -- the goal is to benefit the Cuban people, not the government, ETECSA (or AT&T). Cuba is nearly an "Internet greenfield" -- there is little installed infrastructure so they could plan on future technologies, look for expertise around the world and set goals that are uniquely Cuban.

This article appeared around the same time as an official presentation on technology to students at the University of Havana. I am not enough of a "Cuba watcher" to have an opinion as to whether this signifies a true change -- I hope it does.

-----
Also see this post.

Officials say Cuba is not afraid of new technology -- they just lack the funds

Wilfredo Gonzalez and Abel Prieto

A recent Cuba Sí post quotes Cuban officials saying that money, not fear is responsible for the sad state of the Cuban Internet.

Presidential adviser Abel Prieto told a group of Havana University students that the island is "not afraid of technology." Computer science and communications vice minister Wilfredo Gonzalez assured the students that there were no government policies that restrain development of new technologies. He said economic conditions hinder development and pointed out that the International Telecommunications Union (ITU) ranks Cuba 14th in the world on training people on new technologies and 153rd in access to those technologies.

I will have more to say on those ITU rankings in a future post, but, for now, let me point out that the ITU ranks Cuba last among all nations in Latin America and the Caribbean on information and communication technology development and the United Nations Development Program (UNDP) ranks Cuba second only to Chile in Latin America and the Caribbean on their Human Development Index, which is based upon health, education and income.

UNDP human development index

How is it that a nation that is better off than other Latin American nations is unable to afford better Internet connectivity?

If Cuba were not afraid of information technology, Alan Gross would not be in prison and a modest proposal for satellite connectivity would be implemented and replicated. Don't get me wrong, Cuba has financial need, but a build out using satellite technology would be quite affordable as an interim step toward a modern, fiber-based Internet.

An Alan Gross suggestion from an amateur

Alan Gross is starting his sixth year in prison. As I understand it, Cuba wants to discuss releasing him in exchange for the remaining Cuban Five prisoners and the US refuses to do so because they do not see equivalence in the cases, saying that Gross was trying to facilitate Cuban communication, not doing espionage.

If that negotiation is at an impasse, how about trying to reframe the issue? Alan Gross's project was not unique. It is known that USAID funded another project to get communication equipment into Cuba, Twitter-like ZunZuneo and A program to encourage dissent among youth. We do not deny funding these programs, but say they were not trying to overthrow the Cuban government.

Instead of a prisoner swap, how about the US sincerely apologizing for and promising to end projects like these in return for Gross's freedom? The US has already been outed, so we would not be revealing anything new in apologizing. At the same time, it would give the Cuban government a propaganda win and a rationale for releasing Gross on humanitarian grounds -- making them look good. Would they go for that?

I understand that hard-liners on either side would refuse such a deal and I understand their reasons, but think about poor Alan Gross.

Another round of data retention

[Updated 4 December 2014]
[Further updated 20 January 2015 to add tweet.]
[Also updated 5 January 2015 with this brief commentary on the Home Office Factsheet:

Page 1: Top Lines

"IP resolution is the ability to identify who in the real world was using an Internet IP address at a given point in time." Data retention at best identifies the device or connection being used and any associated subscriber details. The subscriber is not necessarily the user. Page 2 of the Factsheet is accurate: "This data can help identify who has made a communication, when, where and how." (emphasis added) 

Page 1: Background

"However, some IP addresses are shared and allocated dynamically." True, but dynamic allocation is not what Clause 17 is about. Dynamic IP address allocation is sequential temporary allocation of a public IP address to one customer after another. Dynamic IP addresses are already explicitly mentioned in the DRIPA datatypes (Data Retention Regulations 2014, Schedule, Paras 13(1)(b) and 11(3)). It is evident from the diagram on page 3 of the Factsheet that the problem being addressed by Clause 17 is simultaneous sharing of a single public IP address by multiple ISP customers. 

Page 3 : Diagram

"At 4pm 2,500 people are using a single IP address on the internet." Exactly. The issue is simultaneous sharing of a single IP address, not dynamic (sequential) allocation of an IP address. 

"The e-mail service provider now provides police with IP address and port number used to send the e-mail and accurate time."  In order to do this the e-mail service provider in the diagram example will have had to retain IP address, port number and timing data.  Will such providers, as well as internet access providers, be subject to mandatory retention?

"Police seek details from internet access provider. Internet access provider now identifies the individual using the unique combination of IP address and port number provided at 4pm." The internet access provider identifies the customer, who may be but is not necessarily the individual who used the device in question.] 

Four months after DRIPA and 18 months after putting down a marker in the May 2013 Queen’s Speech, the UK government has embarked on a new round of legislation for mandatory retention of communications data. This time it is under the banner of IP address matching.

The Counter-Terrorism and Security Bill had its Second Reading yesterday and is expected to go into Committee on 9 December. Clause 17 will extend DRIPA to new categories of communications data.

DRIPA’s existing data retention obligations, rushed through Parliament in four days in July, are of course controversial. They are the subject of a threatened legal challenge by David Davis MP and Tom Watson MP.  The proposal to add IP address matching dates back to a recommendation of the Joint Committee on the draft CommunicationsData Bill in December 2012.

What new categories of communications data would have to be retained?

Clause 17, like so much UK legislation in this field, is difficult to understand. The Explanatory Notes and the Impact Assessments are more detailed, but still confusing. (The Home Office has subsequently issued a Factsheet.) MPs suggested in the Second Reading that the drafting of Clause 17 needs to be examined critically.  They are right.

The overall aim seems to be to mandate retention of data that can link a given communication made via a simultaneously shared public IP address to one of many devices or connections that may have been using that IP address at a given time.  Clause 17 labels this “relevant internet data”. We might call it linking data.

This appears to break down something along the following lines (the first two of these are illustrated in the useful diagram in the Home Office Factsheet).

• Some ISP and mobile operator systems don’t allocate one public IP address to one customer device or connection, but have many customers sharing an IP address simultaneously. They could be required to retain linking data such as port numbers.
• Even if an ISP retains IP address and (say) port number records, it cannot be sure of identifying a single device or connection unless law enforcement can provide it with a both a port number and an IP address to look up. So a cloud storage or web e-mail provider accessed by the user could also be required to retain logs of linking data visible to it, such as port numbers.
• Operators such as public Wi-Fi hotspots could be required to log MAC addresses.

Weblog data (records of websites accessed by customers) would be excluded from mandatory retention by internet access providers such as ISPs and mobile operators.

The Overarching Impact Assessment provides this summary:

“IP Resolution: Allow for a power to require communications service providers to retain the data necessary to attribute an IP address to an individual.”

Taken literally, that is a power to require the impossible. We don’t have IP addresses tattooed on our foreheads. Even if we did that would not identify us, as opposed to someone else, as the user of the device at any given time. An IP address at best identifies a device or a connection. The ISP may then be able to link that with the identity of its subscriber customer, but no more. The subscriber may or may or not be the user. The Factsheet diagram, unfortunately, perpetuates the myth that an IP address identifies a user.

DRIPA in fact already covers retention of subscriber data for IP addresses (both where the IP address is static and where it is dynamically allocated in sequence to different customer devices and connections). What it doesn’t cover is the single public IP address simultaneously shared among many of an ISP’s customers.

The Bill is meant to be only about IP address matching. So it is not immediately obvious why the Impact Assessments say that the Bill will expand DRIPA to cover a wider range of internet services. On the other hand Clause 17 does not seem to do this, since it only amends the categories of data to be retained. DRIPA has already adopted an extremely broad underlying definition of telecommunication services.

The new obligations would be subject to the same 31 December 2016 sunset clause as DRIPA. As with DRIPA itself, mandatory retention will apply only to data generated or processed in the UK by public providers in the process of providing the telecommunications services concerned; and then only to those on whom the government serves a notice. The Impact Assessment says that the service providers most likely to be affected by the Bill have been consulted.

That is my current stab at what Clause 17 is trying to do.  However it is a puzzling piece of drafting. Here are some questions worth considering.

What is ‘relevant internet data’?

Clause 17(3)(b) defines this as communications data relating to an internet access service or an internet communications service which:

“may be used to identify, or assist in identifying, which internet protocol address, or other identifier, belongs to the sender or recipient of a communication (whether or not a person)”.

This is the most curious part of Clause 17. The problem is surely not identifying which IP address ‘belongs’ to a given sender or recipient of the communication, but identifying which device or connection (of many) was used to make a given communication via a given shared public IP address. Is it drafted the wrong way round?

What is an ‘identifier’?

The Clause says that “identifier” means “an identifier used to facilitate the transmission of a communication”.  More helpfully, Clause 17(3)(b) tells us that an IP address is an identifier. The Explanatory Notes seem to conflate linking data and the shared identifier that we are trying to tie to a device or connection:

“…  An IP address can often be shared by hundreds of people at once – in order to resolve an IP address to an individual other data ("other identifier" in this clause) would be required.”

Whatever the ‘other data’ may be, surely it is not the ‘other identifier’ in Clause 17(3)(b)?

What else might be covered by ‘identifier’? A MAC address, although it operates at a lower (physical) layer than an IP address, would seem to qualify. But Clause 17 is not avowedly about retention of new categories of identifiers, only retention of data capable of linking shared identifiers (such as IP addresses) to an individual device or connection. If a MAC address is itself an identifier, does that prevent it being linking data? The Explanatory Notes suggest that a MAC address could also be linking data:

“Data necessary for the resolution of IP addresses could include port numbers or MAC (media access control) addresses.”

Are there circumstances in which a MAC address could be used to identify the particular device that sent a communication via a shared IP address? Public Wi-Fi hotspots seem a likely candidate. However a MAC address would presumably be less useful than a port number, assuming that the MAC address is not visible from outside the hotspot and so could not be logged at the other end of the communication.

What are an internet access service and an internet communications service? 

These are the foundation stones of Clause 17. Communications data cannot be required to be retained unless it relates to an internet access service or an internet communications service. These terms are also critical to the scope of the weblog data exclusion. Many will be surprised, therefore, to find that neither term is defined.

What do the terms mean? The glib answer is ‘whatever they meant in the EU Data Retention Directive’. That is their origin. They were used (but not defined) in the Directive.

The 2009 Data Retention Regulations, which implemented the Directive, followed its terminology. When the Directive was invalidated DRIPA re-enacted the datatypes that were in the Schedule to the 2009 Regulations. So the 2014Data Retention Regulations that were made under DRIPA again used the two terms, notably in the definition of ‘User ID’: “a unique identifier allocated to persons when they subscribe to, or register with, an internet access service or internet communications service.” Perhaps unsurprisingly given the government’s commitment to re-enact the 2009 datatypes identically, the 2014 Regulations again left the terms undefined. 

That is a plausible historical reason why the terms have been left undefined in Clause 17. But even though there is a breadcrumb trail back to the Directive, the lack of definitions in the Directive means that uncertainty remains particularly over ‘internet communications service’. Does it relate to any type of communication, or is it more limited, for instance to e-mail, messaging or telephony providers? The diagram in the Factsheet uses the example of an e-mail provider. However the Impact Assessment suggests that the government believes it has a broad meaning, covering for instance cloud storage services:

“For example w[h]ere a user uploads an illicit file to a cloud server that server provider, if subject to a data retention notice, would be required to retain sufficient information to enable the internet access provider to identify the user.”

We look forward to illumination of these and no doubt other points as the Bill proceeds. Meanwhile, the bigger question of whether any of this is compatible with the European Convention on Human Rights and the EU Charter of Fundamental Rights remains to be fought out. 

[My 8 point tweet of points on Clause 17:

1/8 Is it about dynamic (sequential) IP address allocation? No. Already covered in DRIPA and so excluded from Cl 17.

— Graham Smith (@cyberleagle) January 20, 2015

2/8 The Home Office Factsheet suggests Cl 17 is about simultaneous use of one public IP address by many customers.

— Graham Smith (@cyberleagle) January 20, 2015

3/8 But you'd never guess that from reading Cl 17. What else might it cover? Its vague drafting gives little clue.

— Graham Smith (@cyberleagle) January 20, 2015

4/8 The Fact Sheet shows it is meant to cover not just internet access, but cloud/web e-mail providers who generate or process data in UK.

— Graham Smith (@cyberleagle) January 20, 2015

5/8 Cl 17 isn't limited to data linking a device or connection to a public IP address. Includes 'other identifiers' as well as IP addresses.

— Graham Smith (@cyberleagle) January 20, 2015

6/8 What is an 'other identifier'? A MAC address, said the Minister on 9 Dec. The EN seems to suggest a MAC address is linking data. Both?

— Graham Smith (@cyberleagle) January 20, 2015

7/8 'Other identifier' is said to 'future proof' Cl 17 by making it 'technologically neutral'. In a provision sunsetted in Dec 2016?

— Graham Smith (@cyberleagle) January 20, 2015

8/8 RIPA was drafted to be technologically neutral. The result was a statute universally acknowledged to be impenetrable. #BeenHereBefore

— Graham Smith (@cyberleagle) January 20, 2015

[Updated 4 December 2014 with references to the Home Office Factsheet and minor clarifications and edits. Further update 5 January 2015 with comments on the Home Office Factsheet. Further updated 20 January 2015 to add tweet.]

Of straws and haystacks

Much post-Snowden attention has been directed to GCHQ’s TEMPORA programme, authorised (so it is thought) by a rolling series of external interception warrants under section 8(4) of RIPA. (See foot of this post for an explanation of Section 8(4) warrants and the restrictions, particularly for communications of persons within the British Isles, on their use.)

TEMPORA captures communications in bulk from transatlantic fibre optic cables, then filters them by computer leaving a residue of sifted material that GCHQ and NSA analysts can examine. It is said to process 40 billion items a day.

The often repeated justification for bulk collection and sifting is that we have no method of identifying and separating individual communications at the point of collection, so we must gather the straws and sift the resulting haystack. The usual metaphor is looking for needles, implying objective distinctions. It may be better to think of looking for straws.

What kind of straws can be looked for? The haystack can, within the restrictions laid down by RIPA, be sifted to detect the straws of pre-existing persons of interest. However Section 8(4) warrants go beyond that.  The captured material can also be searched and analysed to form new suspicions.   Home Office official Charles Farr said of RIPA in his witness statement in the current Investigatory Powers Tribunal proceedings:

“Other information that is obtained via interception is used to identify other previously unknown communications of existing targets, and to identify new targets for investigation. Indeed, a significant proportion of initial intelligence leads derive from interception operations.” (emphasis added)

We do not know what proportion of initial leads are false positives, casting suspicion on blameless people. We do not know how many true positives the system misses. Moreover suspicion is a highly subjective matter.

History suggests that general collection and subject matter analysis was an established approach to external communications long before today’s separation problems arose.

The ancestor of RIPA Section 8(4) was Section 4 of the Official Secrets Act 1920, legislated in the immediate aftermath of the First World War following the lapsing of wartime powers.  It empowered the Secretary of State to issue a warrant requiring a telegraph operator to hand over telegrams entering or leaving the country:

“Where it appears to a Secretary of State that such a course is expedient in the public interest, he may, by warrant under his hand, require any person who owns or controls any telegraphic cable or wire, or any apparatus for wireless telegraphy, used for the sending or receipt of telegrams to or from any place out of the United Kingdom, to produce to him, or to any person named in the warrant, the originals and transcripts, either of all telegrams, or of telegrams of any specified class or description, or of telegrams sent from or addressed to any specified person or place, sent or received to or from any place out of the United Kingdom by means of any such cable, wire, or apparatus, and all other papers relating to any such telegram as aforesaid.”

The Attorney General Sir Gordon Hewart introduced the provision in Parliament as a measure for detecting foreign spies:

“The postal and cable censorship which we had during the War, and which was of the greatest possible value and importance, was removed shortly after the Armistice. That being so, it is necessary that there should be power at least to compel the production of the originals and the transcripts of certain telegrams. It is not a power to stop telegrams. It is merely a power to compel the production of the originals and transcripts sent to, or received from, any place out of the United Kingdom; and the main purpose of that provision is to enable the authorities to detect and deal with attempts at spying by foreign agents.”

Earl Winterton invoked a familiar mix of foreign threats and ‘nothing to hide, nothing to fear’:   

“Everyone knows we do not live in ordinary times. Everyone knows there are plots and conspiracies against this Realm which are being carried out in foreign countries and some parts of the British Empire, and that, however one may dislike the idea of imposing additional restrictions on the subject, it is necessary for the Government to have that power. I suggest there is nothing to interfere with a person going about his legitimate business. The right hon. Gentleman, for example, made great play with Clause 4 of the Bill. … Surely he does not suggest that in the critical time in which we are living to-day a Secretary of State should not have power, if it seem desirable in the opinion of the Government that he should exercise that power, to find out what is being cabled to and from this country. Of course, it is a most necessary power, which every government ought to have.”

John Thorpe MP put the State firmly ahead of the individual:

“… In my view the State is in great danger, and no power which would tend to protect it should be withheld from the Government. We heard something from the same right hon. Gentleman of the liberty of the subject. In my view, the subject has no liberty when it is in conflict with the good-being of the State. When the liberty of the individual conflicts in any way with the well-being of the State, then it becomes license.  

… The law-abiding citizen, the man who says that his country is his first consideration, need have nothing to fear whatever from the Clauses of this Bill. … The only man who has anything to fear is the man who puts self before country, the man who says, "I want liberty, and the State can look after itself." He is a danger, and I congratulate the Government on the efficient manner in which they propose to deal with him.”

The legislation duly passed. For nearly 50 years Section 4 did its work in obscurity. The 1957 Birkett Inquiry into interception of communications did not mention it. (The Birkett Committee’s terms of reference were limited to the executive power to intercept, which was different and separate from the statutory power to issue warrants under Section 4.)

Things changed in February 1967.  Section 4 came to public notice when journalist Chapman Pincher revealed in the Daily Express that cablegrams sent out of Britain were being collected from the Post Office and private cable companies for scrutiny. This incident is most famous for sparking the ensuing D-Notice row. But the substance of Pincher’s ‘Cable Vetting Sensation’ story is of interest here. He revealed that:

“There is no hold-up or censorship of the cables. But on the morning after they have been sent or received they are collected and sifted by a Post Office department concerned with security. Then any cables believed to be of special interest are passed to the Security Services. 

They are studied there, copied if necessary, and returned to the Post Office and cable offices after being held for 48 hours. 

Most of the original cables and telegrams go out through the Post Office, which owns the former Cable and Wireless Company. Cables passed through private companies—mainly branches of foreign concerns operating in Britain—are collected in vans or cars each morning and taken to the Post Office security department. 

The probe is conducted under a special warrant, signed by a Secretary of State under Section 4 of the Official Secrets Act and regularly renewed to keep it valid.”

A week later Alan Watkins in the Spectator wrote:

“Indeed, sources confirm that a Ministry of Works van regularly takes cables—it is not clear whether they form a random sample, or come from a particular sender or class of senders—along to the Ministry of Defence for examination. The authority for such action is section 4(1) of the Official Secrets Act, 1920.”

The Radcliffe Report on the D-Notice affair confirmed the substantial accuracy of Pincher’s story:

“It does involve a regular collection of copies of messages transmitted by the Post Office and other cable offices with a view to the total collected being sorted and certain defined categories of them being set aside for inspection by the intelligence agents of Her Majesty’s Government. … 

The practice is authorised in law by section 4 of the Official Secrets Act, 1920 … . According to the information given to us, this power has been regularly exercised against transmitting companies since the coming into operation of the Act. … 

In fact only a small percentage of the total telegrams handled is put aside [by the sorters for inspection]. … The Daily Express article was … not inaccurate in any sense that could expose it to hostile criticism on that score.”

The Government White Paper published simultaneously with the Radcliffe Report said it would be contrary to the public interest to say in what detailed respects the article was misleading. It also said:

“It was precisely because national security was threatened that, from the outset, the Government regarded the publication of certain information in the Daily Express of 21 February as a matter of the utmost gravity. … It is the duty of the government, in the light of all the advice they have received and the information they possess, to record that the effect on national security of that publication has been to cause damage, potentially grave, the consequences of which cannot even now be fully assessed.”

The White Paper complained that the article created:

“the sensational impression that the Government were responsible for introducing new and sinister procedures.  There were, and are, no such new and sinister procedures. The activities involve no element of prying into the private affairs of the citizen. Such activities are, in fact, carefully controlled and confined and the article was misleading when it inferred the Government might use them improperly.”

Although the government denied (supported by the Report) that any new practice had recently been introduced, the possibility that routine vetting was a long established practice was left open.

Several themes from this episode resonate today:

• Bulk collection, sifting and examination
• Periodically renewed warrants
• Revelations about the extent of use of powers, answered by denials that the powers are abused
• Assertions, to be taken on trust, that publicity has caused damage to national security
• Intrusion into privacy rebutted on the basis of close control over the intrusion
• Bulk collection defended on the basis that only a small percentage of the items collected is inspected

Like the 1957 Birkett Report, the Home Office and Diplock Interception Reports of 1980 and 1981 made no mention of the Section 4 powers.  The reports were limited to statistical information about non-statutory warrants.  

The reports recognised the invasion of privacy involved in interception warrants. Lord Diplock said:

“The exercise by the State of any power to read or listen to communications taking place between private citizens involves an invasion of their privacy which has always been looked upon by the public with suspicion and distaste.”

The 1920 powers lasted until 1985, when they were replaced by the Interception of Communications Act (IOCA). The preceding White Paper had promised that the legislation would include provisions “along the lines currently covered by the Official Secrets Act 1920.” Whilst IOCA folded interception of external communications into the new statutory system for issuing warrants, the warrantry power for external communications continued to be broader than for internal communications.

So what is now the Section 8(4) warrant trod its own quiet path from 1920 to 1985, exposed to public scrutiny only once as a result of the Chapman Pincher cable vetting story – to which the Government of the day reacted almost identically as did the government of today to Edward Snowden’s TEMPORA disclosures.

A vanload of cables is on a smaller scale than 40 billion items of data per day, but the principle and method is the same: general capture, selection, examination. Long before any technical argument that targeted interception is impossible, the 1920 legislation enabled the government to engage in suspicionless bulk capture followed by subject-matter analysis of external communications.

In Chapman Pincher’s day collected telegrams and cables were evidently sorted manually. Human beings looked at them all and decided which were worthy of further examination. Now the initial capture, sift and discard is computerised.  The government argues that capture involves only a technical interference with privacy compared with a human being examining intercept material:

"The Respondents accept that the interception of a communication under a s. 8(4) warrant may be regarded as giving rise to a technical interference with the Art. 8 rights of the parties to the communication even if that communication is not and/or cannot be read, looked at or listened to by any person." (Open Response, IPT proceedings)

Going back further than 1920, in 1765 Lord Camden, the judge in Entick v Carrington, held that general search warrants had no legal basis. It is perhaps idle to speculate how he might have reacted had Lord Halifax (the then Secretary of State) said:

“Fear not, Mr Entick.  True we have ransacked your home, broken the locks on your desks and cupboards and seized your papers and correspondence.  But, since we have not yet examined any of them, that is a merely technical breach of privacy.  We have strict safeguards in place to ensure that we will only look for material about that renegade Wilkes who is outside the British Isles, skulking in Paris.”

Footnote: How does a Section 8(4) warrant work?

The Foreign Secretary can issue a RIPA warrant for purposes of national security; for preventing or detecting serious crime; for safeguarding the economic well-being of the United Kingdom (if related to national security); or, in relation to serious crime, mutual legal assistance treaties with other countries. He must believe the warranted interception and disclosure to be proportionate to what it seeks to achieve; and must take into account whether the information he thinks it necessary to obtain could reasonably be obtained by other means.

A Section 8(4) warrant, unlike an ordinary RIPA Section 8(1) warrant, does not have to be targeted at the communications of a particular person or premises. It can authorise general bulk collection at the level of the cable. But while a targeted Section 8(1) warrant can be used to intercept internal communications (those sent and received within the British Isles), the overall purpose of a Section 8(4) warrant must be the collection of external communications (sent or received outside the British Isles). So external communications are those where both ends of the communication, or only one end, are outside the British Isles.

Internal and external communications tend to be inseparably mingled within a single fibre-optic cable. So RIPA allows a S.8(4) warrant to authorise the capture not only of external communications, but any internal communications unavoidably swept up with them.

After capture of the communications come selection and examination. RIPA constrains these in different ways.

Captured communications (whether internal or external) can be examined if they are within a description certified by the Secretary of State in the warrant. That description could be very broad. However they can be selected for examination only in a way permitted by RIPA’s selection rules. These govern the automated filtering down of the captured communications to a database of material and also the queries made by analysts against the database.

The rules restrict the use of selection factors targeting the communications of people known for the time being to be in the British Isles. But despite this there are several gateways via which a communication sent or received by someone in the British Isles and captured under a S8(4) warrant could end up being examined by a GCHQ analyst.

Foreign Secretary Philip Hammond touched on two of the gateways in a clarification of his recent evidence to the Intelligence and Security Committee of Parliament. He posited a communication (say an e-mail) between someone in the British Isles and someone abroad. In general terms it could not be selected where the subject of interest is the person in the British Isles. That would require a further step such as the Secretary of State’s modification of the S.8(4) warrant under the exception in RIPA S.16(3). However the e-mail could be selected for examination if the person outside the British Isles is the subject of interest.