The problem with the survey showing the public supports the FBI over Apple

In the ongoing, very public conflict between Apple and the Justice Department over the phone of deceased San Bernardino shooter Syed Farook, the Justice Department seems to be winning in the court of public opinion.

A survey conducted by the Pew Research Center found that Americans are by and large siding with the Justice Department. 51% of respondents thought that Apple should cooperate with the FBI and unlock the phone, 38% said the company shouldn’t, and 11% weren’t sure. The poll was conducted among a sample of 1002 adults, half of whom were surveyed by landline and half by cellphone.

The survey follows last week’s decision by California judge Sheri Pym to order Apple to help authorities access the iPhone 5c that belonged to Farook by building a workaround to the encryption and data-wiping programs built into the phone. CEO Tim Cook responded that Apple would challenge the FBI in a letter to customers posted February 16, saying the company supported the investigation but had concerns about the implication the order would have for consumer privacy going forward.

The most thought-provoking finding in the survey is the difference in opinion across age groups, which is quite striking. While more people from all age groups favored unlocking the phone, the margin by which they did so varies a lot, and younger respondents were much more split on whether or not Apple ought to decrypt the phone for the FBI than their older counterparts.

It’s an interesting survey: people broadly support the FBI; young people are more conflicted about the issue, especially people under 30; level of education doesn’t make much difference; and smartphone ownership does. It’s also flawed.

Here’s the key question Pew asked, taken from their questionnaire:

The question and the possible answers to it are entirely phrased around whether or not Apple should “unlock the iPhone,” a broad term that could have any number of meanings. Nowhere in the survey is encryption, or what other data might be compromised, mentioned. Pew did also ask a question about how much their respondents had heard about the case:

61% knew a little, or less than a little, about the case. On top of that (and hopefully without casting aspersions too broadly on the self-awareness of Pew’s sample) we have no idea what “a lot” meant to the other respondents.

Polls are, by necessity, short. You’re trying to a randomized set of people to give you their time and information. The encryption debate going on right now is very, very complicated. In the meantime, the FBI has been rather vague, at least in public, with its goals. FBI Director James Comey wrote the following soothing paragraph about the Bureau’s intentions:

We simply want the chance, with a search warrant, to try to guess the terrorist’s passcode without the phone essentially self-destructing and without it taking a decade to guess correctly. That’s it. We don’t want to break anyone’s encryption or set a master key loose on the land.

As Wired‘s Kim Zetter has pointed out, it’s possible that Apple could do all the FBI is asking and the information on the phone would remain inaccessible. In the meantime, it’d still involve building software that could be used to try brute force attempts to unlock other phones. Privacy advocates are already concerned about public understanding of this, especially in light of the new Pew survey.

Plus, the FBI’s intentions aren’t the only ones that matter. Other law enforcement officials are interested in the case, and some, like Manhattan District Attorney Cyrus Vance, Jr. have already said that if the FBI can get access to this phone they’d like access to other phones.

So, when you see chyrons and headlines blaring that American people support the FBI in their attempt to have Apple unlock this phone, remember that unlocking isn’t all there is to the situation.

On the plus side, maybe (43% of) the children are our future?

Update: Pew has responded to this post, expressing that they disagree with the characterization of the survey as flawed. Here’s their response :

The wording we chose for this question reflects our best judgment about the clearest way to ask about the conflict between Apple and the Justice Department in a public opinion telephone survey. As you point out, polls must concisely use plain language to summarize sometimes complex issues. We disagree the characterization of the survey as flawed as it was designed to best measure the public’s general sentiment on this issue, without sacrificing data quality by the use of longer, more complex language that is not easily understood by all segments of the public.

This question, of course, is just one measure of opinion on the issue; others may well shed additional light on the topic.

While, as I already noted, the survey is interesting and does assess public sentiment, I still believe there’s a problem with it. By phrasing the questions in terms of “unlocking” the phone the poll doesn’t just simplify the complex issues at hand, it mischaracterizes them by describing a different process that is more widely understood. That’s the flaw I was referring to, and it remains, to my mind, a serious one.

Ethan Chiel is a reporter for Fusion, writing mostly about the internet and technology. You can (and should) email him at [email protected]