Apple isn’t spying on your photos in iOS 18 with Enhanced Visual Search

The Photos app overhaul in iOS 18 has been met with mixed approval. We have a set of tips for how to improve iOS 18 Photos by customizing the interface and layout, but the internet seems to have caught on to a new Enhanced Visual Search feature and is worried about the privacy implications.

Starting in iOS 16, a feature called Visual Look Up will identify common objects, landmarks, plants, animals, and even icons and labels in your photos. When you look at an image in your Photos library, the info button (i) will get a little sparkle on it if Visual Lookup can provide further info. It might even change from the letter (i) to an icon of an animal or leaf. The feature got better in iOS 17, and is improved even more in iOS 18.

But there’s only so much your iPhone can do on-device. There just no way to match any of your infinite photo variety to the thousands of popular landmarks and points of interest in the world—the database would fill up all your iPhone storage and then some.

So to make Visual Look Up better at identifying common landmarks and points of interest, Apple will use the cloud to match places in your photos to a big points-of-interest database it maintains. This, of course, requires your photo (or at least, specific markers and data from them) leaving your iPhone. Here’s what the privacy implications are and how to disable this feature if you want to.

Enhanced Visual Search privacy

In Apple’s legal disclaimers about Photos & Privacy, it says:

Enhanced Visual Search in Photos allows you to search for photos using landmarks or points of interest. Your device privately matches places in your photos to a global index Apple maintains on our servers. We apply homomorphic encryption and differential privacy, and use an OHTTP relay that hides IP address. This prevents Apple from learning about the information in your photos.

Note that this is just about matching places, not faces of people or pets (or other animals and plants), which all still happens on-device.

Just what does all that technical jargon mean? Well, “homomorphic encryption” means encryption that allows operations to be done on the data while it’s still in its encrypted state. In other words, your image is encrypted and then Apple matches the data to its database of landmarks and points of interest without ever decrypting it into a visually recognizable image. Nobody at Apple can just access some server and look at your photos.

Differential Privacy is a means of protecting individual data when it’s used in group analysis. You might have your anonymized data as part of a bunch of data sets, and nobody can identify you when looking at just one of them, but if you combine data from multiple sets together you can pinpoint an individual person. Differential Privacy is a way to use several mathematical methods to protect users from this.

And an OHTTP relay is an “Oblivious HTTP Relay.” It’s a trusted internet relay that takes encrypted requests and forwards them along to a specific location (in this case, to Apple’s photo analyzing servers) without sending along any identifying information like IP addresses, device IDs, and so on. In other words, it separates the “what” of a request from the “who” of the request. So Apple has no way of knowing from which one of its users any individual photo has come, nor would any hacker who got access to Apple’s Enhanced Visual Lookup server.

In other words, Apple is going to great lengths to have no ability to know what your photos are or where they came from. This protects your privacy, but it also protects Apple from liability or surveillance requests from government entities.

How to disable Enhanced Visual Search

If you’re still worried about privacy in Photos, even with all that encryption and obscuring identifiable information, you can disable Enhanced Visual Search easily enough. Visual Look Up will still work, it just won’t be as good at identifying things like landmarks or points of interest in your photos.

On iPhone or iPad:

• Open Settings
• Scroll to the bottom and tap on Apps.
• Select Photos.
• You’ll find the toggle for Enhanced Visual Search at the bottom of the Photos settings. Tap once to turn it off.

On Mac, you’ll find it in the settings of the Photos app itself:

• Open Photos
• In the Photos menu, select Settings (or type Command-comma)
• Select the General tab
• You’ll find a checkbox for Enhanced Visual Search at the bottom of the General tab. Click once to turn it off.

Foundry