How to use Visual Intelligence, Apple’s use of Google Lens

U recent implementation of iOS 18.2 finally brings many of the promised features of Apple Intelligence, such as Genmoji and Image Playground. Such a long-awaited tool is Visual intelligencea feature currently reserved for the iPhone 16 Pro and Pro Max that was first introduced to the company’s September event.

What is visual intelligence?

Visual Intelligence is Apple’s answer to Google Lens. Take advantage of the camera system and AI for analyze images in real time and provide useful information. This can help people learn more about the world around them and is particularly useful for shopping, looking for details about a restaurant or business, translating written text, summarizing text or having something read aloud. It can also integrate with Google Image Search and ChatGPT.

Are there any caveats?

There are two caveats. The rollout of Apple Intelligence has been something of a convoluted mess, and this trend continues with Visual Intelligence. For now, the tool only works with the iPhone 16 Pro and Pro Max, which are the the beefiest of the company’s recent phones. Apple indicated that the feature could eventually be available for older models. Google Lens, after all, has been since 2017that was when the Pixel 2 was the hottest phone on the block.

There is also a waiting list, which is true of all Apple Intelligence functions. To join the list, go to settings and search for “Apple Intelligence & Siri”. Then click on “Join Waitlist”. Once approved, the software will be ready to use.

How to use visual intelligence

As of this writing, the only way to launch Visual Intelligence is to long press the Camera Control button. It is the new control interface on the bottom right side of the phone. Once pressed, the Visual Intelligence interface will open.

Now the fun begins. Just point your phone at something and select ChatGPT, through the bottom left icon, or Google Image Search, through the bottom right icon. Alternatively, if the field of view includes text, tap the circle at the bottom of the screen. The phone can also be pointed to a business to get useful information.

How to interact with the text

Hold the phone in front of the text, activate Visual Intelligence and tap the circle at the bottom of the screen. This analyzes the text. Once analyzed, there are several options. Tap “Translate” at the bottom of the screen to translate the text into another language. Tap “Read Aloud” if you want the text to be read aloud by Siri. Tap “Summarize” for a quick summary of the copy.

The tool will also identify contact information in the text, such as phone numbers, email addresses and websites. Users can act according to the type of text. For example, tap the phone number to make a sound. Other actions include starting an email, creating a calendar event or going to a website. Tap the “More” button to see all available options. Tap “Close” or swipe up to end the session.

How to interact with a business

Visual intelligence can provide details about a business that is directly in front of you. Just open the tool and point the camera in front of the signage. The business name should appear at the top of the screen. Tap “Schedule” to see hours of operation or tap “Order” to buy something. See the menu or services available by touching “Menu” and make a reservation by touching “Reservation”. To call the business, read reviews or view the website, tap “More”.

Swipe up or tap “Close” to end the session. This feature is currently only available to US customers.

What to do with ChatGPT

Start by pointing the camera at an object. Activate Visual Intelligence and tap the ChatGPT icon on the bottom left side of the screen. Tap the “Ask” button for information about the object. We used it on a bottle of hand cream, which it identified well. After that, a text field will appear for follow-up questions. Users can ask what they want, but the results may vary. We asked ChatGPT where to buy the hand cream and how much it costs. He did admirably at this task. Yes shopping.

Tap the “Close” button or swipe up to clear all fields, which also closes Visual Intelligence.

What to do with Google Image Search

Selecting Google Image search will bring up a Safari dialog box containing similar photos pulled from the web. A good use case here is to find offers. We took a photo of a bottle of hand cream and the Safari results had many different price points to choose from. However, users have to find the best deal and complete a purchase on their own.

Tap the “Close” button to remove these results and then swipe up from the bottom of the screen to close the tool.