Live Text and Visual Lookup on an Apple Device

We are really excited by a new feature called live text and visual lookup on Apple devices. It is available in Mac OS Monterey and iOS 15.

This feature allows you to grab text out of a photo, or using your camera. This text can then be pasted into an email, a note or a document. The text recognition isn’t perfect, but it is pretty darn good!

SDM helped a client use this technique to recover text that he only had on printed paper. First he took pictures of the pages. Then he used visual lookup on each picture to recognize the text. He transferred this text into a Word document and recreated his work in about 10 minutes.

Visual lookup on an iPhone

Take a picture of the text you want to capture. Open that picture in Photos on either your Mac or your iPhone or iPad.

If text is recognized in the photo, you will see a grey icon in the bottom right corner as shown above.

On a Mac, move your cursor to the bottom right corner of the window and this icon will appear.

Click on the icon and all recognized text will be highlighted.

Select the portion of the recognized text that you want. Click on it and select COPY.

Go to the app you want to put the text into. We are using Notes here. Click PASTE.

Live text on an iPhone

This process also works without taking the picture beforehand.

Go to Notes. Open a new Note. Click or Tap in the body of the note.

Select the text icon in the center. This will open the camera, and if you hold it still, it will recognize text and show it in the note in real time.

Click insert to finish the process.

We used this technique to recognize the text on business’ awnings and windows. We tried it on graffiti and handwritten text with mixed results. When used on typed text, it was near perfect.

Here is another description of the live text and visual lookup on Apple devices feature in MacWorld. If you would like help with this process, just call or email and we’ll be happy to assist you! If you have specific questions, or if our terminology isn’t clear, please use our contact form to send us a message. Thank you!