Apple AR: A New Patent Shows How Apple Is Tackling AR

An Apple patent deposited on October 26th might show how Apple is moving in the direction of AR.

As the abstract explains:

A direct retinal projector may include a gaze tracking system that tracks position of a subject’s pupil and automatically adjusts projection of a scanned light field so that the light field enters the pupil. A control loop adjusts a scanning mirror to substantially center an IR beam on a position sensing detector (PSD). In so doing, the scanning mirror is correctly positioned so that the scanned light field from the projector enters the subject’s pupil. In addition, a direct retinal projector may include an adjustable focusing element that adjusts focus of a combined light beam generated by a projector as the light beam is scanned to an ellipsoid mirror that reflects the light beam to the subject’s pupil. The focusing of the scanned beam may be adjusted as the beam is scanned across the azimuth angle of the curved ellipsoid mirror.

Various parts of the patent also explain how 3D images might be potentially shown into the retina:


This matters a lot, as it shows how Apple might be developing its own AR devices. As highlighted in the patent:

conventional virtual reality and augmented reality systems may suffer from accommodation-convergence mismatch problems that cause eyestrain, headaches, and/or nausea. Accommodation-convergence mismatch arises when a VR or AR system effectively confuses the brain of a user by generating scene content that does not match the depth expected by the brain based on the stereo convergence of the two eyes of the user. For example, in a stereoscopic system the images displayed to the user may trick the eye(s) into focusing at a far distance while an image is physically being displayed at a closer distance. In other words, the eyes may be attempting to focus on a different image plane or focal depth compared to the focal depth of the projected image, thereby leading to eyestrain and/or increasing mental stress. Accommodation-convergence mismatch problems are undesirable and may distract users or otherwise detract from their enjoyment and endurance levels (i.e. tolerance) of virtual reality or augmented reality environments.

Apple also explains how the mechanism might work:

In embodiments of a direct retinal projector system for AR and/or VR applications as described herein, a light beam is generated by a scanning projector, reflected off a curved mirror (e.g., a curved ellipsoid mirror) in front of the subject’s eye and through the subject’s pupil, and forms an image on the subject’s retina–there is no intermediate image on a screen or surface that the subject views. In some embodiments, with relatively small diameter laser beams, the effective depth of focus of the eye can be greatly increased. The direct retinal projector system may at least partially eliminate eye lens accommodation from the retinal projection focus to help eliminate the accommodation convergence mismatch. In some embodiments, the direct retinal projector system may help compensate for user eye lens problems, such as short- or long-sightedness.

As we explain in Apple business strategy this might be the next key hardware to develop a business platform:

Scroll to Top