AI summary by TwinMind

Using Apple Vision Pro: What It’s Actually Like!

Marques Brownlee

37m 19s

Initial Hardware Overview

  • Apple Vision Pro: High-end VR headset (though Apple calls it "spatial computing")
  • Precisely machined aluminum frame with metal and glass construction
  • Weight: 638g with solo knit band (slightly less than Meta Quest Pro but front-weighted)
  • External battery design: 3,366 mAh battery with non-removable 4ft cable
  • Proprietary twist-lock connector prevents easy disconnection
  • No on/off button - only way to turn off is unplugging battery cable
  • Battery life: 2-4 hours realistic usage (can extend via USB-C port on battery)

Physical Design and Components

  • Fan intakes at bottom, vents at top for cooling
  • Digital crown (pressable/turnable) and single button (similar to Apple Watch)
  • Downward-facing speaker pods with spatial audio (some sound bleed to others)
  • Large front glass panel (easily fingerprinted) with outward-facing OLED display
  • External sensors: depth sensors, infrared illuminators, LiDAR scanners, RGB cameras
  • Internal processors: M2 chip and R1 chip
  • Internal eye-tracking sensors for real-time eye control and external eye display

Setup and Calibration Process

  • Initial setup adjusts lens distance to match user's eye spacing
  • Hand scan to understand user's hands
  • Eye calibration: looking at dots around screen and pinching fingers to select
  • Process feels like eye test, establishes eye-tracking baseline

Input Methods and Control System

  • Primary input: eyes and hands (no physical controllers included)
  • Pinching fingers together to select while looking at target
  • Wide sensor coverage allows hand control anywhere in 180° bubble in front of user
  • Digital crown brings up app drawer
  • Control center accessed by physically looking up and selecting arrow above

Text Input Options

  • Hunt-and-peck virtual keyboard (pointer finger only, slow)
  • Look-and-pinch method (faster, look at keys and pinch to select)
  • Voice input for URLs in Safari

Vision OS Interface

  • Similar to iPadOS but with glassy, frosted glass aesthetic
  • 3D spatial positioning with incredible lock-in-place accuracy
  • Windows float in space with precise positioning regardless of head movement
  • App windows can be dragged in X, Y, Z space and resized from corners
  • Challenge: must look exactly at what you want to control (different from traditional UIs)

Pass-Through Technology

  • Best pass-through of any VR headset tested
  • R1 chip processes camera feeds with <12ms latency
  • 90Hz refresh rate with impressive dynamic range
  • Near real-time response allows catching objects, playing ping-pong
  • Limitations: close-up items blur, can't read fine text on phone
  • Can use phone notifications and texting while wearing headset

Immersive Environments

  • Digital crown rotates to dial between pass-through and full immersion
  • Built-in environments: California locations, Mount Hood with rain
  • Not quite photorealistic but close to photorealistic quality
  • Most realistic digital environments seen in VR

App Ecosystem

  • Two types: Native Vision Pro apps and compatible iPhone/iPad apps
  • Native apps take advantage of 3D space and Vision Pro features
  • Apple stock apps: Music, Notes, Settings with glassy redesign
  • Media apps: Apple TV, Disney+ with immersive environments
  • Apple TV has 180° proprietary format videos (e.g., Alicia Keys performance)
  • Photos app: panoramic viewing with parallax effects

Third-Party Native Apps

  • Sky Guide: Interactive constellation viewing and information
  • JigSpace: 3D model manipulation and exploration in real space
  • Keynote: Presentation practice in Steve Jobs Theater or conference room environments

Compatible Apps and Limitations

  • Pre-installed "Compatible Apps" folder with iPad-like functionality
  • Notable absences: Netflix, YouTube, Spotify (developers opted out)
  • Can use browser versions (Safari) for missing apps like YouTube
  • Missing dedicated app features like offline video download

Mac Integration

  • Standout feature: Mac virtual display connection
  • Instantly turns Mac display into 4K window inside headset
  • Keyboard/trackpad continue working seamlessly
  • Can resize and position virtual Mac display anywhere in space
  • Can open additional Vision Pro apps around Mac interface
  • Limitation: Only one virtual monitor (no dual display support)
  • Ideal for airplane use and private work sessions

Personas and Eye Display Technology

  • External eye display shows reconstructed version of user's eyes
  • 90fps eye tracking enables Optic ID security system
  • Persona creation: 3D face scanning process capturing expressions
  • Eyes visible in pass-through mode, covered with blue/purple glow in immersion
  • Breakthrough feature during conversations: person appears through immersive fog
  • Lenticular film creates 3D depth effect for external eye display

Persona Creation Process

  • Face scanning from multiple angles while holding headset
  • Captures various facial expressions (smiles, eyebrow raises, etc.)
  • Customizable lighting, skin tone, brightness settings
  • Can add virtual glasses for users who normally wear them
  • Results vary by skin tone - darker skin tones show less brightly on external display

FaceTime Experience

  • Most impressive Vision Pro feature
  • Persona used as camera feed for video calls
  • Spatial positioning: eye contact direction matches viewing angle
  • Hand gestures tracked and displayed at correct angles to specific participants
  • Advanced spatial audio: voices come from correct directions, distance affects volume
  • Environmental audio changes based on virtual space selection
  • Best avatar technology seen but still in uncanny valley
  • Main weakness: frozen hair and accessories from scan moment

Performance Assessment

Strengths

  • Incredible immersion and spatial positioning lock
  • Best-in-class eye tracking and hand control
  • Superior pass-through technology
  • Strong ecosystem integration
  • Excellent spatial audio implementation

Weaknesses

  • Weight and comfort issues (front-heavy design)
  • Limited external eye visibility (especially darker skin tones)
  • Restricted app selection at launch
  • Short battery life (2-4 hours)
  • High price point ($3,500-4,000)
  • Personas still in uncanny valley territory

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome