AI summary by TwinMind
Using Apple Vision Pro: What It’s Actually Like!
Marques Brownlee
37m 19s
Initial Hardware Overview
- Apple Vision Pro: High-end VR headset (though Apple calls it "spatial computing")
- Precisely machined aluminum frame with metal and glass construction
- Weight: 638g with solo knit band (slightly less than Meta Quest Pro but front-weighted)
- External battery design: 3,366 mAh battery with non-removable 4ft cable
- Proprietary twist-lock connector prevents easy disconnection
- No on/off button - only way to turn off is unplugging battery cable
- Battery life: 2-4 hours realistic usage (can extend via USB-C port on battery)
Physical Design and Components
- Fan intakes at bottom, vents at top for cooling
- Digital crown (pressable/turnable) and single button (similar to Apple Watch)
- Downward-facing speaker pods with spatial audio (some sound bleed to others)
- Large front glass panel (easily fingerprinted) with outward-facing OLED display
- External sensors: depth sensors, infrared illuminators, LiDAR scanners, RGB cameras
- Internal processors: M2 chip and R1 chip
- Internal eye-tracking sensors for real-time eye control and external eye display
Setup and Calibration Process
- Initial setup adjusts lens distance to match user's eye spacing
- Hand scan to understand user's hands
- Eye calibration: looking at dots around screen and pinching fingers to select
- Process feels like eye test, establishes eye-tracking baseline
Input Methods and Control System
- Primary input: eyes and hands (no physical controllers included)
- Pinching fingers together to select while looking at target
- Wide sensor coverage allows hand control anywhere in 180° bubble in front of user
- Digital crown brings up app drawer
- Control center accessed by physically looking up and selecting arrow above
Text Input Options
- Hunt-and-peck virtual keyboard (pointer finger only, slow)
- Look-and-pinch method (faster, look at keys and pinch to select)
- Voice input for URLs in Safari
Vision OS Interface
- Similar to iPadOS but with glassy, frosted glass aesthetic
- 3D spatial positioning with incredible lock-in-place accuracy
- Windows float in space with precise positioning regardless of head movement
- App windows can be dragged in X, Y, Z space and resized from corners
- Challenge: must look exactly at what you want to control (different from traditional UIs)
Pass-Through Technology
- Best pass-through of any VR headset tested
- R1 chip processes camera feeds with <12ms latency
- 90Hz refresh rate with impressive dynamic range
- Near real-time response allows catching objects, playing ping-pong
- Limitations: close-up items blur, can't read fine text on phone
- Can use phone notifications and texting while wearing headset
Immersive Environments
- Digital crown rotates to dial between pass-through and full immersion
- Built-in environments: California locations, Mount Hood with rain
- Not quite photorealistic but close to photorealistic quality
- Most realistic digital environments seen in VR
App Ecosystem
- Two types: Native Vision Pro apps and compatible iPhone/iPad apps
- Native apps take advantage of 3D space and Vision Pro features
- Apple stock apps: Music, Notes, Settings with glassy redesign
- Media apps: Apple TV, Disney+ with immersive environments
- Apple TV has 180° proprietary format videos (e.g., Alicia Keys performance)
- Photos app: panoramic viewing with parallax effects
Third-Party Native Apps
- Sky Guide: Interactive constellation viewing and information
- JigSpace: 3D model manipulation and exploration in real space
- Keynote: Presentation practice in Steve Jobs Theater or conference room environments
Compatible Apps and Limitations
- Pre-installed "Compatible Apps" folder with iPad-like functionality
- Notable absences: Netflix, YouTube, Spotify (developers opted out)
- Can use browser versions (Safari) for missing apps like YouTube
- Missing dedicated app features like offline video download
Mac Integration
- Standout feature: Mac virtual display connection
- Instantly turns Mac display into 4K window inside headset
- Keyboard/trackpad continue working seamlessly
- Can resize and position virtual Mac display anywhere in space
- Can open additional Vision Pro apps around Mac interface
- Limitation: Only one virtual monitor (no dual display support)
- Ideal for airplane use and private work sessions
Personas and Eye Display Technology
- External eye display shows reconstructed version of user's eyes
- 90fps eye tracking enables Optic ID security system
- Persona creation: 3D face scanning process capturing expressions
- Eyes visible in pass-through mode, covered with blue/purple glow in immersion
- Breakthrough feature during conversations: person appears through immersive fog
- Lenticular film creates 3D depth effect for external eye display
Persona Creation Process
- Face scanning from multiple angles while holding headset
- Captures various facial expressions (smiles, eyebrow raises, etc.)
- Customizable lighting, skin tone, brightness settings
- Can add virtual glasses for users who normally wear them
- Results vary by skin tone - darker skin tones show less brightly on external display
FaceTime Experience
- Most impressive Vision Pro feature
- Persona used as camera feed for video calls
- Spatial positioning: eye contact direction matches viewing angle
- Hand gestures tracked and displayed at correct angles to specific participants
- Advanced spatial audio: voices come from correct directions, distance affects volume
- Environmental audio changes based on virtual space selection
- Best avatar technology seen but still in uncanny valley
- Main weakness: frozen hair and accessories from scan moment
Performance Assessment
Strengths
- Incredible immersion and spatial positioning lock
- Best-in-class eye tracking and hand control
- Superior pass-through technology
- Strong ecosystem integration
- Excellent spatial audio implementation
Weaknesses
- Weight and comfort issues (front-heavy design)
- Limited external eye visibility (especially darker skin tones)
- Restricted app selection at launch
- Short battery life (2-4 hours)
- High price point ($3,500-4,000)
- Personas still in uncanny valley territory
