AI summary by TwinMind

WWDC 2024 Recap: Is Apple Intelligence Legit?

Marques Brownlee

18m 23s

WWDC 2024 Overview

  • Potentially biggest Apple WWDC keynote due to 2024 AI expectations
  • Apple structured presentation with AI in second half only
  • First hour: zero mentions of "AI"
  • Second hour: extensive AI feature announcements
  • AI rebranded as "Apple Intelligence"

VisionOS 2.0 Updates

  • Released less than 6 months after Vision Pro announcement
  • More like 1.2 update than full 2.0
  • New wrist turn gesture: quick access to time, battery percentage, control center
    • Eliminates need to look up at ceiling
    • Similar to Oculus Quest functionality
  • Immersive media features:
    • Machine learning converts old photos to spatial photos
    • New tools for creating spatial/immersive videos on non-iPhone cameras
    • Content only viewable on Vision Pro
  • Mac mirroring improvements:
    • Foveated rendering now handled by Mac
    • Up to double resolution capability
    • Supports gigantic ultra-wide displays
  • Additional updates: new environments, home screen rearrangement capability

iOS 18 Major Updates

  • One of biggest/most interesting updates in long time
  • Home screen customization:
    • Icons can be placed anywhere on grid (finally)
    • Got applause during keynote presentation
    • New theme engine for color matching
    • Can change all icon/widget colors to match wallpaper
    • Some apps support color theming well, others apply poor color casts
    • Can create attractive or terrible-looking home screens
  • Control Center redesign:
    • Fully customizable across multiple pages
    • More customizable shortcuts available
    • Can replace flashlight/camera shortcuts without jailbreaking
  • Additional features:
    • Hidden apps in app drawer with locked folder
    • Scheduled text messages in iMessage
    • Text formatting capabilities
    • Messages via satellite
    • RCS support (briefly mentioned)
    • Photos app redesign
    • Game mode: minimizes background activity, reduces Bluetooth latency
    • Mail app automatic categorization

AirPods Updates

  • Voice isolation during phone calls
  • Spatial audio during games
  • Head gesture responses (nodding/shaking for prompts)
    • Similar to Sony headphones feature
    • Voice-free control option

Apple TV Features

  • Real-time content identification:
    • Swipe down on remote during shows
    • Shows all actors, character names on screen
    • Displays currently playing songs
  • Voice boosting feature for dialogue over music

watchOS Updates

  • Training mode balances workout trends over time
  • Evaluates actual workout intensity vs potential intensity

iPadOS 18 Features

  • All iPhone customization features included
  • SharePlay remote control: allows remote iPad control when permitted
    • Massive improvement for family tech support
  • Calculator app with Math Notes:
    • Handwriting recognition for equations
    • Answers provided in user's handwriting style
    • Real-time answer updates when equations modified
    • Variable support across entire page
    • Graphing capabilities with real-time variable adjustment
    • Also coming to iPhone Calculator app

macOS Sequoia Updates

  • Automatic window snapping: built-in functionality (third-party apps no longer needed)
  • Built-in webcam backgrounds for all apps
  • New Password app: standalone app for previously buried settings features
    • Available on Mac, iPad, iPhone
  • iPhone Mirroring:
    • Wireless iPhone display on Mac
    • Merged notifications and audio
    • Drag-and-drop between devices
    • Potential battery impact concerns

Apple Intelligence (AI Features)

  • Device compatibility: iPhone 15 Pro, iPad with M1+, Mac with M1+
  • Processing approach:
    • Most features run on-device using Apple-built models
    • Complex tasks use Apple's Private Cloud Compute (Apple silicon servers)
    • ChatGPT integration requires explicit user permission per request
  • Writing tools: available anywhere with cursor
    • Summarize, rewrite, style changes, proofreading
  • Clean Up tool: Magic Eraser equivalent in Photos app
    • Background item identification and removal
    • Generative fill capability
  • Genmojis: custom emoji generation using diffusion models
  • Image Playground:
    • Built into apps + standalone app
    • Three styles: sketch, illustration, animation
  • Siri improvements:
    • Better context understanding via large language models
    • Future: app information pulling and action execution
    • New full-screen animation
    • Updated voice
    • Type-to-Siri option
  • ChatGPT-4.0 integration:
    • Requires explicit user permission per request
    • Free without account requirement
    • No request storage by OpenAI
    • IP address obscuring for privacy
    • Access to advanced capabilities beyond Apple's models

Privacy and Security Architecture

  • On-device processing: majority of Apple Intelligence features
  • Private Cloud Compute: Apple silicon servers, no data storage/transmission to Apple
  • ChatGPT integration: explicit consent required, no profile building, IP address obscured

Overall Assessment

  • Humane Pin and Rabbit R1 rendered obsolete by phone-based AI integration
  • Apple entering new AI-focused era
  • AI features overshadowing traditional updates
  • Home screen customization potentially biggest non-AI feature
  • Apple Intelligence positioning as primary differentiator going forward

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome