AI summary by TwinMind

Google I/O Afterparty: The Future of Human-AI Collaboration, From Veo to Mariner

Sequoia Capital

53m 51s

Google's AI Momentum and Public Opinion Shift

  • Public opinion on Google's AI position flipped dramatically after Google I/O
  • Models are primary driver of perception change
  • Google now leads many AI leaderboards with state-of-the-art capabilities
  • Internal feeling: end of chapter one, start of chapter 2 in AI development
  • Three years of consistent improvement in Gen AI finally visible externally

Whisk and Flow: Creative Video Generation (Thomas Iljik)

Development History

  • Started 2.5-3 years ago with control net paper enabling model channeling
  • LoRAs allowed showing model range of possibilities for iteration
  • Trained internal teams on ComfyUI-type workflows
  • Created half-episode animation "Not So Super Villain" on YouTube
  • Key convictions: creation must be iterative, media comes with blueprint, show-and-tell approach

Product Strategy

  • Whisk: consumer-focused visual language tool for remixing
  • Flow: high-end filmmaking tool - "generative AI camera" for AI filmmakers
  • Target users: AI filmmakers (not traditional filmmakers) - people without $100K budgets
  • Flow designed for world-building then shooting inside created worlds

Technical Progress

  • VO3 beat VO2 in rankings, major improvement in adherence
  • Physics getting better, six-finger problem resolved
  • Audio co-generation with video launched - major breakthrough
  • Challenges remain: multiple character choreography, cross-scene consistency, output refinement

Future Vision

  • Video generation, simulation, games converging into same technology
  • World-building approach: define stage, assets, appearance, then shoot
  • Show-and-tell interface preferred over pure text prompts
  • New formats between static images/videos and full games
  • Sharing experiences rather than just content

Mariner: Computer Use Agent (Jaclyn Consulman)

Technical Architecture

  • Action-tuned model on recent Gemini version
  • Leverages multimodal capabilities for planning and reasoning
  • Takes screenshots, understands UI, continues trajectory toward user goals
  • Moved from Chrome extension to VM-based background execution

Development Timeline

  • Project started last year after Google AI Studio/Gemini API graduated from Labs
  • December launch as Chrome extension with trusted testers
  • Feedback: "cool but I want my browser back"
  • Recent launch: web app with virtual machines for background tasks

User Experience Evolution

  • Companion extension provides context from open browser tabs
  • Can handle 10 concurrent tasks vs single task limitation
  • Users can watch agent in full-screen mode or run in background
  • Task summaries provided instead of full conversation logs
  • Pause/takeover functionality for user oversight

Current Capabilities and Limitations

  • Research prototype with experiment label
  • Major improvements from December to current version
  • Speed optimization ongoing priority
  • Model quality improvements needed
  • Application-level improvements for seamless context provision
  • Planning and reasoning enhancements required

Future Development

  • Three categories: agent intelligence, environment expansion, ecosystem integration
  • Agent: smarter models, tool use, memory, context
  • Environment: local desktop background operation, cross-device capability
  • Ecosystem: agent-to-agent interaction, business model evolution

Notebook LM: Personalized AI Content (Simon Takamina)

Viral Success and Evolution

  • Audio overviews went viral unexpectedly
  • "Audience of one" content hypothesis - personal rather than mass content
  • First months focused on infrastructure scaling (TPU capacity)
  • Join mode launched end of last year for podcast interaction

Technical Infrastructure Upgrade

  • Moved from research-grade model to native Gemini audio
  • International audio overviews launched
  • Mobile applications launched Monday
  • Collaboration across Labs and GDM teams

Core User Patterns

  • Longer-running projects in hobbies and work
  • Knowledge workers and students as primary users
  • Projects as units of knowledge accumulation
  • Context criticality for AI system usefulness

Three Notebook Superpowers

  • Information accumulation over time (database technologies)
  • Bundled intelligence (Gemini 1.5 Pro, thinking models)
  • Adaptive content transformation (audio, visual, interactive formats)

Content Format Innovation

  • Beyond two-host podcast format
  • LinkedIn feedback analysis use case
  • Different show types in development
  • Comic book, mind map, short movie adaptations
  • 150-page PhD dissertation → comic book example

Mobile-First Innovation

  • Desktop-mobile companion experience rather than carbon copy
  • Native voice recording for conversations
  • Leveraging mobile form factor, sensors, always-on presence
  • Novel mobile experience development priority

Cross-Labs Collaboration and Future Predictions

Favorite Unreleased Projects

  • Virtual try-on for shopping (Thomas, Simon)
  • Stitch for front-end design generation (Jaclyn)

2025 Predictions

  • Video as breakout application
  • Remixable content generation and sharing
  • Show-and-tell interface paradigm adoption

Lessons Learned

  • Timing consistently challenging - often too early
  • Show-and-tell over pure text prompting validated
  • Sticking with LLM technology when business viability questioned
  • Inference cost curves enabling new possibilities
  • Two-year forward planning approach in Labs

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome