AI summary by TwinMind

Human Stories in AI: Rick Marks

StatQuest with Josh Starmer

31m 13s

Origin Story & Early Motivation

  • Started in high school with social insecurity augmentation project
    • Attempted program to suggest perfect jokes/quotes for social situations
    • Would be delivered via earpiece for real-time conversation help
    • Abandoned as infeasible at the time but concept persisted throughout career
  • Repurposed code for senior/junior year science project
    • Built robot with computer displaying different personalities
    • Could switch between joke personality and famous quote personality
    • Voice changed based on personality mode

Academic Journey & Career Pivot

  • PhD thesis originally focused on control theory for underwater robotics
    • Worked with Monterey Bay Aquarium Research Institute
    • Needed computer vision for robot sensing capabilities
    • Used camera and frame grabber board for simple target following
  • Switched from control theory to computer vision focus
    • Colleagues convinced him computer vision should be his thesis
    • "Eventually saw the light and pivoted"
  • Dissertation work included:
    • Fish following system using plastic turtle as target
    • Pan and tilt tracking system
    • Ocean floor mosaicing - stitching multiple photographs together
    • Mathematical model for marine snow (particulates in water)
  • Worked in Soquel Canyon off California coast
    • Deep underwater canyon allowing quick access to deep water
    • No ambient light - robots must provide own illumination
    • Marine snow particles reflect light, creating visibility challenges

Industry Experience Progression

  • Started at computer vision startup after PhD
  • Moved to Sony for video game computer vision applications
    • Used robots for testing PlayStation Move controller
    • Worked on VR input controller development
    • Performed repeated sensor testing with robotic systems
  • Transitioned to Google's Advanced Technology and Projects group
    • Focused on ambient computing environments
    • Worked on new interaction approaches
  • Founded PlayStation Magic Lab at PlayStation R&D
    • Combined graphics, virtual reality, and AI interests
    • Developed expertise in 3D graphics and VR

Smart Speaker Context Problem

  • Gave father smart speaker as gift - father wanted to return it
  • Father asked "when is golf" - device couldn't answer without context
    • With golf cleats and bag: likely asking about tee time
    • While channel surfing: likely asking when Masters tournament airs
  • Realized context is key to effective AI assistance
  • Led to Google work on context understanding and gesture recognition
  • Worked on real-time inference and machine learning applications

Current Academic Position at UNC Chapel Hill

  • Professor in School of Data Science and Society
  • Joint appointment with Computer Science department
  • Started teaching CS classes since Data Science school newly formed
  • Plans to teach data science classes as school develops

Teaching: Introduction to VR and 3D Graphics Course

  • Covers mathematical foundations: projection, transforms, linear algebra
  • Students initially surprised by math requirements
  • Uses Unity package for practical implementation
  • Progression: math foundations → Unity basics → VR experience development
  • Student final projects (2-week, 2-person teams):
    • Magic canvas project: voice-to-text → generative AI → VR art creation
    • Originally planned 3D model generation but pivoted due to latency (1 minute vs few seconds for images)

Generative AI Research Interests

  • Combining AI with virtual reality environments
  • Dynamic content creation based on user desires/context
  • Addressing latency issues in real-time applications
    • Text interfaces tolerate delays better than voice
    • Real-time situations require immediate responses
  • Proposed solutions using storytelling/game mechanics:
    • Robot character searches through boxes while AI generates content
    • Shakes boxes, says "no," continues searching until content ready
    • Integrates waiting time into narrative experience

Future of Storytelling Conference

  • Annual conference in New York
  • Unique format: attendees take ferry to isolated location
  • Combines storytellers, technologists, artists
  • Small, exclusive attendance
  • Explores intersection of technology and narrative

Fantasy-to-Reality VR Concept

  • Inspired by fantasy book series with reality manipulation
  • Characters think/speak desires and reality changes accordingly
  • Potentially implementable with current generative AI + VR technology
  • Users could verbally describe desired environments and watch them form

Career Philosophy & Life Plan

  • Decided at age 16 on three-phase life plan:
    • 25 years learning (through PhD)
    • 25 years industry experience
    • 25 years teaching/sharing knowledge
  • Self-described as "not a planner" except for this one major life decision
  • Framed as learning → practice → service progression

Why UNC Chapel Hill

  • One of birthplaces of VR technology
  • Met Henry Fuchs and Fred Brooks during visits
  • Strong VR legacy influenced decision
  • Welcoming faculty community with lunch invitations and events
  • Data Science school offered flexibility vs traditional CS department constraints
  • Wanted to leave California for different environment
  • Connected through industry friends and conference relationships

Industry-to-Academia Transition Challenges

  • Teaching preparation more challenging than expected
  • Still learning effective teaching methods
  • Advantage: can share real-world product examples with students
  • Video games particularly relatable for student engagement
  • Consumer products easier to explain than abstract business applications

Hiring & Career Advice

  • Advocates for "T-shaped" professionals:
    • Deep expertise in one area (the vertical stroke)
    • Broad knowledge across multiple disciplines (the horizontal stroke)
  • Avoid being too narrow (PhD risk) or too broad without specialty
  • Ability to communicate across disciplines crucial
  • Each field has its own language for describing similar concepts
  • Deep contribution + team collaboration ability = ideal combination
  • Students often ask about marketability and study recommendations

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome