AI summary by TwinMind
Human Stories in AI: Rick Marks
StatQuest with Josh Starmer
31m 13s
Origin Story & Early Motivation
- Started in high school with social insecurity augmentation project
- Attempted program to suggest perfect jokes/quotes for social situations
- Would be delivered via earpiece for real-time conversation help
- Abandoned as infeasible at the time but concept persisted throughout career
- Repurposed code for senior/junior year science project
- Built robot with computer displaying different personalities
- Could switch between joke personality and famous quote personality
- Voice changed based on personality mode
Academic Journey & Career Pivot
- PhD thesis originally focused on control theory for underwater robotics
- Worked with Monterey Bay Aquarium Research Institute
- Needed computer vision for robot sensing capabilities
- Used camera and frame grabber board for simple target following
- Switched from control theory to computer vision focus
- Colleagues convinced him computer vision should be his thesis
- "Eventually saw the light and pivoted"
- Dissertation work included:
- Fish following system using plastic turtle as target
- Pan and tilt tracking system
- Ocean floor mosaicing - stitching multiple photographs together
- Mathematical model for marine snow (particulates in water)
- Worked in Soquel Canyon off California coast
- Deep underwater canyon allowing quick access to deep water
- No ambient light - robots must provide own illumination
- Marine snow particles reflect light, creating visibility challenges
Industry Experience Progression
- Started at computer vision startup after PhD
- Moved to Sony for video game computer vision applications
- Used robots for testing PlayStation Move controller
- Worked on VR input controller development
- Performed repeated sensor testing with robotic systems
- Transitioned to Google's Advanced Technology and Projects group
- Focused on ambient computing environments
- Worked on new interaction approaches
- Founded PlayStation Magic Lab at PlayStation R&D
- Combined graphics, virtual reality, and AI interests
- Developed expertise in 3D graphics and VR
Smart Speaker Context Problem
- Gave father smart speaker as gift - father wanted to return it
- Father asked "when is golf" - device couldn't answer without context
- With golf cleats and bag: likely asking about tee time
- While channel surfing: likely asking when Masters tournament airs
- Realized context is key to effective AI assistance
- Led to Google work on context understanding and gesture recognition
- Worked on real-time inference and machine learning applications
Current Academic Position at UNC Chapel Hill
- Professor in School of Data Science and Society
- Joint appointment with Computer Science department
- Started teaching CS classes since Data Science school newly formed
- Plans to teach data science classes as school develops
Teaching: Introduction to VR and 3D Graphics Course
- Covers mathematical foundations: projection, transforms, linear algebra
- Students initially surprised by math requirements
- Uses Unity package for practical implementation
- Progression: math foundations → Unity basics → VR experience development
- Student final projects (2-week, 2-person teams):
- Magic canvas project: voice-to-text → generative AI → VR art creation
- Originally planned 3D model generation but pivoted due to latency (1 minute vs few seconds for images)
Generative AI Research Interests
- Combining AI with virtual reality environments
- Dynamic content creation based on user desires/context
- Addressing latency issues in real-time applications
- Text interfaces tolerate delays better than voice
- Real-time situations require immediate responses
- Proposed solutions using storytelling/game mechanics:
- Robot character searches through boxes while AI generates content
- Shakes boxes, says "no," continues searching until content ready
- Integrates waiting time into narrative experience
Future of Storytelling Conference
- Annual conference in New York
- Unique format: attendees take ferry to isolated location
- Combines storytellers, technologists, artists
- Small, exclusive attendance
- Explores intersection of technology and narrative
Fantasy-to-Reality VR Concept
- Inspired by fantasy book series with reality manipulation
- Characters think/speak desires and reality changes accordingly
- Potentially implementable with current generative AI + VR technology
- Users could verbally describe desired environments and watch them form
Career Philosophy & Life Plan
- Decided at age 16 on three-phase life plan:
- 25 years learning (through PhD)
- 25 years industry experience
- 25 years teaching/sharing knowledge
- Self-described as "not a planner" except for this one major life decision
- Framed as learning → practice → service progression
Why UNC Chapel Hill
- One of birthplaces of VR technology
- Met Henry Fuchs and Fred Brooks during visits
- Strong VR legacy influenced decision
- Welcoming faculty community with lunch invitations and events
- Data Science school offered flexibility vs traditional CS department constraints
- Wanted to leave California for different environment
- Connected through industry friends and conference relationships
Industry-to-Academia Transition Challenges
- Teaching preparation more challenging than expected
- Still learning effective teaching methods
- Advantage: can share real-world product examples with students
- Video games particularly relatable for student engagement
- Consumer products easier to explain than abstract business applications
Hiring & Career Advice
- Advocates for "T-shaped" professionals:
- Deep expertise in one area (the vertical stroke)
- Broad knowledge across multiple disciplines (the horizontal stroke)
- Avoid being too narrow (PhD risk) or too broad without specialty
- Ability to communicate across disciplines crucial
- Each field has its own language for describing similar concepts
- Deep contribution + team collaboration ability = ideal combination
- Students often ask about marketability and study recommendations
