AI summary by TwinMind

Live 2020-01-20!!! Favorite ML, Data Leakage, How to Learn ML

StatQuest with Josh Starmer

25m 42s

Introduction & Stream Format

  • Josh Dharma hosting StatQuest's second live stream
  • Format: 3 pre-selected comments followed by live chat Q&A
  • Plans for twice monthly streams: evening/morning to cover global time zones
  • Covers Asia, Europe, North America

Comment 1: Favorite Machine Learning Algorithm

  • Question: What is your favorite machine learning algorithm?
  • Answer: Random Forests
  • Reasoning for Random Forests:
    • Provides insight into underlying mechanisms/structure vs pure accuracy
    • Easy to interpret - shows what variables doing and why
    • Handles all data types: continuous, discrete, categorical
    • Provides clustering and visualization of sample relationships
    • Simple confidence estimation method
  • Confidence Calculation Example:
    • 100 trees example: if all agree → high confidence
    • If only 51% agree → lower confidence
    • Easy to count tree agreement for decision confidence
  • Acknowledges other algorithms are interesting but Random Forests best for his research work

Comment 2: Data Leakage in Machine Learning

  • Definition: When training and testing datasets are not independent
  • Obvious Example: Using same data for training and testing
    • Causes overestimation of model performance
  • Proper Approach:
    • Split original data into independent training/testing subsets
    • Train on training data, test on testing data
  • Sneaky Data Leakage Example: Missing data imputation
    • Wrong: Impute missing values first, then split data
    • Problem: Testing data involved in filling training data values
    • Correct: Split data first, then impute missing values separately for each subset
    • Ensures training/testing independence

Comment 3: Research and Learning Process

  • Question: Where do you learn nitty-gritty details?
  • Answer: Internet/Google research
  • Process:
    • Read everything available on topic
    • Often don't understand initially - complicated math, notation, unfamiliar terms
    • Keep reading multiple sources
    • Google unfamiliar words/terms
    • Gradually build understanding
    • Return to original sources - make more sense second time
    • Iterative process of reading, googling, re-reading
  • Timeline Examples:
    • Can take days to months to years
    • PCA video: ~2 years of research
    • Not full-time work but consistent daily reading
  • Verification Process:
    • Annotate materials
    • Redo math personally
    • Derive everything to ensure understanding
    • Run basic datasets through algorithms
    • Verify step-by-step matches paper calculations
  • Current Work: XG Boost videos - 2+ more coming with math details and optimizations

Live Chat Q&A Session

  • R-squared Discussion:
    • Never seen R-squared > 1 personally
    • R-squared < 0 possible when model worse than simple model
    • Adjusted R-squared: Compensates for adding variables
    • Problem: Adding random variables can increase R-squared by chance
    • Adjusted R-squared penalizes for number of variables used
  • Video Difficulty Classification:
    • Website (statquest.org) has video index
    • Already ordered simple to complicated within categories
    • Basics section starts with most basic concepts
    • Machine learning section progresses from simple to complex
  • Learning Formulas vs Software:
    • Depends on goals: user vs power user vs algorithm developer
    • For practitioners: main ideas most important (executive summary approach)
    • Math helpful for understanding strengths/weaknesses
    • Full derivation knowledge needed for developing new algorithms
  • Book Recommendation: Introduction to Statistical Learning
    • Contains calculus but can skip detailed math
    • Read introductions and examples without calculus
    • Covers 90% of needed knowledge
    • Available free download

Upcoming Content Plans

  • Next Month: XG Boost completion (2+ videos on math details, optimizations)
  • Housekeeping Videos:
    • Updated p-value video (2-part series)
    • Possibly naive Bayes
    • Pearson vs Spearman correlation
  • Major Topics Planned:
    • Neural networks (after current work)
    • Time series (summer/early fall)
  • Research Phase: Will spend time researching neural networks for unique StatQuest approach

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome