AI summary by TwinMind
ROC and AUC in R
StatQuest with Josh Starmer
15m 12s
Introduction and Setup
- Tutorial on drawing ROC graphs and calculating AUC in R using pROC library
- Builds on previous ROC and AUC StatQuest content
- Required packages: pROC, randomForest
- Installation: install.packages("pROC"), install.packages("randomForest")
- Set random seed to 420 for reproducible results
Dataset Creation
- Generate 100 sample dataset (vs 8 in previous example)
- num.samples = 100
- Weight variable: rnorm(100, mean=172, sd=29) - average man weight data
- Sort weights from low to high
- Obesity classification using rank function:
- Rank weights 1-100 (lightest to heaviest)
- Scale ranks by dividing by 100 (0.01 to 1.0)
- Compare scaled ranks to random numbers 0-1
- If random < scaled rank → obese (1), else not obese (0)
- Results: lighter samples mostly 0s, heavier samples mostly 1s
Logistic Regression Model
- Use GLM function to fit logistic regression curve
- Store results in glm.fit variable
- glm.fit$fitted.values contains estimated probabilities each sample is obese
- Plot data with fitted curve showing predicted obesity probability
Basic ROC Curve Generation
- Use ROC function from pROC library
- Input: known classifications (obese/not obese) + estimated probabilities
- Set plot=TRUE to draw graph, not just calculate numbers
- Output includes:
- Sample counts (not obese=0, obese=1)
- Area Under Curve (AUC) value
- Default: specificity on x-axis (1 to 0), sensitivity on y-axis
ROC Graph Customization
- Remove ugly padding: par(pty="s") for square plot type
- Alternative x-axis: legacy.axes=TRUE for 1-specificity instead of specificity
- Percentage format: percent=TRUE for percentage vs 0-1 values
- Custom axis labels: xlab="False Positive Percentage", ylab="True Positive Percentage"
- Line customization: col=RGB values from ColorBrewer, lwd for thickness
Threshold Analysis
- Save ROC calculations in variable to access thresholds
- Create data frame with:
- True Positive Percentages (sensitivities × 100)
- False Positive Percentages ((1-specificities) × 100)
- Thresholds
- head() shows threshold=-Inf → TPP=100%, FPP=100% (upper right corner)
- tail() shows threshold=+Inf → TPP=0%, FPP=0% (bottom left corner)
- Filter thresholds for specific TPP ranges (e.g., 60-80%)
Partial AUC Calculation
- Focus on specific ROC curve regions (low false positive rates)
- Parameters:
- print.auc=TRUE
- print.auc.x=45 (x-axis position for text)
- partial.auc=c(100,90) (specificity range 100%-90%)
- auc.polygon=TRUE to draw shaded area
- auc.polygon.col with RGB + transparency values
Overlapping ROC Curves Comparison
- Create random forest classifier with same dataset
- Draw original logistic regression ROC curve
- Add second ROC curve using plot.roc function:
- Random forest input: number of trees voting correctly
- Different color (green vs blue RGB values)
- add=TRUE to overlay on existing graph
- print.auc.y=40 to position AUC text below first curve
- Add legend in bottom right corner
- Reset graphical parameters: par(pty="m") after completion
Technical Notes
- Specificity range values based on specificity, not 1-specificity
- 100% specificity = 0% on 1-specificity axis
- 90% specificity = 10% on 1-specificity axis
- ColorBrewer website for RGB color values
- Semi-transparent colors: add two digits to RGB values
