AI summary by TwinMind
Lecture 16: Microarray Disease Classification II
MIT OpenCourseWare
1h 7m 23s
Previous Session Review
- Covered classification methods in genomic technologies for clinical medicine applications
- Three types: class exploration, class prediction, class discovery
- Finished discussion on class prediction approach
- Starts with microarray data from patients
- Builds model to predict outcomes
- Basic steps: choose gene set, construct prediction function, determine classification rule, validation
Current Session Focus
- Detailed analysis of breast cancer Nature 2002 paper
- New England Journal of Medicine articles on AML classification methods published today
- Editorial perspective claims clinical trials will change dramatically
- Future: small trials with high data density
- Precise patient stratification by expression profiles
- Highly tailored microarray analysis
- Author skeptical of dramatic changes prediction
Nature 2002 Breast Cancer Study Overview
- Gene expression profiling predicting clinical outcome
- 117 young patients with primary breast tumors
- Applied supervised classification for gene expression signature
- Predictive of short interval to distant metastasis
- Focus on prediction rather than specific tool used
Hierarchical Clustering Analysis (Class Exploration)
- Used 98 tumors from original 117 patients
- Clustered based on ~5,000 significant genes from 25,000 total
- Figure 1a shows specimens as rows, genes as columns
- Dendrograms show relationships between samples
- Binary splitting at each node
- Can spin clusters at any node without changing organization
- Identified two main classes from clustering
Class Discovery Phase
- Correlated clustering results with clinical phenotypes
- Six different phenotypic markers examined per patient
- Notable correlations:
- Estrogen receptor status
- Distant metastases at 5 years
- ER positive patients: better outcomes
- ER negative patients: worse outcomes
- No novel class discovery - correlates with known ER status effects
Refined Dataset for Prediction Model
- Focused on 78 patients with sporadic lymph node negative disease
- Excluded BRCA1/BRCA2 mutation carriers
- 44 patients: good prognosis (disease-free after 5 years)
- 34 patients: poor outcome (developed metastasis)
Three-Step Supervised Classification Method
Step 1: Gene Selection
- Started with 25,000 genes on microarray
- Filtered to 5,000 "significant" genes
- Significantly regulated in >3 tumors out of 78
- Ad hoc filtering process, varies between papers
- Further selection by correlation with ideal outcome vector
- Arranged data: good prognosis patients first, poor prognosis second
- Created ideal vector: ones for good, zeros for poor
- Calculated correlation coefficients for each gene
- Used cutoff of ±0.3 correlation coefficient
- Resulted in 231 genes
Step 2: Discriminative Gene Set Optimization
- Rank ordered 231 genes by correlation coefficient magnitude
- Iterative process:
- Started with top 5 genes
- Built model, validated with leave-one-out cross-validation
- Added 5 more genes, repeated process
- Continued until accuracy stopped improving
- Final discriminative set: 70 genes
- Critical Issue: Serious overfitting problem
- Using same data for gene selection and validation
- Repetitive cross-validation on training data
- Similar to fitting perfect curve through points that won't generalize
Step 3: Prediction Function and Rule
- Prediction function: correlation coefficient approach
- Created average "good profile" and "bad profile" from training data
- For new patient: calculate correlation to good profile
- Threshold rule: above threshold = good prognosis, below = bad prognosis
- Explored different threshold values (sensitivity vs specificity trade-off)
Validation Results
- Used separate validation set of 19 samples
- 7 good outcome, 12 bad outcome patients
- 2 out of 19 incorrect classifications
- Important: accuracy may differ between good vs bad predictions
- Follow-up New England Journal study showed concerning validation results:
- Without proper gene selection validation: 27% error rate
- With proper repeated gene selection: 41% error rate (approaching chance)
Clinical Implementation Discussion
- Netherlands Cancer Institute planned to use 70-gene set clinically
- Patent filed on gene set as breast cancer predictor
- Major concerns about premature clinical implementation
Validation and Clinical Trial Issues
- Proper validation requires prospective studies
- Problem: interventions based on predictions affect outcomes
- Saint Galen scale comparison flawed:
- Historical patients treated based on Saint Galen predictions
- Interventions specifically designed to improve poor prognosis cases
- Gene expression profiles weren't used historically for treatment decisions
- Need drug approval-like process: prospective controlled trials
FDA Regulation Challenges
- Roche diagnostics attempted P450 genotyping chip approval
- FDA rejected waiver request, required full approval process
- Harvard School of Public Health helping FDA develop standards
- No current guidelines for microarray data interpretation
- Internal medical center use possible without FDA approval if not sold commercially
- Partners HealthCare example: resequencing chips for hearing loss/cardiomyopathy genes
Methodological Concerns
- Small sample sizes with high-dimensional data problematic
- Author disagrees with perspective that trials can be smaller with microarray data
- Believes trials need to be larger due to increased data complexity
- Massive overfitting risk with small trials and high-density data
