AI summary by TwinMind
Lecture 17: Direct Prediction of Outcome / Mortality
MIT OpenCourseWare
1h 15m 2s
Microarray Technology Platforms Overview
- cDNA arrays: cheaper, widely used in biology labs
- Labs print their own arrays, have their own libraries
- Affymetrix arrays: most accurate but expensive (~$2,000 per array)
- Consensus view: most accurate platform currently available
- Agilent oligonucleotide arrays: emerging alternative
- Whole setup required, cheaper than Affymetrix
- People seem to like it but future unclear
- Custom oligonucleotide printing: ~1/3 cost of Affymetrix
- Companies make oligonucleotides, users print themselves
- Quality uncertain, cheaper but not as good
- Harvard Partner Center attempting human and mouse printing
- SAGE: very accurate for low copy number transcripts
- Involves sequencing every tag
- Expensive but accurate for small transcript numbers
- Large libraries available locally
Systems Biology and Data Integration
- Integration of genomics and proteomics high-throughput technologies
- Challenge: biologists often unaware of available tools/databases
- Need to guide biologists on appropriate methods rather than assume knowledge
- Commercial tools: Ingenuity software for expression data analysis
- Generates networks, links literature citations
- Hand-curated database but expensive
- Free alternatives exist but lower quality
- Protein-DNA binding data: Whitehead/Rick Young lab yeast study
- ~100+ transcription factors mapped to genome binding sites
- In vivo binding locations identified
- Human genome efforts underway, progress unclear
- Protein microarrays: less successful than initially expected
- Manufacturing difficulties
- Harvard Partner Center: 50-60 molecule arrays with antibody detection
- Technology not yet cost-effective for most investigators
- Mass spectrometry: major growth area in past 1-2 years
- Enormous amounts of data and informatics challenges
Reliability Issues in Early Studies
- Nature Genetics 2001 paper controversy: "Transcriptional regulation and function during the human cell cycle"
- Claimed 700 mammalian genes with cell cycle specific patterns
- 2002 PNAS rebuttal paper challenged statistical approach
- Found patterns could result from random chance due to multiple testing
- Ongoing debate with websites detailing rebuttals
- Truth likely somewhere between extremes
- Lesson: ensure robust methodology before publication
Platform Comparison and Reproducibility Problems
- Substantial differences between microarray platforms (cDNA vs Affymetrix)
- Common question: which platform to use, do they agree?
- Most studies show surprisingly small overlap between platforms
- Study comparing U95 vs U133 Affymetrix generations:
- 14 muscle biopsy samples from inflammatory myopathy patients
- Hierarchical clustering separated by array type rather than sample characteristics
- Differential gene expression overlap: ~20% for top 100 genes, ~15% for top 50
- Same RNA, same labeling, same facility - expected higher agreement
- Probe sequence differences cause major expression changes
- Identical probe sequences: very reproducible (correlation 0.98-99)
- Different sequences detecting same transcript: no relationship
- Affymetrix improvements over generations through trial-and-error elimination of bad probes
- Recommendation: avoid combining different platforms unless using specialized normalization methods
Sample Size and Statistical Considerations
- Biological variability dominates technical measurement errors for established platforms
- Don't need many technical replicates for Affymetrix
- Sample size depends on unknown variability in samples
- General estimates: 10-15 samples per group for two-group comparison
- ~75% false positive rate for detecting 3-fold variable genes
- Applies to high-variability data like cancer studies
Multiple Testing Corrections
- Example: 10 coin flips all heads = 0.1% probability
- But with 10,000 people flipping coins, one getting 10 heads not surprising
- Same problem with thousands of genes tested simultaneously
- P-value depends on chip size (10K vs 30K genes affects conclusions)
- False Discovery Rate (FDR) recommended approach
- Calculation methods exist but not simple to implement
Classification and Phenotype Analysis Framework
- Clinical applications require incorporating phenotypic data
- Relationship questions: genes correlated with phenotypes, predictive variables
- Phenotype types in order of complexity:
- Binary (disease vs normal): most studied, many methods applied
- Multiple subclasses: non-ordered subtypes
- Ordered subclasses: severity ratings 1-5
- Continuous: invasive ability scores
- Censored data: survival times with dropouts
- Many can be reduced to binary but lose information
- Better to preserve original phenotype structure
Feature Selection and Dimensionality Reduction
- Necessary due to high gene numbers vs sample sizes
- Methods include:
- Gene filtering: expression thresholds, variation filtering
- Linear combinations: PCA, singular value decomposition
- Direct gene selection: t-tests, various statistical methods
- Trade-offs between interpretability and performance
Classification Accuracy Problems in Literature
- Many papers overestimate prediction accuracy
- Rick Simon NIH simulation study (2002):
- 20 random expression profiles, 10 per class
- No true underlying differences
- Resubstitution error: 98.2% perfect classification (clearly wrong)
- Leave-one-out after gene selection: 90% accuracy (still biased)
- Correct cross-validation: ~50% accuracy (expected for random data)
- Problem: gene selection using all data, then cross-validation
- Proper method: remove test sample before any analysis steps
- Most current papers acknowledge bias but often report inflated rates
Survival Analysis Applications
- Censoring rates often ~50% (study termination, patient dropout)
- Right censoring most common
- Univariate approaches: log-rank tests for individual genes
- Voting methods: combine individual gene predictions
- Often suboptimal compared to multivariate approaches
Clinical Implementation Challenges
- No hospitals currently using microarray-based predictions clinically
- Netherlands planning implementation based on published results
- Barriers identified:
- Simple existing tests often very cheap and effective
- Unclear cost-benefit advantage over current methods
- Insurance reimbursement issues
- Per-patient payment systems don't cover unproven procedures
- Need proof of necessity for insurance approval
- Platform reliability concerns affect adoption
Literature Examples
Nature Paper (Netherlands Group)
- Claims gene expression profiles outperform clinical primers
- Used for breast cancer outcome prediction
- Approach: build classifier, use Kaplan-Meier plots for validation
- Show expression data captures information not in traditional prognostic factors
- Integrate expression signature as single variable in Cox multivariate model
Recent Stanford Study (Tibshirani Methods)
- 116 adult patients: 59 training, 57 testing
- Advanced gene selection methods allowing gene interactions
- Stepwise regression-type search through gene combinations
- Prediction algorithm from PNAS paper
- Multivariate analysis showed gene expression as independent prognostic factor
Advanced Methodology Development
- Partial Least Squares (PLS) for survival data
- Compromise between PCA (maximizes information) and regression (phenotype-focused)
- PCA: optimal dimension reduction but ignores phenotype
- Regression: phenotype-focused but limited dimension reduction
- PLS: balances both objectives
- Extended to censored survival data (previously worked out but unused)
- More significant patient group separation than standard methods
- Requires mathematical training but provides better insights
Software Recommendations
R Statistical Software
- Free, Linux-like collaborative development
- Preferred over SAS (clunky, outdated graphics) and S-Plus (commercial)
- Developed at Bell Labs, maintained by original S language developers
- Co-author Robert Gentleman at Dana-Farber
- MATLAB-like feel: no variable declaration, vector operations
- Superior statistics routines compared to MATLAB
- Memory management issues but improving
- One line in R can replace hours of Excel work
Bioconductor Project
- R-based microarray analysis package collection
- Started by Robert Gentleman at Dana-Farber
- Originally proposed name "MADMAN" (rejected by NIH)
- Website: bioconductor.org
- Downloadable packages for various algorithms
- Normalization methods, analysis tools available
- Easy Windows installation
- Standard platform for algorithm developers
- Search challenge: "R" difficult to search, use "bioconductor R" instead
