AI summary by TwinMind
Unit 9: Value of Information, Video 2: Value of Test
MIT OpenCourseWare
6m 23s
Expected Value Concept
- Expected value of information is always dealing with expectations in uncertain world
- Information may indicate something is good but reality could be different
- Not absolutely sure outcomes - probabilistic nature of information value
- Baseline understanding important for full analysis approach
Test Results Complexity
- Calculations are complex for information value analysis
- Tests can have many different results (not just yes/no answers)
- Example: prototype might work very well, poorly, or fail completely
- Range of possible outcomes rather than binary results
- Test results don't prove what will happen - they are samples
- Medical test analogy: COVID/TB tests can have false negatives or false positives
- Results indicate probability, not certainty
Information Updates and Bayes Theorem
- New information updates prior estimates rather than providing definitive answers
- Each piece of new information (whether performs well, very well, poorly, etc.) updates prior estimates
- Technical method: Bayes theorem for updating probabilities
- Complex topic due to multiple possible outcomes
- Five different test results → five different value implications
- Each result has different probability
- Five different updates to system
- Each has different set of prior probabilities updated to posteriors
- Results in five different solutions
Bayes Theorem Formula
- Simple formula structure: P(A|B) = P(A) × factor
- Posterior probability = prior probability × correction factor
- Example scenario:
- Prior probability: 50% chance product will sell well
- Test at state fair: people gushed over tractor
- Updated posterior: 70% chance it will sell well
- Factor calculation:
- Numerator: probability that B correlates with A (test responses lead to market results)
- Denominator: prior probability of B
Practical Implementation Challenges
- Formula is simple but elements unavailable in practice
- Missing data on correlations:
- How state fair responses correlate with actual sales
- Probability of specific test outcomes
- Conditional probabilities generally unknown for new systems
- Works better for repetitive tests with historical data:
- Agricultural sampling
- Test fields with lots of accumulated data
- For general system design and management: data typically not available
- Need alternative approaches due to practical limitations
