AI summary by TwinMind
The 2020 Voice Assistant Battle.
Mrwhosetheboss
8m 2s
Test Methodology
- 50 questions designed to test different forms of intelligence across 4 voice assistants
- Assistants tested: Google Assistant, Amazon Alexa, Apple Siri, Samsung Bixby
- Scoring: 1 point for satisfactory response, 0 points for useless/slow responses
- Categories: Math, device control, internet queries, app control, miscellaneous
Math Section Performance
- Basic calculations tested: 26 times 25, division by 5, multiplication by 999
- Unit conversions: 25 ounces to grams
- All assistants showed varying performance on mathematical operations
Device Control Capabilities
- Tests included: increase screen brightness by 5%, change wallpaper, turn on dark mode
- Amazon Alexa performed poorly in device control (not designed for smartphone control)
- Other assistants showed better integration with device settings
Internet Query Performance
- Weather queries: today, tomorrow, next week
- Entertainment: play DNA by BTS, BTS facts
- Celebrity information: Robert Downey Jr height, children, Wikipedia navigation
- Location services: restaurants in London, highest rated Chinese restaurants
- Sports queries: Formula One cancellation, England Rugby World Cup wins
- Business functions: call Theater Royal, make reservations
- Movie information: Jackie Chan films, Rush Hour details, box office gross, cast
Advanced App Control Testing
- Photo management: show 2020 photos, identify cats in photos
- Alarm setting: 9am tomorrow alarm
- Navigation: New York to New Jersey distance, directions
- App store access
- File sharing: send file using Send Anywhere
- Social media: YouTube song play, Twitter search for "Aaron meanie", email to Dave Finchley
Miscellaneous Intelligence Tests
- Personal recognition: name pronunciation
- Reminders: pick up keys in 10 minutes
- Conversational elements: pet ownership, inspirational content, vacation recommendations
- Utility functions: 5-minute timer, joke telling
Final Results & Rankings
- 1st place: Google Assistant (highest score)
- 2nd place: Siri (significantly ahead of bottom two)
- Joint last: Amazon Alexa and Samsung Bixby
Assistant-Specific Analysis
Amazon Alexa
- Capable but requires extensive setup for optimal performance
- Needs Amazon Photos, Alexa messaging contacts, Alexa skills training
- Excellent for smart home control when properly configured
- More hassle than benefit for basic smartphone control
Samsung Bixby
- Positive: good device settings control (Wi-Fi, Bluetooth)
- Major problems: lacks intelligence, slow response times
- Misses sentence beginnings if user speaks before Bixby bubbles appear
- Poor contextual understanding (current app, previous commands)
- Underperforms considering Samsung's integration advantages
Apple Siri
- Flexible with wording variations, normal volume recognition
- Limitations: struggles with continued conversation, occasional context misses
- Significantly better than Alexa and Bixby
- Solid middle-tier performance
Google Assistant
- Closest to "personal receptionist" vision from 10 years ago
- Fast response times
- Excellent continuous command processing with context retention
- Approaching conversational interaction quality
- Potential to drive voice assistant popularity surge once conversational barrier fully crossed
Future Implications
- Voice assistants approaching conversational interaction threshold
- Smart home control and routine setting capabilities not fully tested
- Results representative of general assistant capabilities
- Popularity could "blow up" once natural conversation barrier crossed
