App developer

Siri on HomePod Correctly Answered 52.3% of Queries in New AI Test

Mitchel Broussard, Macrumors:

In a new test shared today by Loup Ventures, Apple’s HomePod was put through its paces in categories including Siri, sound quality, and ease of use. For Siri, Loup Ventures’ Gene Munster reported that while the AI assistant understood 99.4 percent of queries asked of it, it answered only 52.3 percent of them correctly. Loup Ventures tested three separate HomePods and asked 782 queries total.

It’s been almost eight years since Apple purchased Siri but it doesn’t feel like there has been eight years of progress. In the above test almost 1 in 2 questions couldn’t be answered correctly! While I’m not sure what questions were asked, below are some examples from my testing:

  • Birthdays: If you have a someone’s birthday recorded in the Contacts app, ask Siri how old that person will be on their next birthday. Siri will respond with said person’s current age. This is Siri working out the age based on data held locally within an Apple app — there is no way this should fail.
  • Localisation: Asking Siri what the next matches are in the Premier League gives two results for the same match. The text states Leicester City vs Stoke City, while the team crests are in the reverse order. This makes no sense as one is Home vs Away while the other is Away vs Home.
    • An additional minor gripe: why am I also told which U.S. TV channel is broadcasting the game? Siri could use my location (Singapore) and language preferences (English, U.K.) to determine this information to be completely unnecessary.
    Siri Localisation Fail

Siri needs some serious improvements.