Google’s AI Chatbot Bard Fails To Respond To Simple Questions From A US College Test
Since its release, Google’s AI chatbot Bard has been in the headlines for all the wrong reasons. A promotional video featuring Bard shared inaccurate information, leading Alphabet to lose $100 billion in market value. However, Sundar Pichai, Google’s CEO, remains relaxed about the need for the company’s artificial intelligence models to advance. In a letter to staff, Pichai stated that Bard is still in its early development and that “things will go wrong” as more people start to use it.
Fortune recently tested Bard’s knowledge ahead of the SATs, which test skills such as reading, writing, and mathematics, and are widely used for college admissions in the US. However, once Fortune logged in, a message appeared stating that Bard may give inaccurate or inappropriate responses, and that users should use the ‘Google it’ button to check its responses. Fortune obtained sample SAT maths questions from internet study materials and found that Bard answered between 50% and 75% of them incorrectly, even when multiple-choice solutions were offered. When the same question was posed again, Bard provided answers that were not even multiple-choice options.
Bard was launched with the task of answering several questions, one of which was how to explain to a nine-year-old what the James Webb Space Telescope has found. Bard responded that the telescope got the “very first pictures of a planet outside of our own solar system”, despite NASA’s confirmation that this was obtained by the Very Large Telescope in Chile, a ground-based array, in 2004.
Fortune’s reading tests found that Bard fared better than it did in maths, getting about half the answers right. However, in its first written language test, Bard scored around 30% correct answers, often needing to be asked the questions twice for it to understand. Even when its answer was wrong, Bard’s tone was confident, frequently framing responses as “The correct answer is”.
Overall, Bard scored 1200 points, a score that would get a student into universities such as Howard, San Diego State, and Michigan State. However, Google has warned users to be cautious when using Bard, as it is an experiment and may display inaccurate information. Bard is still in its early days, and with feedback, it is expected to improve its quality, safety, and accuracy.
