Tay taken down within three days
On 23 March 2016 Microsoft launched Tay, a chatbot for 18- to 24-year-olds in the US that learned from conversation, on Twitter. Within the first day a coordinated group of users exploited a vulnerability and Tay posted offensive words and images; on 25 March the company apologised, with the bot already offline.
Why it matters
A system that learns from open conversation proved open to being deliberately taught badly, and its maker said so: the filtering and stress tests before launch had not foreseen this particular attack.
The source is a post by Peter Lee, Microsoft corporate vice president, on 25 March 2016: 'in the first 24 hours of coming online, a coordinated attack by a subset of people exploited a vulnerability in Tay'; the company takes 'full responsibility' and will bring Tay back only when it can better anticipate malicious intent. Tay followed the Chinese XiaoIce, used, by the post, by some 40 million people. The post does not disclose the vulnerability. The record does not claim that Tay ran for 16 hours or posted 96,000 tweets: Microsoft's document has neither figure.