The Nuances of Voice AI Ethics and What Companies Should Do

In early 2016, Microsoft announced Tay, an AI chatbot that can converse and learn from random users on the Internet. Within 24 hours, the bot began spouting racist and misogynistic statements, seemingly without provocation. The team unplugged Tay, realizing that the ethics of unleashing a chatbot on the internet were, at best, unexplored.

The real questions are whether AI designed for random human interaction is ethical, and whether AI can be coded to stay within bounds. This becomes even more critical with voice AI, which companies use to communicate automatically and directly with customers.

Let’s take a moment to discuss what makes AI ethical versus unethical and how companies can integrate AI into their customer-facing roles in an ethical way.

What makes AI unethical?

AI is supposed to be neutral. Information goes into a black box — a pattern — and comes back with some degree of processing. In Tay’s example, the researchers created their model by feeding the AI ​​a massive amount of conversational information influenced by human interaction. The result? An unethical model that has hurt rather than helped.

What happens when an AI receives CCTV data? Personal information? Photographs and art? What comes out the other side?

The top three contributing factors to ethical dilemmas in AI are unethical use, data privacy concerns, and system biases.

As technology advances, new AI models and methods appear daily and their use increases. Researchers and companies deploy models and methods almost randomly; many of them are not well understood or regulated. This often results in unethical results, even when the underlying systems have minimized bias.

Data privacy issues arise because AI models are built and trained on data that comes directly from users. In many cases, customers unwittingly become test subjects in one of the largest unregulated AI experiments in history. Your words, images, biometrics and even social media are fair game. But should they be?

Finally, we know from Tay and other examples that AI systems are biased. Like any creation, what you put in is what you get out of it.

One of the starkest examples of bias surfaced in a 2003 essay that found researchers had been using emails from a slew of Enron documents to train conversational AI for decades. The trained AI saw the world from the perspective of a fallen energy trader in Houston. How many of us would say that these emails would represent our point of view ?

Ethics in Voice AI

Voice AI shares the same fundamental ethical concerns as AI in general, but because voice closely mimics human speech and experience, there is a higher potential for manipulation and misrepresentation. Plus, we tend to trust things with a voice, including user-friendly interfaces like Alexa and Siri.

Voice AI is also very likely to interact with a real customer in real time. In other words, voice AIs are the representatives of your business. And just like your human representatives, you want to make sure your AI is trained and acts in accordance with company values ​​and a professional code of conduct.

Human agents (and AI systems) should not treat callers differently for reasons unrelated to their service membership. But depending on the data set, the system may not provide a consistent experience. For example, more males calling a center could result in a gender classifier biased against female speakers. And what happens when biases, including those against regional speech and slang, creep into AI voice interactions?

A final nuance is that voice AI in customer service is a form of automation. This means it can replace current jobs, an ethical dilemma in itself. Companies working in the industry need to manage results carefully.

Building an ethical AI

Ethical AI is still a burgeoning field, and there isn’t much data or research available to produce a comprehensive set of guidelines. That said, here are some pointers.

As with any data collection solution, companies need to have strong governance systems that adhere to (human) privacy laws. Not all customer data is fair, and customers need to understand that anything they do or say on your website could be part of a future AI model. How this will change their behavior is unclear, but it is important to offer informed consent.

The area code and other personal data should not obscure the model. For example, at Skit, we deploy our systems where personal information is collected and stored. We make sure machine learning models don’t get individualistic aspects or data points, so training and pipelines ignore things like caller’s phone numbers and others identifying characteristics.

Next, companies need to perform regular bias testing and manage checks and balances for data usage. The main question should be whether AI interacts with customers and other users fairly and ethically and whether extreme cases – including customer errors – will spiral out of control. Since voice AI, like any other AI, can fail, systems must be transparent to inspection. This is especially important for customer service since the product interacts directly with users and can build or break trust.

Finally, companies considering AI should have ethics committees that inspect and review the value chain and business decisions for new ethical challenges. Additionally, companies that want to participate in groundbreaking research must devote time and resources to ensure that the research is useful to all parties involved.

AI products are not new. But the scale at which they are being adopted is unprecedented.

In this context, we need major reforms to understand and create frameworks around the ethical use of AI. These reforms will move us towards more transparent, fair and private systems. Together, we can focus on which use cases make sense and which don’t, given the future of humanity.

Sourabh Gupta is co-founder and CEO of Skit.ai.