The bottom line: Customer service AI with autonomous decision-making requires transparency, auditability, and clear accountability structures under the EU AI Act, especially when classified as a high-risk AI system.
AI systems make autonomous decisions in customer support with direct impact on users. Shana Simmons, Chief Legal Officer at Zendesk, explains in an interview with ITWELT.at how companies can meet legal transparency requirements and when customer service AI falls under the high-risk categories of the EU AI Act.
Artificial intelligence is deployed in customer service applications to process inquiries automatically, generate solution proposals, or make escalation decisions. This autonomous decision-making raises fundamental questions about accountability and liability: who is responsible for faulty or discriminatory decisions made by the system?
Under the EU AI Act, certain AI use cases in customer service are subject to high-risk classifications, particularly when they process personal data on the basis of automated decision-making or affect the availability of services. In such cases, comprehensive documentation, audit capability, monitoring, and transparency mechanisms towards users are required. Shana Simmons emphasizes that genuine transparency does not consist solely of technical compliance documentation, but must communicate clearly to end users when they are interacting with AI and what decision logic underpins it.
Companies must continuously assess whether their AI systems in customer service exceed the thresholds for high-risk classification and implement corresponding governance processes. This includes risk assessments before deployment, regular performance monitoring for bias and error rates, and an escalation mechanism for users who can request manually reviewed decisions.
Source: itwelt.at · Published 24 June 2026
Lumi AI News — AI-assisted curation in accordance with Article 50 EU AI Act. Paraphrase and classification by Lumi News Pipeline v1.7.1.