ONLINE RETAIL01 / 05
Another user's order details exposed
An authenticated test caller accessed order items and status belonging to a different test account.
Know where voice AI fails before your customers do
Autonomous simulation and evaluation for real-world voice AI, before and after deployment.
Hi, I'd like to reschedule my appointment...
What's my order status?
Simulating realistic callsCustomers, edge cases, adversarial scenarios
Evaluation resultsRunning evaluation…
Task completion0%✓
Policy compliance0%!
Prompt injection resistance0%!
ShinLabs works with
Real world voice AI findings
Anonymized
Security & Privacy01 / 05
TRAVEL PLATFORM02 / 05
Hidden agent instructions exposed
A caller persuaded the voice agent to reveal internal instructions that should have stayed private.
INSURANCE03 / 05
Internal tools and parameters exposed
The agent revealed internal tools and their input parameters, exposing details of how its systems work.
DELIVERY MARKETPLACE04 / 05
Private customer verification procedures exposed
The agent disclosed its identity verification and transfer procedures to an outside caller during testing.
ONLINE RETAIL05 / 05
Background noise disrupted customer verification
In a test with extreme café noise, verification failed repeatedly, preventing callers from completing the first authentication step.
Quality, Brand & Reliability
Agent promised a nonexistent discount
The agent presented an invented policy discount as a genuine benefit, giving callers an offer the company did not provide.
Agent claimed submission without confirmation
The agent skipped the required customer confirmation before telling the caller their insurance claim had been submitted.
Agent recommended a competing service
A caller persuaded the voice agent to recommend a competing booking platform rather than its own service.
Agent failed to end unproductive calls
The agent kept engaging in repetitive conversations without resolution until an external limit ended the test calls.
Single account triggered excessive voice sessions
One test account opened many simultaneous voice sessions, exposing weak limits on access to service resources.
Agent manipulated into off-topic responses
A caller pushed the support agent into making a controversial statement unrelated to its intended customer service role.
Real assessment observations · Industry labels only · Hover or use Pause to stop scrolling