Configuring the AI Agent

Overview#

Klyra's AI agent reads your knowledge base, understands customer questions, and responds automatically when it's confident in the answer. When it's not sure, it hands off to your team with a summary of what the customer needs.

The key difference from other AI chatbots: Klyra's agent uses confidence scoring to decide when to respond and when to escalate. It won't make things up.

Step 1: Enable the AI agent#

1. Go to Settings > AI Agent
2. Toggle the agent on for each channel where you want automatic responses
3. The agent is disabled by default — you control exactly where it operates

Step 2: Configure confidence thresholds#

The confidence threshold controls how sure the AI must be before responding:

Set this in Settings > AI Agent > Confidence Threshold.

When the AI's confidence is below the threshold, it:

1. Does not respond to the customer
2. Creates a conversation in your inbox
3. Adds an internal note with its analysis: what it thinks the customer needs, relevant KB articles, and suggested replies

Step 3: Feed it knowledge#

The AI agent is only as good as your knowledge base. For best results:

Write clear, specific articles#

Good: "How to reset your password" with step-by-step instructions Bad: "Account management" with 15 different topics in one article

Cover common questions first#

Look at your recent support conversations. The top 10 questions should each have a dedicated KB article.

Use natural language#

Write articles the way a customer would ask the question. The AI uses semantic search, so natural phrasing works better than technical documentation.

Keep articles updated#

Outdated articles lead to wrong answers. Review your KB monthly.

How the AI agent processes a message#

Here's what happens when a customer sends a message:

1. Intent classification — the AI determines what the customer is asking
2. Knowledge base search — relevant articles are retrieved using semantic search
3. Response generation — a response is drafted based on the matched articles
4. Confidence scoring — the AI evaluates how confident it is in the response
5. Decision — if confidence meets the threshold, it responds; otherwise, it escalates
6. PII redaction — personal information is automatically redacted from AI training data

The entire process takes 2-5 seconds.

Resolution and handoff#

Auto-resolution#

When the AI agent responds and the customer indicates their question is answered (e.g., "Thanks!", "That worked!"), the conversation is automatically marked as resolved. This:

• Updates your resolution metrics
• Triggers a CSAT survey (if configured)
• Generates training data to improve future responses

Human handoff#

When the agent escalates to your team, the conversation includes:

• The full customer message history
• The AI's analysis of the customer's intent
• Suggested replies your team can use or edit
• Links to relevant KB articles

Your team picks up seamlessly — the customer doesn't know the AI was involved.

Advanced: Persona configuration#

You can create different AI personas for different contexts:

1. Go to Settings > AI Agent > Personas
2. Create a persona with:
   • Name — the display name customers see
   • Tone — professional, friendly, technical, etc.
   • Knowledge scope — limit which KB categories the persona can access
   • Instructions — custom behavior rules (e.g., "Always suggest booking a demo for pricing questions")

This is useful if you have different products or audience segments.

Monitoring AI performance#

Track your AI agent's effectiveness in Reporting > AI Agent:

• Resolution rate — percentage of conversations resolved without human intervention
• Confidence distribution — how often the AI is confident vs. uncertain
• Handoff rate — how often conversations get escalated to humans
• CSAT for AI-resolved conversations — customer satisfaction for auto-resolved tickets
• Average response time — typically 2-5 seconds

Troubleshooting#

AI agent not responding:

• Verify it's enabled for the specific channel (Settings > AI Agent)
• Check that your knowledge base has published articles
• The agent won't respond if the confidence is below threshold — this is expected behavior

Responses are inaccurate:

• Review and improve the relevant KB articles
• The AI can only use what's in your knowledge base — if the information isn't there, it can't answer correctly
• Raise the confidence threshold temporarily while you improve content

Agent responding too slowly:

• Response time depends on AI model load (typically 2-5 seconds)
• If consistently slow, check Settings > AI Agent for any rate limiting indicators

Next steps#

• Understanding health scores — see how AI resolution impacts customer health
• Create KB articles — improve your AI agent's accuracy
• Set up Slack — extend AI capabilities to Slack conversations