Skip to main content

Auto-Answer

Auto-Answer allows Chat Aid to automatically respond to questions in Slack channels without being explicitly mentioned, providing instant answers to your team 24/7.

Plan Differences

Auto-Answer is available on all plans. Individual (Free) plans use default settings and cannot customize thresholds or disable the feature. Team plans and above can fully customize Auto-Answer settings.

Platform Limitation

Auto-Answer is currently only supported on Slack. This feature is not available on Microsoft Teams, Web Dashboard, or other platforms.

What is Auto-Answer?​

Auto-Answer is a premium feature that enables Chat Aid to proactively monitor Slack channels and automatically respond to questions. Instead of requiring users to mention @Chat Aid, the system intelligently detects questions and provides answers when it's confident in the response quality.

How it works:

  1. Chat Aid listens to messages in enabled channels
  2. ML models detect if a message is a question
  3. Confidence evaluation determines if Chat Aid should answer
  4. If thresholds are met, Chat Aid responds automatically
  5. Users get instant answers without any extra steps

Benefits​

  • ⚡ Instant Answers - No waiting for team members or searching documentation
  • 🎯 Reduced Friction - Users don't need to remember to mention Chat Aid
  • 📊 Lower Support Load - Common questions answered automatically
  • 🤖 Smart Detection - Only answers when confident in response quality
  • 📈 Better Adoption - Seamless integration into existing workflows

Prerequisites​

  • Slack connected to Chat Aid
  • Premium plan or higher
  • Admin permissions
  • At least one data source connected and trained

Enabling Auto-Answer TeamBusinessEnterprise​

Step 1: Enable Auto-Answer​

  1. Go to Settings > Auto-Answer
  2. Toggle Auto-Answer to "Enabled"
  3. This enables the feature globally

Step 2: Select Channels​

  1. In the Channels table, you'll see all your Slack channels
  2. Select channels where you want auto-answer enabled
  3. Click Enable Auto-Answer for selected channels

Start Small: We recommend starting with 1-2 channels to test:

  • Support/help channel
  • General/company-wide channel
  • FAQ channel

Step 3: Configure Behavior​

Choose between:

  • Default Settings - Recommended for most teams
  • Custom Settings - Fine-tune confidence thresholds

See Configuration Options below for details.

Step 4: Monitor Performance​

  1. Navigate to Auto-Answer Logs to view activity
  2. Review when Chat Aid did/didn't respond
  3. Adjust settings based on results

Configuration Options TeamBusinessEnterprise​

Auto-Answer Logic​

Default (Recommended):

  • Pre-configured confidence thresholds
  • Optimized for most use cases
  • Safe starting point

Custom:

  • Fine-tune individual thresholds
  • Requires understanding of evaluation metrics
  • Best for advanced users

Confidence Thresholds TeamBusinessEnterprise​

Auto-answer uses a multi-stage evaluation pipeline. At each stage, Chat Aid checks confidence against a threshold:

1. Is Question (0-1)​

What it checks: Is this message actually a question?

Default: 0.9 (90% confidence)

Adjust when:

  • Too many false positives (responding to non-questions) → Increase
  • Missing obvious questions → Decrease

Examples:

  • "What's our refund policy?" ✓ (high confidence)
  • "Looking at the docs" ✗ (low confidence)

2. Is Fact Question (0-1)​

What it checks: Is this asking for factual information vs. opinion or discussion?

Default: 0.5 (50% confidence)

Adjust when:

  • Responding to opinion questions → Increase
  • Missing factual questions → Decrease

Examples:

  • "What is our pricing?" ✓ (factual)
  • "What do you think about this approach?" ✗ (opinion)

3. Pre-Evaluation (0-1)​

What it checks: Before retrieving documents, how likely are we to find an answer?

Default: 0.5 (50% confidence)

Adjust when:

  • Attempting answers for topics outside training data → Increase
  • Not attempting enough questions → Decrease

4. Post-Evaluation (0-1)​

What it checks: After generating an answer, how good is it?

Default: 0.4 (40% confidence)

Adjust when:

  • Providing low-quality answers → Increase
  • Being too conservative (not answering good responses) → Decrease

Behavior Settings​

Post in Channel vs. DM​

Post in Channel (Default: OFF):

  • Answer appears in the channel for everyone
  • Good for: Transparency, shared knowledge
  • Risk: Public mistakes

Send as DM (Default: ON):

  • Answer sent as direct message to questioner
  • Good for: Testing, privacy, reducing noise
  • Risk: Others don't benefit from answer

Suggest Only (Default: OFF)​

When enabled:

  • Chat Aid asks "Would you like me to answer this?"
  • User confirms before getting answer
  • More conservative approach

When disabled:

  • Chat Aid answers directly
  • Faster, but less control

Answer Thread Messages (Default: ON)​

When enabled:

  • Auto-answer works on thread replies
  • Good for: Follow-up questions, detailed discussions

When disabled:

  • Only answers top-level messages
  • Threads are ignored

Allow AI Fallback (Default: OFF)​

When enabled:

  • If no answer found in training data, uses general AI knowledge
  • Risk: Potentially inaccurate information

When disabled:

  • Only answers from training data
  • Says "I don't know" if answer not found
warning

We recommend keeping AI Fallback disabled to ensure all answers come from your verified company data.

Auto-Enable for New Channels TeamBusinessEnterprise​

When enabled (Default: ON):

  • New Slack channels automatically get auto-answer enabled
  • Convenient for growing teams

When disabled:

  • You manually enable auto-answer for each new channel
  • More control

Monitoring Auto-Answer​

Viewing Logs​

  1. Go to Settings > Auto-Answer
  2. Click View Logs in the top-right
  3. Review all auto-answer activity

Understanding Logs​

Each log entry shows:

  • Message - The original question
  • Channel - Where it was asked
  • User - Who asked
  • Breakpoint - Where evaluation stopped
  • Answer - What Chat Aid responded (if anything)
  • Confidence Scores - All evaluation metrics

Breakpoints​

isQuestion:

  • Message didn't pass question detection
  • Likely not a question

isFactQuestion:

  • Detected as opinion or discussion question
  • Not suitable for factual answer

preEvaluation:

  • Low confidence in finding an answer
  • Topic might be outside training data

answerNotFound:

  • No relevant documents found
  • Need more training data on this topic

postEvaluation:

  • Answer generated but quality too low
  • Retrieved documents weren't relevant enough

answerFound:

  • ✅ Answer passed all checks and was sent

Using Logs to Improve​

High "isQuestion" failures:

  • Team writes questions in unusual ways
  • Consider lowering threshold

High "answerNotFound":

  • Missing training data
  • Connect more data sources
  • Update existing documentation

High "postEvaluation" failures:

  • Answers being generated but not confident enough
  • Review specific examples
  • Adjust threshold or improve data quality

Best Practices​

1. Start Conservative​

Week 1:

  • Enable for 1-2 channels
  • Use default settings
  • Monitor closely

Week 2:

  • Expand to 2-3 more channels
  • Adjust based on feedback

Week 3+:

  • Roll out to all appropriate channels
  • Fine-tune thresholds if needed

2. Choose the Right Channels​

Good Channels:

  • ✅ #support
  • ✅ #help
  • ✅ #general
  • ✅ #onboarding
  • ✅ Department channels with clear topics

Bad Channels:

  • ❌ #random (too conversational)
  • ❌ #off-topic (unrelated to work)
  • ❌ Leadership/sensitive channels
  • ❌ Channels with mostly opinions/discussions

3. Monitor Weekly​

Set a recurring reminder to:

  • Check auto-answer logs
  • Review feedback
  • Look for patterns in failures
  • Adjust settings as needed

4. Gather Feedback​

Ask team:

  • "Is Chat Aid helpful in #channel?"
  • "Are the automatic answers accurate?"
  • "Any channels where it's too noisy/quiet?"

Use feedback to adjust.

5. Keep Data Updated​

Auto-answer quality depends on training data:

  • Refresh data sources regularly
  • Update documentation when policies change
  • Remove outdated content
  • Add new documentation proactively

6. Set Expectations​

Communicate to team:

  • Which channels have auto-answer
  • That Chat Aid is still learning
  • How to provide feedback (👍/👎 reactions)
  • When to use @mention vs. waiting for auto-response

Troubleshooting​

Chat Aid Not Responding​

Check:

  1. Auto-answer is enabled globally
  2. Specific channel has auto-answer enabled
  3. Training data is complete
  4. Question is clear and factual
  5. Review logs to see which threshold failed

Too Many Responses​

Solutions:

  • Increase thresholds (be more conservative)
  • Disable for chatty/conversational channels
  • Enable "Suggest Only" mode

Not Enough Responses​

Solutions:

  • Decrease thresholds (be less conservative)
  • Add more training data
  • Check logs to see where questions fail

Wrong Answers​

Solutions:

  • Improve training data quality
  • Remove outdated documents
  • Increase post-evaluation threshold
  • Consider disabling AI fallback

Channel Not Listed​

Solutions:

  • Refresh the page
  • Check Chat Aid is added to the channel
  • For private channels, ensure Chat Aid is a member
  • Verify Slack connection is active

FAQ​

Does auto-answer work in DMs?​

No, auto-answer only works in channels. DMs always require explicit messaging to Chat Aid.

Can I set different thresholds per channel?​

Not currently, but this is on our roadmap.

What happens if multiple people ask the same question?​

Chat Aid will answer each one independently based on thresholds.

Can I disable auto-answer temporarily?​

Yes, toggle it off in Settings > Auto-Answer. This immediately stops all automatic responses.

Does auto-answer count toward my question limit?​

Yes, auto-answered questions count the same as manually asked questions.

Can I exclude specific users?​

Not currently. Auto-answer applies to all messages in enabled channels.

Ready to enable auto-answer?

Need Help?

Our support team is here to help you get the most out of Chat Aid.