Building Your Enterprise AI Chatbot Compass

# Driving Internal Innovation and Departmental Productivity

Chatbots and virtual agents powered by generative AI (GenAI) are redefining how businesses connect with customers, streamline processes, and improve productivity. These conversational tools are becoming integral to modern enterprises, offering opportunities for efficiency and innovation.

However, as with any technological revolution, the path to long-term enterprise value comes with challenges. The rapid development of AI, compared to traditional tech, introduces new uncertainties that require careful navigation, like adopting platforms that quickly become obsolete or siloed initiatives that lead to tech sprawl. Companies are grappling with difficult questions: How can we embrace this technology without locking into choices that might soon be outdated? How do we balance experimentation with the need for scalable, sustainable solutions?

As organizations move from experimenting with tools and pilot programs to more coordinated use cases across departments, they need a reliable framework to guide their decisions. We call this framework the Enterprise AI Chatbot Compass—a strategic tool to steer AI implementation and keep businesses focused on what really matters: long-term value, cost management, and departmental productivity.

Related Content: Leader’s Guide to Building AI Acumen and Maximizing Business Impact

# The Enterprise Chatbot and Virtual Assistant Landscape

Despite AI's hype, turning its promise into reality is not guaranteed. According to Gartner, nearly half of AI project leaders report not seeing the value they expected from their investments, and 90% of organizations implementing AI projects aren't achieving their anticipated outcomes.

While many organizations struggle to scale AI efforts enterprise-wide, AI remains early in its technology curve. Large and small language models (LLMs and SLMs) are entering the market, each offering specific context windows and use cases. And each day, more and more platforms are introduced, adding to decision fatigue and uncertainty.

Companies must navigate complex decisions about building in-house AI solutions or leveraging external platforms. The continuous evolution of AI compounds this challenge, making roadmaps unstable and complicating long-term planning.

# The Emerging Chatbot Ecosystem: From Personal Tools to Enterprise Solutions

Organizations are adopting chatbots to meet various needs within their internal operations at different altitudes. Internal chatbots and virtual assistants serve distinct purposes, from personal productivity enhancers to enterprise-level solutions that enhance knowledge access and streamline operations.

Organizations face increasing ambiguity as they contemplate various technology stacks to solve use case needs at different altitudes. User design, technology choice, legal approvals, governance, and orchestration will look different at various altitudes, and effective AI governance is key to managing sprawl, chatbot performance, and enterprise risk.

In this blog, we’ll aim to offer a few guiding principles at the enterprise level.

# Building Your Chatbot Compass

As AI advances, companies face the challenge of frequently updating their AI and business strategies. This constant flux can be both a catalyst for innovation and a disruptor of established business models.

Achieving enterprise value with AI demands more than technology. Organizations must select and prioritize technology platforms and patterns aligned with top use cases, balance bottoms-up experimentation with top-down strategic decision-making, provide visibility into what’s happening in the organization, and create the right governance and oversight to mitigate risks.

To navigate this complexity, organizations must rely on their Chatbot Compass — a set of guiding principles that ensures its enterprise AI strategy stays agile and value-driven.

Across our clients, we’ve seen the benefit of anchoring around three core principles to guide organizations to the right-sized investment during this time of rapid innovation.

1. Build Good Use Cases
2. Anchor in a Product Mindset
3. Align on Preferences and Guardrails

# 1. Build Good Use Cases

While many organizations prioritize experimentation to learn and grow their teams’ skills, these efforts can sometimes lead to limited results. To avoid this, businesses must carefully calculate returns, consider ongoing maintenance, and focus on strategic use cases with clear business value. Here are a few key principles to guide use case qualification:

1. Start with a Proof of Concept: Like any software or business solution, start with a clear hypothesis— “By automating responses to these emails, we’ll save 30 minutes of each team member’s time each week”—and identify the easiest path to an MVP to test it. Consider only using a subset of data, knowledge material, or team processes rather than boiling the ocean. This will help with attribution and will uncover blind spots during development. Once you’ve proven your hypothesis, you can expand the scope and make the case for further scale and investments.
2. Adopt an 80/20 Mindset: Focus on identifying the 20% of use cases, tasks, or intents that will drive 80% of the value. For example, if you’re a support function looking to build an end-user chatbot, rather than train the bot on 20,000 support articles, focus on the top support volume drivers. Train the bot to get very good at driving deflection on those questions before expanding the scope.
3. Set Measurable ROIs: Define clear, easily quantifiable success metrics and a measurable improvement target, such as a 50% self-serve deflection rate or 20% reduced response time. Establishing a baseline Key Performance Indicators (KPI) for each business function’s goals (or Key Experience Indicators on the user-facing side) will help show attribution and ROI. Organizations must consider the costs, such as software licenses, development overhead, integration, and training, to accurately assess ROI. If an attributable change in key metrics doesn’t become evident within 6-12 months, you may want to return to the drawing board.

Related Content: How to Develop a Winning AI Strategy: A Step-By-Step Guide

# 2. Anchor in a Product Mindset

Many organizations prioritize opportunities for employees to experiment with AI capabilities, allowing them to test new tools and identify valuable patterns. However, to ensure chatbots remain productive, they must be treated as long-term products that require ongoing monitoring, maintenance, and improvement.

Here are some ways to take a product mindset:

1. Prioritize User Experience: Solutions should solve a specific user need and be designed with user behaviors in mind. Partner with UX researchers to design chatbot products with users at the center and set Key Experience Indicators to track user experience. The entire user journey—from problem identification to resolution—should be considered. For example, if most employees search for information on an intranet homepage, consider building a bot directly into the page rather than outside the user flow. Let users know the scope and limitations of the bot to avoid aversion and provide some prompts to help them understand how to engage and what types of questions they can ask.
2. Plan for Long-term Maintenance: Many teams are excited to build and experiment with chatbots. Before investing heavily in certain solutions, consider long-term maintenance and monitoring. These “costs” should be considered when calculating your ROI and tracked to evaluate the solution's effectiveness. Without this mindset, chatbots may offer diminishing returns over time. Rather than freeing up developer and team resources, they may just shift ownership from one team to another.
3. Establish a Community of Practice: Given that most organizations will be in experimentation mode for a while as they decide on preferred solutions, establishing communities of practice will be key to advancing the organization’s shared understanding and identifying reusable patterns. Resource hubs where business functions, developers, and architects can share their stories, use cases, and patterns; Slack channels for chatbot engineers to ask questions and share information; and sandbox environments like Microsoft Copilot Studio can all contribute to effective experimentation and collective AI literacy that can later be codified into best practices.

# 3. Align on Preferences and Guardrails

The pace of change for Natural Language Processing (NLP), permissions-aware RAG models, and AI-embedded native apps will continue for the foreseeable future, with rapid release cycles and frequent upgrades. When building AI-driven chatbots, it’s crucial to design systems that allow for flexibility—whether to swap out models or experiment with new platforms.

To prevent tech sprawl and restrictive architecture, follow a couple of principles:

1. Orient Towards Preferred Platforms with Some Flexibility: Given the velocity of technology change, organizations face balancing costs and technology sprawl with the interest and need for onboarding new technology for niche use cases. Bring together architects, developers, business owners, and legal and compliance colleagues to identify a few preferred technology platforms that can meet 80% of use cases while leaving room for edge cases. Organizing these by complexity and use case type can help teams choose preferred technology from no-code to fully customizable configurations. Preferred platforms can be reviewed periodically to ensure they meet business needs and new use cases.
2. Develop Guardrails Where They Matter: There are some circumstances where organizations will want to create more rigidity, like when it comes to security permissions, ethical reviews, data loss, and other risks. Not all use cases pose the same risk (e.g., using an approved GPT to perform tasks with public information vs. building custom with PII data). Establishing risk-weighted use case reviews that run through a central team can provide the right level of oversight. In the same vein, convening cross-functional teams to collaborate on enterprise solutions for functions like HR, IT, and Finance can help manage long-term costs and coordinate end-user experiences, minimizing sprawl and process inefficiency.

# Conclusion: Navigating Your AI Chatbot Journey

The journey to successful AI chatbot implementation is complex, but it’s also filled with opportunity. By building your Chatbot Compass – building good use cases, anchoring in a product mindset, and aligning on preferences and guardrails– you can move through these challenges and unlock AI’s transformative potential for your organization.

At Propeller, we understand that AI is more than just technology; it’s about aligning tools with your broader business goals. Whether you’re focusing on improving customer engagement, operational efficiency, or strategic innovation, our experience in AI strategy and enablement ensures your chatbot initiatives deliver long-term value and measurable ROI.

By applying these principles, you can ensure that your chatbot initiatives are not just reactive but deliver tangible business value.

As AI capabilities continue to evolve, the decisions you make today will define your success tomorrow. From optimizing data management to choosing the right platforms, being strategic in your approach now will position you to thrive in the AI-driven future.

Are you ready to take the next step? Let’s work together to chart your course and build a future where your chatbot initiatives are not just reactive but transformative.