Building Reliable AI Systems: Workflows vs. Agents

Building Reliable AI Systems: Workflows vs. Agents

The rapid proliferation of AI has led to widespread discussions and tutorials on building AI agents. However, many companies struggle to implement effective AI features within their products. This raises a fundamental question: What truly constitutes an AI agent, and how can developers build reliable AI systems?

Defining AI Systems: Workflows vs. Agents

It is crucial to distinguish between different types of AI systems. While many online resources equate AI agents with simple systems that make API calls to large language models (LLMs), experts argue that true AI agents are far more sophisticated. Anthropic's distinction between workflows and agents provides a helpful framework:

• Workflows: These systems orchestrate LLMs and tools through predefined code paths. They involve a series of steps with a call to an LLM at some point.
• Agents: These are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish a task.

As a developer, understanding when to apply each pattern is crucial. The term "agent" is often used loosely, leading to the misconception that complex tools and frameworks are always necessary. For many applications, simpler, predefined workflows are sufficient and more effective.

Core Components of Effective AI Systems

Regardless of whether you're building a workflow or an agent, certain building blocks are fundamental:

1. Augmented LLM: Enhancing a basic LLM call through:
   • Retrieval: Pulling information from external sources (e.g., databases, vector databases) to provide context to the LLM. Retrieval Augmented Generation (RAG) is a common technique here.
   • Tools: Integrating external services or APIs to access additional information, such as weather data or shipping updates.
   • Memory: Incorporating past interactions with the LLM to maintain context and improve responses.
2. Prompt Chaining: Linking multiple LLM calls, using the output of one call as input for the next. This breaks down complex problems into manageable steps, improving control and accuracy.
3. Routing: Using the LLM to categorize incoming requests and direct them to the appropriate function or workflow. This is particularly useful when dealing with multiple scenarios or solutions.
4. Parallelization: Making multiple independent LLM calls simultaneously to speed up processing. This is useful for tasks that can be divided into independent sub-tasks, such as evaluating different aspects of an output.
5. Orchestrator Worker: Using an LLM to assess a situation (e.g., a customer email) and determine the necessary actions, such as looking up information in a customer care playbook or calling a shipping API.
6. Evaluator Optimizer: Using one LLM to generate an output, another to review and provide feedback, and a third to improve the output based on the feedback.

The Agent Pattern: Sophistication and Challenges

True AI agents operate in a loop, making decisions, taking actions, assessing the results, and adjusting their approach until they achieve a goal or meet a stopping criterion. Unlike workflows, agents do not have a predefined endpoint. While agentic systems can handle sophisticated tasks, achieving reliable results is challenging.

Practical Tips for Building AI Systems

• Be Cautious with Agent Frameworks: While they offer a quick start, ensure you understand the underlying mechanisms and consider building core components from scratch for better control.
• Prioritize Deterministic Workflows: Start with simple, isolated problems and build complexity gradually. Optimize workflows to achieve near-perfect accuracy before scaling.
• Account for Scaling Challenges: Scaling AI applications can introduce unexpected issues like hallucinations. Implement proper testing and evaluation systems.
• Implement Guardrails: Perform checks on outputs before sending them to users to protect your brand and reputation.

Building effective and reliable AI systems requires a deep understanding of the different patterns and tools available. By starting simple, prioritizing control, and implementing robust testing and evaluation processes, developers can create AI solutions that deliver real value.

Which approach—workflow or agent—best suits the specific challenge at hand?

Comments are closed.