Skip to main content
This guide demonstrates how to build sophisticated RAG (Retrieval-Augmented Generation) workflows using the Linq Protocol, starting from basic quote generation to advanced semantic search and storage systems.

Initial Setup

Create Milvus Collection

Before building workflows, you need to create a Milvus collection for storing quotes with embeddings:
This collection setup is a one-time operation. Once created, it will be available for all your RAG workflows.

Basic Historical Saying Generator

Create Workflow Definition

Start with a simple 2-step workflow that generates inspirational quotes from historical figures:
The system automatically adds several fields when saving the workflow:
  • id: Auto-generated MongoDB ObjectId
  • teamId: Set from current user’s team context
  • workflowId: Added to query (same as document ID)
  • params.teamId: Added to query parameters for team context
  • createdAt/updatedAt: Timestamps in MongoDB format
  • createdBy/updatedBy: Username of creator/updater
  • version: Starts at 1 for new workflows
  • public: Boolean flag for workflow visibility

Execute the Workflow

Once created, execute the workflow using the returned ID:
This basic workflow demonstrates:
  • Step Results: Each step shows actual results from services
  • Final Result: Clean quote text extracted from OpenAI response
  • Metadata: Detailed execution information including duration and token usage
  • Caching: Step 1 uses caching to avoid repeated API calls

Advanced RAG Workflow

Historical Figure Data Storage

This workflow demonstrates RAG preparation by storing historical figure data in Milvus for future semantic search:
This workflow builds a knowledge base over time. Each execution adds a new historical figure to the Milvus collection for future semantic search operations.

Enhanced RAG Workflow with Smart Retrieval

The most advanced workflow combines retrieval and generation intelligently:
This advanced workflow demonstrates:
  • Smart Retrieval: Finds existing relevant quotes before generating new ones
  • Diversity: Uses quotes from different people to avoid repetition
  • Quality Assurance: Reuses proven, existing quotes when available
  • Language Detection: Automatically detects and tags quote language
  • Comprehensive Storage: Stores results with rich metadata for future use

Workflow Execution Results

Basic Workflow Response

Here’s the complete execution response from the basic workflow:

Advanced RAG Workflow Response

Here’s the complete execution response from the advanced RAG workflow:
Key RAG Features Demonstrated:
  • Smart Retrieval: Found existing relevant quote instead of generating new one
  • Diversity: Used quote from Karl Marx instead of Atticus Finch to avoid repetition
  • Quality: Reused high-quality existing quote rather than generating potentially inferior new one
  • Storage: Still stored the result for future reference
  • Language Detection: Automatically detected and tagged the quote language
  • Metadata: Added comprehensive tags and categorization

Key Features

Step-by-Step Execution

Each workflow step provides detailed results:
  • Step Results: Actual output from each service
  • Execution Metadata: Duration, token usage, and timestamps
  • Error Handling: Clear status indicators for each step
  • Caching: Configurable caching to optimize performance

RAG Integration

The workflows demonstrate full RAG capabilities:
  • Semantic Search: Find relevant existing content
  • Intelligent Generation: Create new content when needed
  • Knowledge Accumulation: Build databases over time
  • Quality Optimization: Balance between retrieval and generation

Performance Optimization

  • Caching: Reduce API calls with configurable TTL
  • Token Management: Track and optimize OpenAI usage
  • Async Support: Handle long-running operations
  • Team Isolation: Multi-tenant data security

Best Practices

Workflow Design

  1. Start Simple: Begin with basic workflows and add complexity gradually
  2. Use Caching: Enable caching for frequently accessed data
  3. Error Handling: Plan for service failures and timeouts
  4. Team Context: Always include teamId for multi-tenant isolation

RAG Implementation

  1. Collection Design: Plan your Milvus schema carefully
  2. Embedding Strategy: Choose appropriate embedding models
  3. Search Optimization: Configure similarity thresholds
  4. Data Quality: Ensure high-quality training data

Performance Monitoring

  1. Token Tracking: Monitor OpenAI usage and costs
  2. Execution Times: Track step durations for optimization
  3. Cache Hit Rates: Measure caching effectiveness
  4. Error Rates: Monitor workflow success rates
These workflows provide a foundation for building sophisticated AI applications. Start with the basic workflow and gradually implement the advanced features as your needs grow.