Component Features

Key Concepts

Concept	What it Means
RAG (Retrieval-Augmented Generation)	Combining document retrieval with AI generation for accurate, source-backed answers
Multi-Tenancy	Complete data isolation between different enterprise environments (e.g., DEV, SBX)
LangGraph	The graph-based processing engine that routes and handles user requests
Intent Routing	Automatic classification of user queries to the right processing pipeline
FAISS	Vector database used for fast semantic document search
Process Node	A specialized unit within LangGraph that handles a specific task type (SQL, code, summary, etc.)
JWT Token	Authentication token that also carries tenant/environment information
Embedding	A numerical representation of text used for semantic similarity search

---

Database Layer & Multi-Tenancy

Complete data isolation between tenants while maintaining shared application logic and optimal resource utilization.

Multi-tenant implementation:

• Dynamic Engine Creation — Automatic database connection setup per tenant
• Connection Pooling — Efficient resource management across environments
• Schema Isolation — Separate database schemas per tenant environment
• Configuration-Driven — Database connections configured through discovery service

Key capabilities:

• Automatic tenant detection from JWT tokens
• Dynamic schema caching for performance optimization
• Connection pool management with automatic cleanup
• Support for multiple HANA database environments

Authentication & JWT Handling

Stateless JWT-based authentication with tenant information embedded in tokens for seamless multi-tenant access control.

JWT token carries:

• User identification and roles
• Tenant/environment mapping
• Session metadata and preferences
• Security permissions and access levels

Authentication flow:

1. JWT token validation and parsing
2. Tenant environment extraction
3. Database connection routing
4. User context establishment

Model Integration (SAP AI Core)

Enterprise-grade LLM access through SAP's AI infrastructure, ensuring compliance, security, and cost optimization.

SAP AI Core benefits:

• Enterprise Compliance — Data governance and security standards
• Cost Management — Optimized model usage and billing
• Model Variety — Access to multiple LLM providers and models
• Performance Monitoring — Built-in usage analytics and monitoring

Graph-Based Processing Pipeline (LangGraph)

Architecture Overview

LangGraph enables sophisticated, multi-step AI processing workflows that intelligently route requests to specialized processing nodes based on user intent and context.

Core benefits:

• Intelligent Routing — Automatic determination of the best processing path
• Stateful Processing — Maintains context throughout multi-step operations
• Extensible Design — Easy addition of new processing capabilities
• Error Recovery — Built-in fallback mechanisms and error handling

Each user query flows through a directed graph of specialized nodes, ensuring optimal processing for different types of requests (documents, SQL, code, etc.).

Processing Node Categories

Node Category	What It Does
Routing Nodes	Intent classification and request direction
Generation Nodes	Content creation (SQL, code, summaries)
Integration Nodes	External service connectivity
Processing Nodes	Document analysis and data manipulation
Utility Nodes	Formatting and error handling

Intent Routing System

Automatically determines user intent and routes requests to the most appropriate processing pipeline.

Intent categories:

Intent	Description
Document Queries	Questions about uploaded or SharePoint documents
Data Analysis	SQL generation and database querying
Code Generation	Programming assistance and code creation
Summarization	Content analysis and executive summaries
Scheduling	Calendar and meeting management
General Chat	Conversational AI interactions

💡 How it works: The router uses advanced NLP techniques to analyze user input and context to make intelligent routing decisions — users don't need to specify what type of request they're making.

---

Processing Nodes

Intent Router Node

Acts as the intelligent dispatcher that analyzes user requests and determines the optimal processing path.

Capabilities:

• Natural language intent classification
• Context-aware routing decisions
• Multi-intent request handling
• Confidence scoring and fallback routing

Hybrid Q&A System (RAG)

Combines retrieval-augmented generation with semantic search to provide accurate, contextual answers from enterprise document collections.

Core RAG components:

Component	Function
Document Ingestion	Multi-format document processing (PDF, DOCX, images)
Vector Storage	FAISS-based semantic indexing for fast retrieval
Semantic Search	Advanced similarity matching for relevant content
Context Assembly	Intelligent combination of retrieved content with user queries
Response Generation	LLM-powered answer synthesis with source attribution

Advanced features:

• Document-specific filtering and relevance scoring
• Hybrid search combining semantic and keyword approaches
• Multi-modal processing including image and text analysis
• Automatic document summarization and metadata extraction

SQL Generator

Converts business questions into optimized database queries, democratizing data access across the organization.

Business value:

• Data Democratization — Non-technical users can query enterprise databases
• Query Optimization — Automatically generated efficient SQL queries
• Schema Awareness — Understands database structure and relationships
• Security — Built-in query validation and access control

Technical features:

• Dynamic schema analysis and caching
• Query validation and security checks
• Multi-table join optimization
• Result formatting and visualization preparation

Document Processing

Advanced document analysis combining OCR, semantic understanding, and structured data extraction.

Processing capabilities:

• Multi-format document ingestion and parsing
• Automatic content categorization and tagging
• Key information extraction and structuring
• Document similarity analysis and clustering

Integration features:

• SharePoint synchronization and management
• Version control and change tracking
• Automated compliance and policy checking
• Searchable document indexing

MCP Dynamic Node

Enables dynamic integration with external tools and services through standardized Model Context Protocol communication.

Benefits:

• Extensible tool integration without system modifications
• Real-time external service connectivity
• Standardized communication protocols
• Scalable third-party integrations

---

API Endpoints & Routers

Chat Management APIs

Comprehensive APIs for managing multi-turn conversations with context preservation and intelligent routing.

Endpoint Group	Purpose
Chat Creation	Initialize new conversations with context setup
Message Processing	Handle user messages with intent routing
Context Management	Retrieve and update conversation history
Chat Naming	Automatic and manual conversation labeling

Document Upload & Processing APIs

Secure document upload, processing, and integration with the RAG system for intelligent querying.

Core capabilities:

• Multi-format upload support (PDF, DOCX, images, structured data)
• Automatic processing: document parsing, chunking, and vector indexing
• Metadata extraction: automatic document classification and tagging
• Seamless integration with existing document management systems

Security features:

• File type validation and security scanning
• Per-tenant document isolation
• Access control and permission management
• Audit trail for document operations

Enhanced Q&A APIs

Sophisticated APIs providing document-specific querying with advanced filtering and relevance scoring.

Advanced features:

• Document-specific query filtering
• Confidence scoring and source attribution
• Multi-document cross-referencing
• Context-aware follow-up question handling

Smart Query Interface

Advanced query processing with automatic optimization and intelligent result presentation.

Query intelligence:

• Automatic query optimization and enhancement
• Intent-aware result formatting
• Multi-modal query support (text, voice, image)
• Predictive query suggestions

Tracing & Monitoring APIs

Comprehensive monitoring and tracing APIs for performance optimization and issue resolution.

Monitoring capabilities:

• Real-time performance metrics
• Request tracing and debugging
• Error tracking and analysis
• Usage analytics and reporting

Configuration Management APIs

APIs for managing system configurations, prompts, and tenant-specific settings without system restarts.

Configuration management:

• Runtime configuration updates
• Tenant-specific customization
• Prompt template management
• Feature flag control

---

Services Layer

Chat Services

Sophisticated service layer managing multi-tenant conversations with intelligent context preservation and routing.

Service Component	Purpose
Chat Service Manager	Orchestrates conversation flow and state management
Context Management	Maintains conversation history and user preferences
Session Manager	Handles user sessions and authentication state
Auto-Naming Service	Intelligent conversation title generation

Document Q&A Service

RAG-powered document question-answering service delivering accurate, source-attributed responses.

Core RAG implementation:

• Embedding Generation — High-quality semantic embeddings for document chunks
• Vector Search — FAISS-powered similarity search with optimization
• Context Assembly — Intelligent combination of relevant document segments
• Response Synthesis — LLM-powered answer generation with source citations

Enterprise features:

• Multi-tenant document isolation and security
• Advanced document filtering and relevance scoring
• Real-time document synchronization and updates
• Comprehensive audit trails for compliance

SharePoint Service

Seamless integration with SharePoint for automated document synchronization and collaborative document management.

Integration capabilities:

• Automatic document synchronization and indexing
• Real-time change detection and processing
• Metadata preservation and enhancement
• Collaborative filtering and access control

Schema Extraction Service

Dynamic database schema analysis and caching service enabling intelligent SQL generation and data querying.

Schema intelligence:

• Automatic table relationship discovery
• Column type analysis and constraint detection
• Performance optimization through intelligent caching
• Multi-tenant schema isolation and management

Evaluation Service (DeepEval)

Comprehensive response quality evaluation using the DeepEval framework for continuous improvement.

Quality metrics:

• Response accuracy and relevance scoring
• Factual consistency validation
• Bias detection and mitigation
• Performance benchmarking and trend analysis

Feedback Management

Comprehensive feedback collection, analysis, and system improvement orchestration.

Feedback processing:

• Automated feedback categorization and analysis
• Trend identification and reporting
• System improvement recommendation generation
• User satisfaction tracking and reporting

---

Utilities & Support

Session Management

Capability	Description
Secure Sessions	Session creation and validation
Context Preservation	Cross-session context retention
User Preferences	Preference management per user
Multi-Device Sync	Session synchronization across devices

Performance Monitoring

Real-time system observability providing comprehensive insights into system performance and user experience.

Monitoring features:

• Real-time performance metrics and alerting
• Request tracing and bottleneck identification
• Resource utilization tracking
• Predictive performance analysis

Document Loaders

Multi-format document ingestion system with intelligent content extraction.

Supported formats:

Format	Capability
PDF	Full text extraction with OCR
DOCX	Microsoft Word document parsing
PPTX	PowerPoint presentation support
XLSX	Excel spreadsheet processing
Images	Text extraction via vision models

Vision & Image Handling

Advanced image analysis and vision capabilities for comprehensive document understanding.

Vision capabilities:

• OCR for text extraction from images
• Document structure recognition
• Chart and diagram analysis
• Multi-modal content understanding

Data Masking

Intelligent data masking system ensuring sensitive information protection while maintaining system functionality.

Protection features:

• PII detection and automated masking
• Customizable masking rules per tenant
• Compliance framework support (GDPR, CCPA)
• Audit trails for data access and processing

Caching Mechanisms

Multi-layer caching system optimizing response times and resource utilization.

Cache Layer	What It Caches
Configuration Cache	Tenant configs for rapid access
Schema Cache	Database schemas for query optimization
Embedding Cache	Vector embeddings for RAG performance
Response Cache	Answers for frequently asked questions

---

Data Management

Database Schema

Multi-tenant data architecture ensuring complete tenant isolation while maintaining optimal performance.

Schema design principles:

• Per-tenant database isolation with shared application logic
• Optimized indexing for conversational AI workloads
• Comprehensive audit trails and compliance support
• Scalable schema evolution and migration support

Embedding System (FAISS)

High-performance vector storage enabling lightning-fast semantic search across enterprise document collections.

RAG vector architecture:

Component	Details
Embedding Generation	State-of-the-art semantic embeddings via sentence transformers
Vector Indexing	Optimized FAISS indices for sub-second search performance
Similarity Search	Advanced similarity algorithms with configurable thresholds
Index Management	Automatic index optimization and maintenance

Performance features:

• Per-environment vector store isolation for multi-tenancy
• Incremental index updates for real-time document additions
• Compressed storage with optional zstandard compression
• Thread-safe operations with per-environment locking

Chat History Management

Comprehensive chat history management enabling context-aware conversations and conversation analytics.

History features:

• Complete conversation threading and context preservation
• Intelligent conversation summarization and key point extraction
• Advanced search capabilities across conversation history
• User preference learning and personalization

Document Storage

Robust document storage system with intelligent organization, versioning, and access control.

Storage capabilities:

• Multi-format document storage with metadata preservation
• Version control and change tracking
• Automated backup and disaster recovery
• Integration with existing enterprise storage systems

---

Integrations

SAP AI Core

Deep integration with SAP AI Core providing enterprise-grade AI capabilities with governance and compliance.

Benefit	Description
Enterprise Compliance	Built-in data governance and security standards
Model Access	Latest LLM models through SAP's infrastructure
Cost Optimization	Intelligent model usage and cost management
Performance Monitoring	Comprehensive usage analytics and optimization

Technical implementation:

• Automatic token management and refresh mechanisms
• Load balancing across multiple model endpoints
• Fallback strategies for high availability
• Cost tracking and budget management

SharePoint Integration

Seamless SharePoint integration enabling automatic document synchronization and collaborative content management.

SharePoint benefits:

• Automatic Synchronization — Real-time document updates and processing
• Metadata Preservation — Complete metadata and permission preservation
• Collaborative Features — Multi-user document management and sharing
• Version Control — Comprehensive version tracking and history management

LangSmith Tracing

Comprehensive tracing system providing deep insights into AI processing workflows and performance optimization.

Tracing capabilities:

• End-to-end request tracing across all processing nodes
• Performance bottleneck identification and optimization
• AI model performance monitoring and analysis
• Debugging support for complex AI workflows

External API Connections (MCP)

Robust framework for connecting with external APIs and services through standardized protocols.

Integration framework:

• MCP (Model Context Protocol) support for tool integration
• RESTful API connectivity with automatic retry mechanisms
• Webhook support for real-time event processing
• API rate limiting and usage monitoring

---