#Tech - Tech Stack

Website:

• Next JS 15+ / Typescript / Tailwind CSS

• Google Cloud Console for web access

• MongoDB / cosmos DB through Azure

• Dedicated RPC node

• Helius back-up RPC node

• Hosted on Vercel Serverless (Lightweight)

Telegram Bot

• Typescript / node.js server

• PostgreSQL database

• Weaviate Vectorization

• Dedicated RPC node

• Helius back-up RPC node

• Google Cloud Console for web access

• Hosted on Railway.app (Lightweight)

Morris' Brains c.q. Agentic Layer

• Python 3.+

• Tensorflow

• R for analytics

• Azure Environment

In a complex data processing workflow, breaking down tasks into specialised agents significantly enhances efficiency, particularly for large-scale, multi-step processes. By leveraging an agentic architecture, each agent handles a specific task—such as data ingestion, profiling, knowledge graph creation, or query orchestration—independently, which reduces latency and enables parallel processing.

For example, the Profiling, Knowledge Graph, and Embedding Generation Agents can work simultaneously to prepare data, while the Model Inference and Query Processing Agents handle query-specific tasks in sequence. This agent-based structure not only improves processing speed but also allows for real-time system optimization through continuous monitoring by a dedicated Monitoring and Logging Agent. Altogether, this approach streamlines the end-to-end process, providing users with faster, accurate responses and an overall optimized data interaction experience.

Throughout his agentic layer, he will use proven algorithms and methods in order to train himself:

• Reinforcement training: https://www.ibm.com/topics/reinforcement-learning

• K-means: https://www.javatpoint.com/k-means-clustering-algorithm-in-machine-learning

• Predictive models:

Agent-Based Architecture

1. Ingestion Agent

   1. Ingest data

   2. Trigger notifications to subsequent agents

2. Profiling Agent

   1. Run Great Expectations profiling on ingested data in Cosmos DB

   2. Identify schema elements, data types, missing values, and basic data distributions

   3. Store profiling metadata for quick access

3. Knowledge Graph Agent

   1. Use profiling data to define entity relationships in Cosmos DB Knowledge Graph

   2. Update or create a dynamic knowledge graph with inferred relationships

   3. Alert the Query Orchestration Agent of any schema changes.

4. Embedding Generation Agent

   1. Generate semantic embeddings using Weaviate based on profiling metadata

   2. Store embeddings for column names and values in Cosmos DB for fast semantic retrieval

   3. Notify the Query Orchestration Agent once embeddings are updated

5. Model Training Agent

   1. Train or fine-tune company-specific models using Azure Machine Learning based on data and relationships

   2. Regularly check for data updates, retrain models as needed, and version new models in Azure ML Model Management

   3. Notify the Model Inference Agent of any model updates

6. Model Inference Agent

   1. Load the most recent Morris model from Azure ML Model Management

   2. Interpret and translate user queries based on the trained model

   3. Coordinate with the Query Orchestration Agent to ensure query requests align with the model’s understanding of the data

7. Query Orchestration Agent

   1. Translate natural language queries into optimised queries using GraphQL

   2. Retrieve column metadata from the Profiling Agent and embeddings from the Embedding Generation Agent

   3. Use the Knowledge Graph to understand relationships and direct queries to the correct data collections

   4. Send refined queries to the Query Processing Agent

8. Query Processing Agent

   1. Execute complex queries using Apache Spark on Azure and Presto on Azure

   2. Perform aggregations and calculations, fetching results in the desired format

   3. Send processed results to the Response Generation Agent

9. Response Generation Agent

   1. Receive processed results and convert them into natural language responses via the Morris API

   2. Format responses for readability and contextual accuracy, adjusting tone and detail as per the user’s question

   3. Send final responses to user

10. Monitoring and Logging Agent

    1. Track performance metrics, response times, and error rates for each agent using Azure Monitor & Log Analytics

    2. Detect bottlenecks or failures and optimise resource allocation across agents

    3. Trigger alerts or corrective actions if latency issues are detected