1_PrivateArchitecture.md

Demonstration: Building a Private ChatBot with Azure OpenAI

Costa Rica

Cloud2BR OSS - Learning Hub

Last updated: 2026-04-06

---

This demo is about how to setup secure document searches within a designated network, ensuring that your data remains protected while leveraging the capabilities of Azure OpenAI.

You could use the RAG pattern to improve the search experience in your web application. For instance, when a user queries the search system, it can retrieve relevant documents from Azure Storage Blob Containers and use the retrieved information to generate a more accurate and detailed search result. Click here for more information about RAG.

List of References (Click to expand)

• Create a private endpoint for a secure connection to Azure AI Search

Table of Contents (Click to expand)

• Overview
• Create an Azure OpenAI Resource
• Set Up Azure AI Search
• Integrate with Virtual Network VNet
• Configure Private Endpoints for Azure AI Search
• Configure Private Endpoints for Azure OpenAI
• Set Up Network Security Groups NSGs
• Create index/Upload Documents
• Configure and Deploy AI model

Overview

same/different vnet -> shared access can be used
same vnet -> private endpoint
different vnet -> needs vnet peering

Click here to see more about Workflow in Zero Trust Architecture

Create an Azure OpenAI Resource

• Sign in to the Azure portal.

• Navigate to Create a resource and search for Azure OpenAI.

• Configure the Resource: Follow the prompts to configure the resource

  • Subscription: Choose your subscription.
  • Resource Group: Create a new resource group or select an existing one.
  • Region: Choose the region closest to your users.
  • Name: Provide a unique name for your Azure OpenAI resource.

• Review and Create: Review your configuration and select Create.

  

Set Up Azure AI Search

• Create Azure AI Search Resource: In the Azure portal, create a new Azure AI Search resource.

• Configure Search Service: Provide the necessary details

  • Name: Enter a name for your search service.

  • Resource Group: Use the same resource group as your Azure OpenAI resource.

  • Location: Use the same region for reduced latency.

  • Pricing Tier: Select a pricing tier based on your needs.

    

• Establish the network connection by choosing to either set up the resource with a public configuration and adjust the network settings later, or integrate the network configuration during the resource creation process.

  

  Option	Description	Use Case
  All networks	Allows access from any public IP address.	Testing or development environments where security is not a primary concern.
  Selected IP addresses	Restricts access to specified public IP addresses.	Scenarios where you know the IP addresses of the clients that need to connect.
  Disabled	Disables public network access entirely.	Resources that should only be accessed from within a virtual network or through private endpoints.

Note

About the exception checkmark Allow Azure services on the trusted services list to access this search service:
This setting allows trusted Azure services to bypass the network rules and access your resource directly. These include services like Azure Backup, Azure Site Recovery, and others that are part of the trusted services list. Even with this setting enabled, proper authentication is still required to access the resource, such as Managed Identity or Service Principal. Only resources within the specified IP address ranges or virtual networks will have access. Resources from other tenants or subscriptions will not have access unless they are explicitly granted access through the whitelist or fall under the allowed exceptions. This setting is particularly useful for scenarios where you want to allow Azure Site Recovery to access your search service for disaster recovery purposes without needing to configure additional network rules.

Integrate with Virtual Network (VNet)

Deploy your VMs, Azure AI Search, and Azure OpenAI within the VNet.

graph TD
    subgraph VNet["Virtual Network"]
        direction TB
        VM["VM"]
        SearchService["AI Search"]
        OpenAI["OpenAI"]
        NSG["NSG"]
        Subnet["Subnet"]
        NIC_VM["NIC VM"]
        NIC_SearchService["NIC AI Search"]
        NIC_OpenAI["NIC OpenAI"]
        PE_VM["Private Endpoint VM"]
        PE_SearchService["Private Endpoint AI Search"]
        PE_OpenAI["Private Endpoint OpenAI"]
        
        VM --> NIC_VM
        NIC_VM --> NSG
        NIC_VM --> Subnet
        NIC_VM --> PE_VM
        
        SearchService --> NIC_SearchService
        NIC_SearchService --> NSG
        NIC_SearchService --> Subnet
        NIC_SearchService --> PE_SearchService
        
        OpenAI --> NIC_OpenAI
        NIC_OpenAI --> NSG
        NIC_OpenAI --> Subnet
        NIC_OpenAI --> PE_OpenAI
        
        NSG --> Subnet
    end

Loading

1. Navigate to VNet: In the Azure portal, go toVirtual networks and select your VNet.

2. Subnets: Ensure that your subnets are correctly configured and have the necessary address space.

3. Service Endpoints: Add service endpoints for Azure OpenAI and Azure AI Search.

   

Configure Private Endpoints for Azure AI Search

1. Navigate to Private Endpoint: In the Azure portal, go to your Azure AI Search resource and select Networking > Private endpoint connections.
2. Add Private Endpoint: Click on + Private endpoint to add a new private endpoint.
3. Configure Private Endpoint: Follow the prompts to configure the private endpoint
   • Name: Provide a name for the private endpoint.
   • Virtual Network: Select the same virtual network and subnet as used for Azure OpenAI.
   • Integration: Integrate with your DNS for name resolution.
4. Approve Connection: Once the private endpoint is created, approve the connection.

Network Configuration	Use Case	Considerations
Shared Private Access	- Simplifies the configuration by allowing you to create a private endpoint connection from Azure AI Search to Azure OpenAI without manually setting up private endpoints. - Suitable when you want to establish a secure connection between resources in the same or different VNets. - The connection must be approved by the owner of the target resource, adding an extra layer of security.	- VNet Peering or VPN Gateway is required if Azure AI Search and Azure OpenAI are in different VNets. - Network Security Groups (NSGs) should be configured to allow traffic between the VNets if they are peered.
Private Endpoint Connection	- Provides a network interface that connects you privately and securely to a service powered by Azure Private Link. - Ideal for securing the connection between Azure AI Search and Azure OpenAI within the same Virtual Network. - Offers a higher level of security by ensuring that the connection remains within the VNet. - Can offer better performance as the traffic remains within the Azure backbone network.	- Requires manual setup of private endpoints for both Azure AI Search and Azure OpenAI. - DNS settings must be correctly configured to resolve the private endpoints.
VNet Peering	- Enables resources in different VNets to communicate with low latency and high bandwidth, as if they were within the same network. - Useful when Azure AI Search and Azure OpenAI are in different VNets. - Allows full connectivity between VNets, making it suitable for scenarios where multiple resources need to communicate across VNets.	- Peering links must be created in both VNets. - NSGs should be configured to allow traffic from the peered VNet. - Additional costs may be incurred for data transfer between VNets.

• Search for the resource ID of the Azure OpenAI service:

  

• Create the shared private link or the private endpoint as needed:

  Shared private access:

  

  Private Endpoint:

  

  • While doing this you can also setup the Network Security Group (NSG) if it's not already set up.

    

Configure Private Endpoints for Azure OpenAI

1. Navigate to Private Endpoint: In the Azure portal, go to your Azure OpenAI resource and select Networking > Private endpoint connections.
2. Add Private Endpoint: Click on + Private endpoint to add a new private endpoint.
3. Configure Private Endpoint: Follow the prompts to configure the private endpoint:
   • Name: Provide a name for the private endpoint.
   • Virtual Network: Select the virtual network and subnet where the endpoint will be deployed.
   • Integration: Integrate with your DNS for name resolution.
4. Approve Connection: Once the private endpoint is created, approve the connection.

Set Up Network Security Groups (NSGs)

1. Create NSG: In the Azure portal, create a new Network Security Group.
2. Associate NSG with Subnet: Associate the NSG with the subnet where your private endpoints are deployed.
3. Configure Security Rules: Add inbound and outbound security rules to allow traffic only from your specific network.

Create index/Upload Documents

Since now we are in a private network, Azure AI Search only admits requests from clients in a virtual network instead of over a public internet. So we need to create a VM, and set that VM in a VNET. Click here for a more detailed guide on how to Create a private endpoint for a secure connection to Azure AI Search

Connect the Azure AI Search service with the VNET:

• Create a virtual machine:

  

  • Provide the necessary details:

    

• Login into the VM, and go the AI Search. You can use Azure Bastion to connect.

  

Now you are able to access the AI Search behind the same private network.

• Create Index: Set up an index to store your documents.

  

• Upload Documents: Use the data import wizard to upload your documents and configure the indexer to parse the content.

  

Configure and Deploy AI model

• Navigate to Azure OpenAI Studio

  1. Open Azure OpenAI Studio: In your browser, go to the Azure OpenAI Studio.
  2. Sign In: Sign in with your Azure account.

• Explore the Model Catalog

  1. Model Catalog: From the sidebar, select Model catalog under the Get started section.

  2. Choose a Model: Browse through the available models and select the one you want to deploy to get more information about it.

     

  3. Deploy the model: You can do ir directly by the Model Catalog or under Deployments:

     • Deployments: From the sidebar, select Deployments under the Shared resources section.
     • Create Deployment: Click on Create deployment and follow the prompts to deploy the selected model.
       • Name: Provide a name for the deployment.
       • Model: Choose the model from the model catalog.
       • Scale Settings: Configure the scale settings as needed.
     • Deploy: Click Deploy to start the deployment process.

     

• Test the Deployment
  

  1. Playgrounds: Use the Playgrounds section in the sidebar to test the deployed model.
     

     • Chat: Test the model with chat interactions.
       

     • Add Data as AI Index in Azure OpenAI: If any authentication error happens please see how to allow the services to authorize each other

       1. Select Deployment: In the Setup section, choose the deployment you want to add data to.
          

       2. Add Data Source: Click on + Add a data source to add a new data source.
          

       3. Configure Data Source: Follow the steps to configure the data source. This may involve selecting the type of data source and providing the necessary connection details.
          

       4. Review and Finish: Review the configuration and finish the setup to add the data source.
          

       5. Index Data: The data from the configured data source will be indexed and available for search and retrieval.
          

          

     • Completions: Test the model with completion tasks.

     • Verify: Ensure that the model is responding as expected.
       

  2. Integrate with Your Application
     
     1. Get Endpoint and Key: From the Deployments section, get the endpoint and API key for the deployed model.
        
     2. Application Code: Use the endpoint and API key in your application code to send requests to the model.
        
     3. Handle Responses: Process the responses from the model within your application.
        
  3. Monitor and Manage
     
     1. Quota: Check the Quota section under Shared resources to monitor your usage.
        
     2. Content Filters: Use Content filters to manage the content filtering settings for your deployment.
        
     3. Data Files: Manage your data files in the Data files section.
        
     4. Vector Stores: Use Vector stores to manage vector representations of your data.
        

Refresh Date: 2026-04-06