feature: support k8s service discovery. #5

Wangdai-0800 · 2025-11-11T01:34:54Z

No description provided.

pkg/aigateway/discovery/staticdemo/provider.go

pkg/aigateway/discovery/staticdemo/cds_server.go

pkg/aigateway/discovery/staticdemo/provider.go

doujiang24 · 2025-11-17T08:47:32Z

pkg/aigateway/discovery/xdsserver/cds_server.go

@@ -25,6 +25,7 @@ import (
 	"google.golang.org/grpc/codes"
 	"google.golang.org/grpc/status"

+	managertypes "github.com/aigw-project/aigw/pkg/aigateway/clustermanager/types"


it's not a good idea to reuse the static discovery, introduce a new implementation should be better.
For the code reusing we may introduce a common xds server to avoding copying code, it could be another PR as the second step.

ping @Wangdai-0800

Fixed. We abstract the xds server into independent modules and implement two class "StaticClusterProvider" & "DynamicClusterProvider" based on the "BaseClusterInfoProvider".

docs/en/developer-guide.md

pkg/aigateway/discovery/staticdemo/provider.go

doujiang24 · 2025-11-21T15:03:35Z

@Wangdai-0800 it's not a good idea to use force push, it makes review changes harder.

amazingapple · 2025-11-13T07:03:18Z

pkg/aigateway/discovery/staticdemo/provider.go

+
+// updata the snapshot form istio
+func (p *StaticClusterProvider) AutoUpdateFromPilot(nodeID string, interval time.Duration) {
+	go func() {


The AutoUpdateFromPilot function lacks an exit mechanism. It is recommended to use a for+select loop to ensure it can exit at least when context.Done() is triggered.

pkg/aigateway/discovery/staticdemo/provider.go

amazingapple · 2025-11-13T07:13:53Z

pkg/aigateway/discovery/xdsserver/cds_server.go

@@ -100,6 +101,28 @@ func (s *cdsServerImpl) processAllClusters() {
 	s.responseChan <- resp
 }

+func (s *cdsServerImpl) processSubscribedClusters(subscribeClusters []string) {


Add some INFO logs in the key steps to facilitate issue troubleshooting.

pkg/aigateway/discovery/staticdemo/provider.go

docs/zh/developer-guide.md

pkg/aigateway/discovery/dynamic_provider/dynamic_provider.go

doujiang24 · 2025-11-25T15:02:19Z

pkg/aigateway/discovery/dynamic_provider/dynamic_provider.go

+			if err != nil {
+				api.LogErrorf("failed to pull from pilot: %v", err)
+			}
+			time.Sleep(interval)


why need sleep here, we should do it by realtime subscribing.

Fixed. Remove the sleep. Could there be an asynchronous implementation？just like epoll_wait() works.

…service_discover

amazingapple · 2025-12-02T00:57:58Z

Makefile

+start-aigw-k8s-pod:
+	kind load docker-image aigw-image/aigw:v1 --name aigw-llm-service
+	ISTIO_K8S_HOST := "istiod.istio-system.svc.cluster.local"
+	cat etc/envoy-istio.yaml \


Use "sed -i.bak -n xx etc/envoy-istio.yaml" directly rather than "cat etc/envoy-istio.yaml | sed xx", "-i.bak" will create one backup file.

amazingapple · 2025-12-02T01:15:23Z

pkg/aigateway/clustermanager/types/clusterinfoprovider.go

+	GetAllClusters() []*ClusterInfo
+}
+
+type BaseClusterInfoProvider struct {


The best practice is using doc comments for all exported functions, types, and packages to provider clear documentation. Please reference https://go.dev/doc/comment.

amazingapple · 2025-12-02T01:17:13Z

pkg/aigateway/discovery/dynamic_provider/dynamic_provider.go

+	defaultNodeId   = "router~127.0.0.1~aigw.default~cluster.local"
+)
+
+type StaticEndpoint struct {


move Static* to static_provider.go?

amazingapple · 2025-12-02T01:36:49Z

pkg/aigateway/discovery/dynamic_provider/dynamic_provider.go

+}
+
+// updata the snapshot form istio
+func (p *DynamicClusterProvider) AutoUpdateFromPilot(nodeID string, interval time.Duration) {


interval is not used. Do you mean to try every internal until subscribe successfully?

amazingapple · 2025-12-02T01:38:36Z

pkg/aigateway/discovery/dynamic_provider/dynamic_provider.go

+	p.WarmupReady.Store(false)
+	p.AutoUpdateFromPilot(defaultNodeId, 10*time.Second)
+	// wait for the first pull of xds to complete
+	for p.WarmupReady.Load() == false {


Commonly, sync.WaitGroup is used to wait go routine to stop.

amazingapple · 2025-12-02T01:40:05Z

pkg/aigateway/discovery/dynamic_provider/dynamic_provider.go

+		grpc.WithTransportCredentials(insecure.NewCredentials()),
+	)
+	if err != nil {
+		return err


should log in every failure case?

Wangdai-0800 force-pushed the service_discover branch 3 times, most recently from f307f7d to fe28e1c Compare November 12, 2025 13:34

taoyuanyuan reviewed Nov 13, 2025

View reviewed changes

pkg/aigateway/discovery/staticdemo/provider.go Outdated Show resolved Hide resolved

Wangdai-0800 force-pushed the service_discover branch 6 times, most recently from bdb78a8 to 5f533ea Compare November 13, 2025 07:09

fly869 reviewed Nov 13, 2025

View reviewed changes

pkg/aigateway/discovery/staticdemo/cds_server.go Outdated Show resolved Hide resolved

pkg/aigateway/discovery/staticdemo/provider.go Outdated Show resolved Hide resolved

Wangdai-0800 force-pushed the service_discover branch 5 times, most recently from b974eb4 to 05c0f1e Compare November 14, 2025 08:34

doujiang24 reviewed Nov 17, 2025

View reviewed changes

fly869 reviewed Nov 20, 2025

View reviewed changes

docs/en/developer-guide.md Show resolved Hide resolved

fly869 reviewed Nov 20, 2025

View reviewed changes

pkg/aigateway/discovery/staticdemo/provider.go Outdated Show resolved Hide resolved

Wangdai-0800 force-pushed the service_discover branch from 05c0f1e to 09c33f1 Compare November 21, 2025 06:27

amazingapple reviewed Nov 22, 2025

View reviewed changes

feature: support k8s service discovery.

f62648c

Wangdai-0800 force-pushed the service_discover branch from 09c33f1 to f62648c Compare November 24, 2025 12:55

Wangdai-0800 added 2 commits November 25, 2025 22:03

feature: refactor the cluster provider & xds server implemention

5e26b16

remove redundant doc modification

d216e07

doujiang24 reviewed Nov 25, 2025

View reviewed changes

Wangdai-0800 and others added 3 commits November 29, 2025 18:50

support run aigw in k8s pod

596f3d0

Merge branch 'main' into service_discover

2641a32

revise the import

e34743d

Wangdai-0800 added 3 commits November 29, 2025 22:50

Merge branch 'service_discover' of github.com:Wangdai-0800/aigw into …

2b412e4

…service_discover

add a mock llm service for testing aigw with k8s

b09474c

revise MakeFile

b6a2124

Wangdai-0800 closed this Dec 1, 2025

Wangdai-0800 reopened this Dec 1, 2025

amazingapple reviewed Dec 2, 2025

View reviewed changes

feature: support k8s service discovery. #5

Are you sure you want to change the base?

feature: support k8s service discovery. #5

Uh oh!

Conversation

Wangdai-0800 commented Nov 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

doujiang24 commented Nov 21, 2025

Uh oh!

amazingapple Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

amazingapple Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

amazingapple Nov 13, 2025 •

edited

Loading

amazingapple Nov 13, 2025 •

edited

Loading