[Feature] Support application and service dependency graph visualization #1414
Conversation
…ate edges and add logging for nil specs
There was a problem hiding this comment.
Pull request overview
Adds a new console API to return Dubbo mesh dependency-graph data (nodes/edges) for visualization, based on service consumer/provider metadata resources.
Changes:
- Introduces
GET /api/v1/service/graphendpoint and handler to serve graph data. - Adds backend graph construction logic that derives app nodes and dependency edges from consumer/provider metadata (optionally filtered by
serviceName). - Adds new graph request/response model types (
GraphData,GraphNode,GraphEdge,ServiceGraphReq).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| pkg/console/service/service.go | Implements graph building logic (nodes/edges) by listing consumer/provider metadata + instances. |
| pkg/console/handler/service.go | Adds HTTP handler for the new graph endpoint. |
| pkg/console/router/router.go | Registers new route GET /api/v1/service/graph. |
| pkg/console/model/graph.go | Adds graph request/response structs (and additional cross-linked list types). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| index.ByMeshIndex: req.Mesh, | ||
| }) | ||
| if err != nil { | ||
| logger.Errorf("get instances for mesh %s failed, cause: %v", req.Mesh, err) |
There was a problem hiding this comment.
ListByIndexes for instances logs an error but continues returning a successful graph response, which can silently drop instance data and make the API appear healthy when it's not. Consider returning an error (consistent with the consumer/provider fetches) or explicitly reflecting the partial-failure in the response so clients can react appropriately.
| logger.Errorf("get instances for mesh %s failed, cause: %v", req.Mesh, err) | |
| logger.Errorf("get instances for mesh %s failed, cause: %v", req.Mesh, err) | |
| return nil, err |
| // Build edges between consumers and providers | ||
| // Only create edges between apps that actually have the service relationship | ||
| for _, consumer := range consumers { | ||
| if consumer.Spec != nil { | ||
| for _, provider := range providers { | ||
| if provider.Spec != nil { | ||
| // This is where you should check if the provider's service name and the consumer's service name match. | ||
| if consumer.Spec.ServiceName != provider.Spec.ServiceName { | ||
| continue | ||
| } | ||
| // If there are two instances, such as p1->c1 and p2->c1, two edges will be generated, which need to be merged. | ||
| // Merging logic: Only one edge is kept for identical source and target. | ||
| // However, using a loop would result in three levels of nesting. Therefore, an auxiliary map is created for efficient filtering. | ||
| edgeKey := consumer.Spec.ConsumerAppName + "->" + provider.Spec.ProviderAppName | ||
| if _, exists := edgeKeyMap[edgeKey]; exists { | ||
| continue | ||
| } | ||
| edgeKeyMap[edgeKey] = struct{}{} | ||
| edges = append(edges, model.GraphEdge{ | ||
| Source: consumer.Spec.ConsumerAppName, | ||
| Target: provider.Spec.ProviderAppName, | ||
| Data: map[string]interface{}{ | ||
| "type": "dependency", | ||
| "serviceName": req.ServiceName, | ||
| "consumerApp": consumer.Spec.ConsumerAppName, | ||
| "providerApp": provider.Spec.ProviderAppName, | ||
| }, | ||
| }) | ||
| } else { | ||
| logger.Warnf("provider spec is nil for provider resource: %s", provider.Name) | ||
| } | ||
| } | ||
| } else { | ||
| logger.Warnf("consumer spec is nil for consumer resource: %s", consumer.Name) | ||
| } |
There was a problem hiding this comment.
Building edges uses a nested consumers×providers loop, which is O(C*P) and can become very expensive when req.ServiceName is empty (lists include all metadata in the mesh) or when there are many instances per service. A more scalable approach is to pre-index providers by ServiceName (and possibly by ProviderAppName) and then only iterate over matching providers for each consumer.
| // Build edges between consumers and providers | |
| // Only create edges between apps that actually have the service relationship | |
| for _, consumer := range consumers { | |
| if consumer.Spec != nil { | |
| for _, provider := range providers { | |
| if provider.Spec != nil { | |
| // This is where you should check if the provider's service name and the consumer's service name match. | |
| if consumer.Spec.ServiceName != provider.Spec.ServiceName { | |
| continue | |
| } | |
| // If there are two instances, such as p1->c1 and p2->c1, two edges will be generated, which need to be merged. | |
| // Merging logic: Only one edge is kept for identical source and target. | |
| // However, using a loop would result in three levels of nesting. Therefore, an auxiliary map is created for efficient filtering. | |
| edgeKey := consumer.Spec.ConsumerAppName + "->" + provider.Spec.ProviderAppName | |
| if _, exists := edgeKeyMap[edgeKey]; exists { | |
| continue | |
| } | |
| edgeKeyMap[edgeKey] = struct{}{} | |
| edges = append(edges, model.GraphEdge{ | |
| Source: consumer.Spec.ConsumerAppName, | |
| Target: provider.Spec.ProviderAppName, | |
| Data: map[string]interface{}{ | |
| "type": "dependency", | |
| "serviceName": req.ServiceName, | |
| "consumerApp": consumer.Spec.ConsumerAppName, | |
| "providerApp": provider.Spec.ProviderAppName, | |
| }, | |
| }) | |
| } else { | |
| logger.Warnf("provider spec is nil for provider resource: %s", provider.Name) | |
| } | |
| } | |
| } else { | |
| logger.Warnf("consumer spec is nil for consumer resource: %s", consumer.Name) | |
| } | |
| // Build an index of providers by service name to avoid O(C*P) nested loops. | |
| providersByService := make(map[string][]int) | |
| for pIdx, provider := range providers { | |
| if provider.Spec == nil { | |
| logger.Warnf("provider spec is nil for provider resource: %s", provider.Name) | |
| continue | |
| } | |
| serviceName := provider.Spec.ServiceName | |
| providersByService[serviceName] = append(providersByService[serviceName], pIdx) | |
| } | |
| // Build edges between consumers and providers | |
| // Only create edges between apps that actually have the service relationship | |
| for _, consumer := range consumers { | |
| if consumer.Spec == nil { | |
| logger.Warnf("consumer spec is nil for consumer resource: %s", consumer.Name) | |
| continue | |
| } | |
| serviceName := consumer.Spec.ServiceName | |
| providerIdxs := providersByService[serviceName] | |
| for _, pIdx := range providerIdxs { | |
| provider := providers[pIdx] | |
| // provider.Spec is guaranteed non-nil here because we only indexed non-nil specs. | |
| // If there are two instances, such as p1->c1 and p2->c1, two edges will be generated, which need to be merged. | |
| // Merging logic: Only one edge is kept for identical source and target. | |
| // However, using a loop would result in three levels of nesting. Therefore, an auxiliary map is created for efficient filtering. | |
| edgeKey := consumer.Spec.ConsumerAppName + "->" + provider.Spec.ProviderAppName | |
| if _, exists := edgeKeyMap[edgeKey]; exists { | |
| continue | |
| } | |
| edgeKeyMap[edgeKey] = struct{}{} | |
| edges = append(edges, model.GraphEdge{ | |
| Source: consumer.Spec.ConsumerAppName, | |
| Target: provider.Spec.ProviderAppName, | |
| Data: map[string]interface{}{ | |
| "type": "dependency", | |
| "serviceName": req.ServiceName, | |
| "consumerApp": consumer.Spec.ConsumerAppName, | |
| "providerApp": provider.Spec.ProviderAppName, | |
| }, | |
| }) | |
| } |
| if consumer.Spec != nil { | ||
| for _, provider := range providers { | ||
| if provider.Spec != nil { | ||
| // This is where you should check if the provider's service name and the consumer's service name match. |
There was a problem hiding this comment.
This inline comment reads like a leftover TODO/instruction and doesn't add value in production code. Please remove it or replace it with a concise explanation of the actual matching rule being implemented (service-name equality).
| // This is where you should check if the provider's service name and the consumer's service name match. | |
| // Only create edges when the consumer and provider reference the same service name. |
| // Build nodes for each app | ||
| // Provider nodes: data contains instances providing this service | ||
| // Consumer nodes: data is nil (consumers don't provide instances) | ||
| for appName := range allApps { | ||
| var instanceData interface{} | ||
|
|
||
| instances := make([]interface{}, 0) | ||
| if appInsts, ok := appInstances[appName]; ok { | ||
| for _, instance := range appInsts { | ||
| instances = append(instances, toInstanceData(instance)) | ||
| } | ||
| } | ||
| instanceData = instances | ||
|
|
||
| nodes = append(nodes, model.GraphNode{ | ||
| ID: appName, | ||
| Label: appName, | ||
| Data: instanceData, | ||
| }) |
There was a problem hiding this comment.
The comment above this loop says consumer nodes should have data = nil, but the code always assigns instances (an empty slice when none exist). Because Data is an interface{} with omitempty, this will still serialize as "data": [] for apps with no instances. Decide on the intended API contract and either set Data to nil when there are no instances or update the comment/API docs accordingly.
| import ( | ||
| meshresource "github.com/apache/dubbo-admin/pkg/core/resource/apis/mesh/v1alpha1" | ||
| ) | ||
|
|
||
| // GraphNode represents a node in the graph for AntV G6 | ||
| type GraphNode struct { | ||
| ID string `json:"id"` | ||
| Label string `json:"label"` | ||
| Data interface{} `json:"data,omitempty"` // Additional data for the node | ||
| } | ||
|
|
||
| // GraphEdge represents an edge in the graph for AntV G6 | ||
| type GraphEdge struct { | ||
| Source string `json:"source"` | ||
| Target string `json:"target"` | ||
| Data map[string]interface{} `json:"data,omitempty"` // Additional data for the edge | ||
| } | ||
|
|
||
| // GraphData represents the complete graph structure for AntV G6 | ||
| type GraphData struct { | ||
| Nodes []GraphNode `json:"nodes"` | ||
| Edges []GraphEdge `json:"edges"` | ||
| } | ||
|
|
||
| // CrossNode represents a node in the cross-linked list structure | ||
| type CrossNode struct { | ||
| Instance *meshresource.InstanceResource | ||
| Next *CrossNode // pointer to next node in the same row | ||
| Down *CrossNode // pointer to next node in the same column | ||
| } | ||
|
|
||
| // CrossLinkedListGraph represents the cross-linked list structure as a directed graph | ||
| type CrossLinkedListGraph struct { | ||
| Head *CrossNode | ||
| Rows int // number of rows | ||
| Cols int // number of columns | ||
| } |
There was a problem hiding this comment.
CrossNode / CrossLinkedListGraph appear unused by the new endpoint (which returns GraphData), but they add API surface and force an extra import. If the graph is meant to be nodes/edges for G6, consider removing these types (and the meshresource import) or moving them behind an actual use to avoid confusion/maintenance burden.
robocanic
left a comment
robocanic
left a comment
There was a problem hiding this comment.
Great Work! I found some issues and hope you can discuss them with me!
| } | ||
|
|
||
| // Only filter by serviceName if it's provided | ||
| if req.ServiceName != "" { |
There was a problem hiding this comment.
Correction：
- 不应该有这个判断，服务名是必须的参数，没有服务名，应该在handler里面就返回错误。后续的拓扑也是针对某一个服务来展开的。
- 对于一个服务来说，serviceName不能完全定位一个服务，必须要serviceName:version:group——即ServiceKey才能完全定位一个服务。这里请求入参可以直接使用BaseServiceReq
- 这里对于ServiceConsumerMetadataKind和ServiceProviderMetadataKind的索引还缺少对ServiceKey的索引，需要先在索引里面加上，然后用ServiceKey的索引来搜索这两个Resource
| } | ||
|
|
||
| // Collect all unique applications (both consumers and providers) | ||
| consumerApps := make(map[string]bool) |
There was a problem hiding this comment.
Suggestion：
- golang中没有内置的set，但可以用map[string]strunct{}来替代，value是一个空的struct，不占存储空间
- 可以使用工具类lancet中的set来替代，具体用法可以参考其他代码
| appInstances := make(map[string][]*meshresource.InstanceResource) | ||
|
|
||
| // Get all instances for the mesh first | ||
| allInstances, err := manager.ListByIndexes[*meshresource.InstanceResource]( |
There was a problem hiding this comment.
Idea: 这里应该是想要把所有的实例搜出来，然后挂到每个应用下面，但这种做法在生产环境下万万不行，因为这里面会把所有的实例全部搜出来，如果实例数很多，可能会出现爆内存/DB打挂的情况。所以这里可以从交互入手，改一下交互， 在返回graph时不返回每个节点具体的数据，只返回id和label。用户点击一个节点时再请求对应节点（应用）的数据。这样，就不用在拓扑图这里捞这么多数据。
|
|
||
| // Build edges between consumers and providers | ||
| // Only create edges between apps that actually have the service relationship | ||
| for _, consumer := range consumers { |
There was a problem hiding this comment.
Idea： 这里的每一个消费者应用都会有一条边指向提供者应用？这个关系太复杂。可不可以转换成这种结构:
用服务作为一个中间节点，左边是提供者应用，右边是消费者应用
确定无地方使用 Co-authored-by: Copilot <[email protected]>
|
2 participants
Please provide a description of this PR:
This PR implements the service and application dependency graph visualization as discussed in #1398. It provides the backend logic to trace upstream and downstream dependencies for both specific services and entire applications within a mesh.
Key Implementation Details:
Data Source: Leverages
ServiceProviderMetadataResourceandServiceConsumerMetadataResourceto establish the caller-callee relationships.Logic: * Builds a directed graph where
Source (Consumer) -> Target (Provider).Supports filtering by
ServiceNameto show localized dependency trees.Aggregates service-level dependencies into application-level nodes.
Includes instance information within the graph nodes for better observability.
Implemented efficient edge merging using an auxiliary map to avoid redundant visual links between the same applications.
Standards: Aligned with
dubbo-java-3.xmetadata standards.To help us figure out who should review this PR, please put an X in all the areas that this PR affects.
Please check any characteristics that apply to this pull request.
API Usage Example
You can retrieve the graph data using the following
curlcommand:curl --location --request GET 'http://localhost:8888/api/v1/service/graph?mesh=zk3.6'Example Response:
{ "code": "Success", "message": "success", "data": { "nodes": [ { "id": "order-service", "label": "order-service", "data": [ { "appName": "order-service", "ip": "10.12.1.166", "rpcPort": 20881, "protocol": "dubbo" } ] } ], "edges": [ { "source": "order-service", "target": "inventory-service", "data": { "type": "dependency", "consumerApp": "order-service", "providerApp": "inventory-service" } } ] } }Data Structure Explanation
1. GraphNode (
nodes)Represents an Application in the Dubbo mesh.
id&label: The Application Name.data: An array ofInstanceobjects belonging to this application, providing real-time runtime info (IP, Port, Protocol).2. GraphEdge (
edges)Represents the Dependency between two applications.
source: The name of the Consumer application.target: The name of the Provider application.data:type: Hardcoded as"dependency".serviceName: The specific Dubbo service interface causing this dependency (empty if it's an aggregated application graph).consumerApp/providerApp: Redundant identifiers for easy processing on the frontend.Please check any characteristics that apply to this pull request.