Implement custom dashboard from APIM Analytics data

APIM analytics support the following ways to pull data to implement custom dashboards for feed to 3rd path analytics or dashboard engines.

1. Using Store REST API [1]

Analytics REST API[2][3] can be used to retrieve data from the analytics data layer. Store REST is a powerful tool and the same can be used to retrieved post-process data. You can use tables and aggregation streams created in siddhi apps. To get data new siddhi app can be implemented and deployed. Or else default siddhi apps ship in the APIM analytics can be used.

The following example shows getting data from a default siddhi. For that, I used APIM_ACCESS_SUMMARY siddhi app and the ApiVersionPerAppAgg aggregation stream.

• Curl request

curl -X POST \
  https://localhost:7444/stores/query \
  -H 'Authorization: Basic YWRtaW46YWRtaW4=' \
  -H 'Content-Type: application/json' \
  -d '{
	"appName": "APIM_ACCESS_SUMMARY",
	"query": "from ApiVersionPerAppAgg on apiCreatorTenantDomain=='\''carbon.super'\'' within 0L, 2585820450000L per '\''minutes'\'' select apiName, apiVersion, applicationName, sum(totalRequestCount) as RequestCount group by apiName, apiVersion, applicationName order by RequestCount desc"
}' -k

• Response

{
    "records": [
        [
            "simple",
            "1.0.0",
            "demoApp",
            8
        ],
        [
            "PizzaShack",
            "1.0.0",
            "demoApp",
            6
        ]
    ]
}

2. Event Sinking/Publishing

APIM analytics proceed data using streams of data. This data move from stream to stream and data layer by processing. This data can be published to the different data receivers using an event sink. For this also we can use existing siddhi apps or implement new siddhi apps.

APIM analytics set of event sink types and those are documented here[4]. Here we used Log sink as an example to publishing data to the default log file.

We used the same siddhi application and Request stream for this.

• Strema need to be modify like this and in order to apply Log sink, sink annotation need to define to the stream

@source(type='inMemory' , topic='APIM_REQUEST')
@sink(type='log', prefix='ApiVersion Details')
define stream Request (meta_clientType string, applicationConsumerKey string, applicationName string, applicationId string, applicationOwner string, apiContext string,apiName string, apiVersion string, apiResourcePath string, apiResourceTemplate string, apiMethod string, apiCreator string, apiCreatorTenantDomain string, apiTier string, apiHostname string, username string, userTenantDomain string, userIp string, userAgent string, requestTimestamp long, throttledOut bool, responseTime long, serviceTime long, backendTime long, responseCacheHit bool, responseSize long, protocol string, responseCode int, destination string, securityLatency long, throttlingLatency long, requestMedLat long, responseMedLat long, backendLatency long, otherLatency long, gatewayType string, label string);

• Logging test would be like this

osgi> [2020-04-02 16:41:35,439]  INFO {io.siddhi.core.stream.output.sink.LogSink} - ApiVersion Details : Event{timestamp=1585825895419, data=[{"correlationID":"f353c00c-1a67-4fc5-bc58-3e3331148f00","keyType":"PRODUCTION"}, Fxi2x7R9WM0yXhChdRWIN9fpec4a, demoApp, 2, admin, /pizza/1.0.0, PizzaShack, 1.0.0, /, /*, GET, admin, carbon.super, Unlimited, localhost, admin@carbon.super, carbon.super, 127.0.0.1, Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36, 1585825894844, false, 575, 1, 574, false, 0, https-8243, 200, http://www.mocky.io/v2/5cf89c053400006b0001b03d, 0, 0, 0, 0, 574, 0, SYNAPSE, Synapse], isExpired=false}

3. References

[1] https://docs.wso2.com/display/SP430/Public+APIs

[2] https://docs.wso2.com/display/SP430/Store+APIs

[3] https://docs.wso2.com/display/SP430/Managing+Stored+Data+via+REST+APIs

[4] https://docs.wso2.com/display/SP430/Publishing+Events