Introduce API Tokens with `cluster_permissions` and `index_permissions` directly associated with the token by cwperks · Pull Request #5443 · opensearch-project/security

cwperks · 2025-06-25T17:56:21Z

Description

Re-basing #5225 with the latest changes from main.

This PR introduces API Tokens — a new capability in the Security plugin that allows security admins to issue long-lived, scoped tokens and associate permissions directly with the token.

How it works

API Tokens are opaque tokens with the format os_<random>. When a token is created, a SHA-256 hash of the plaintext token is stored in a system index, .opensearch_security_api_tokens. The plaintext token is returned once at creation time and never stored. On each request, the incoming token is hashed and looked up in an in-memory cache populated from the index.

Tokens are authenticated via the Authorization: ApiKey <token> header.

What is novel about this approach compared to OBO tokens is that permissions are scoped directly to the token rather than derived from the issuing user's roles. An admin can issue a token with only the permissions it needs — for example, read-only access to a single index — regardless of the admin's own permissions. This enforces the principle of least privilege and is a key building block toward deprecating Roles Injection, the current practice for how plugins run async jobs with user-scoped permissions.

Revocation model

Tokens use a soft-delete revocation model. When a token is revoked via DELETE /_plugins/_security/api/apitokens/{id}, the document in the index is updated with a revoked_at timestamp rather than being deleted. This means:

Revoked tokens remain visible in the list endpoint with a revoked_at field, enabling UIs to display revocation history and audit trails.
Revocation is synchronous — the cache refresh is broadcast to all nodes and confirmed before the response is returned, so the token is immediately unusable cluster-wide.
During cache reload, tokens with revoked_at set are excluded from the in-memory authentication maps, so they cannot be used to authenticate.

API Reference

Create API Token

POST /_plugins/_security/api/apitokens

Request:

{
  "name": "my-token",
  "cluster_permissions": ["cluster:monitor/health"],
  "index_permissions": [
    {
      "index_pattern": ["logs-*"],
      "allowed_actions": ["indices:data/read/search"]
    }
  ],
  "expiration": 1800000
}

Response:

{
  "id": "Nd_pMRWeAC93ZGMhRa5CxX",
  "token": "os_abc123..."
}

The id is used to manage the token, such as listing or revoking it. The plaintext token is returned once and never stored — save it immediately.

List API Tokens

GET /_plugins/_security/api/apitokens

Returns all tokens, including revoked ones. Revoked tokens include a revoked_at field (epoch millis).

Response:

[
  {
    "id": "Nd_pMRWeAC93ZGMhRa5CxX",
    "name": "my-token",
    "iat": 1742000000000,
    "expiration": 1800000,
    "cluster_permissions": ["cluster:monitor/health"],
    "index_permissions": [
      {
        "index_pattern": ["logs-*"],
        "allowed_actions": ["indices:data/read/search"]
      }
    ]
  },
  {
    "id": "Xf_qNSZeBC04AHNiSb6DyY",
    "name": "old-token",
    "iat": 1741000000000,
    "expiration": 1800000,
    "revoked_at": 1741500000000,
    "cluster_permissions": ["cluster:monitor/health"],
    "index_permissions": []
  }
]

Revoke API Token

DELETE /_plugins/_security/api/apitokens/{id}

Response:

{
  "message": "Token Nd_pMRWeAC93ZGMhRa5CxX revoked successfully."
}

Revocation is a soft-delete — the token metadata is retained with a revoked_at timestamp. The token is immediately unusable after the response is returned. The cache refresh is broadcast synchronously to all nodes before the response is sent.

Using a Token

Pass the token in the Authorization header using the ApiKey scheme:

Authorization: ApiKey os_abc123...

Example — search a permitted index:

GET /logs-2025/_search
Authorization: ApiKey os_abc123...

Response:

{
  "hits": {
    "total": { "value": 3, "relation": "eq" },
    "hits": [ ... ]
  }
}

Example — attempt a forbidden action:

DELETE /logs-2025
Authorization: ApiKey os_abc123...

Response:

{
  "error": {
    "type": "security_exception",
    "reason": "no permissions for [indices:admin/delete]"
  },
  "status": 403
}

Issues Resolved

Partially resolves #4009, limited to security admins in the initial release.

Check List

New functionality includes testing
New functionality has been documented
New Roles/Permissions have a corresponding security dashboards plugin PR
API changes companion pull request created
Commits are signed per the DCO using --signoff

…4921) Signed-off-by: Derek Ho <dxho@amazon.com>

…ect#4967) Signed-off-by: Derek Ho <dxho@amazon.com>

Signed-off-by: Derek Ho <dxho@amazon.com>

…00 tokens outstanding (opensearch-project#5147) Signed-off-by: Derek Ho <dxho@amazon.com>

Signed-off-by: Craig Perkins <cwperx@amazon.com>

codecov · 2025-06-25T19:51:51Z

Codecov Report

❌ Patch coverage is 81.50183% with 101 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.95%. Comparing base (adf4a40) to head (eacd135).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
...arch/security/action/apitokens/ApiTokenAction.java	82.64%	18 Missing and 3 partials ⚠️
...opensearch/security/action/apitokens/ApiToken.java	82.35%	10 Missing and 8 partials ⚠️
.../security/action/apitokens/ApiTokenRepository.java	89.21%	7 Missing and 4 partials ⚠️
...ecurity/action/apitokens/ApiTokenIndexHandler.java	82.75%	10 Missing ⚠️
...pensearch/security/http/ApiTokenAuthenticator.java	81.25%	7 Missing and 2 partials ⚠️
...ction/apitokens/TransportApiTokenUpdateAction.java	65.00%	7 Missing ⚠️
...urity/action/apitokens/ApiTokenUpdateResponse.java	28.57%	5 Missing ⚠️
...ecurity/authtoken/jwt/ExpiringBearerAuthToken.java	0.00%	5 Missing ⚠️
...curity/action/apitokens/ApiTokenUpdateRequest.java	33.33%	4 Missing ⚠️
...search/security/securityconf/impl/v7/ConfigV7.java	78.57%	3 Missing ⚠️
... and 6 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5443      +/-   ##
==========================================
+ Coverage   73.79%   73.95%   +0.15%     
==========================================
  Files         440      451      +11     
  Lines       27259    27868     +609     
  Branches     4052     4115      +63     
==========================================
+ Hits        20117    20610     +493     
- Misses       5227     5317      +90     
- Partials     1915     1941      +26

Files with missing lines	Coverage Δ
...ecurity/action/apitokens/ApiTokenUpdateAction.java	`100.00% <100.00%> (ø)`
...ensearch/security/compliance/ComplianceConfig.java	`89.17% <100.00%> (+0.06%)`	⬆️
...rg/opensearch/security/dlic/rest/api/Endpoint.java	`100.00% <100.00%> (ø)`
...dlic/rest/api/RestApiAdminPrivilegesEvaluator.java	`74.54% <100.00%> (+0.47%)`	⬆️
...h/security/privileges/PrivilegesEvaluatorImpl.java	`83.15% <100.00%> (+0.30%)`	⬆️
...ch/security/securityconf/DynamicConfigFactory.java	`66.66% <100.00%> (+0.21%)`	⬆️
...ch/security/securityconf/DynamicConfigModelV7.java	`70.22% <100.00%> (+1.03%)`	⬆️
...g/opensearch/security/ssl/util/ExceptionUtils.java	`45.83% <100.00%> (+2.35%)`	⬆️
...g/opensearch/security/support/ConfigConstants.java	`77.27% <ø> (ø)`
...a/org/opensearch/security/util/AuthTokenUtils.java	`66.66% <100.00%> (+4.16%)`	⬆️
... and 16 more

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

nibix

Thanks for this! Cool to see API tokens going forward.

One general question: I am wondering whether we also need to modify the DLS/FLS code to be API token aware. Otherwise, the deny-by-default policy might block index access.

nibix · 2025-06-26T05:36:33Z

src/main/java/org/opensearch/security/privileges/PrivilegesEvaluator.java

+                    pluginIdToActionPrivileges.put(
+                        entry.getKey(),
+                        new SubjectBasedActionPrivileges(entry.getValue(), flattenedActionGroups)
+                    );


With this, we might need to make the pluginIdToActionPrivileges HashMap thread-safe, for example, by converting it into a ConcurrentHashMap.

Alternatively, we could have two HashMaps, one for plugins and one for API tokens.

i vote for single thread-safe map

nibix · 2025-06-26T05:46:44Z

src/main/java/org/opensearch/security/action/apitokens/ApiTokenRepository.java

+
+    private final Map<String, RoleV7> jtis = new ConcurrentHashMap<>();
+
+    void reloadApiTokensFromIndex(ActionListener<Void> listener) {


I see this is getting called when an update action is received. I think I did not see anything regarding initial node startup. Did I miss something?

Cache seems to be update only when a token is created or deleted. We should add one that loads on node bootstrap.

nibix · 2025-06-26T05:48:05Z

src/main/java/org/opensearch/security/action/apitokens/ApiTokenAction.java

+                    if (tokenCount >= 100) {
+                        sendErrorResponse(
+                            channel,
+                            RestStatus.TOO_MANY_REQUESTS,
+                            "Maximum limit of 100 API tokens reached. Please delete existing tokens before creating new ones."
+                        );
+                        return;


This seems to be a quite low limit. Is there a reason for that?

nibix · 2025-06-26T05:52:52Z

src/main/java/org/opensearch/security/action/apitokens/ApiTokenRepository.java

+    public void createApiToken(
+        String name,
+        List<String> clusterPermissions,
+        List<ApiToken.IndexPermission> indexPermissions,
+        Long expiration,
+        ActionListener<String> listener
+    ) {
+        apiTokenIndexHandler.createApiTokenIndexIfAbsent(ActionListener.wrap(() -> {
+            ExpiringBearerAuthToken token = securityTokenManager.issueApiToken(name, expiration);
+            ApiToken apiToken = new ApiToken(name, clusterPermissions, indexPermissions, expiration);
+            apiTokenIndexHandler.indexTokenMetadata(


Is the parameter name an identifier for the token? Do we need uniqueness guarantees for this?

yea it is a human readable identifier and uniqueness is not enforced...similar to github personal access tokens.

src/main/java/org/opensearch/security/http/ApiTokenAuthenticator.java

nibix · 2025-06-26T06:01:07Z

src/main/java/org/opensearch/security/action/apitokens/ApiTokenRepository.java

+    private final List<TokenListener> tokenListener = new ArrayList<>();
+    private static final Logger log = LogManager.getLogger(ApiTokenRepository.class);
+
+    private final Map<String, RoleV7> jtis = new ConcurrentHashMap<>();


What do you think about changing RoleV7 into SubjectBasedActionPrivileges here? PrivilegesEvaluator could then retrieve these instances just from here.

+1. It might not be straightforward since JTIs seem to be populated on getTokenMetadata request and there doesn't seem to be a way to pass flattenedActionGroups to that call. I may be seeing the complete picture here, but something like updateJTIs(FlattenedAGs) from PrivilegeEvaluator to update jtis and then use that to populate pluginIdToActionPrivileges which will then be used to create PrivilegeEvalContext?

yea I don't think its straightforward, but we should capture this as a refactor that would be good to have

nibix · 2025-06-26T06:03:02Z

src/main/java/org/opensearch/security/action/apitokens/ApiTokenRepository.java

+    public Map<String, RoleV7> getJtis() {
+        return jtis;
+    }


IMHO, this map should be kept private and managed only by this class.

Made package-private and annotated as @VisibleForTesting

nibix · 2025-06-26T06:05:56Z

src/integrationTest/java/org/opensearch/test/framework/ApiTokenConfig.java

+    public XContentBuilder toXContent(XContentBuilder xContentBuilder, Params params) throws IOException {
+        xContentBuilder.startObject();
+        xContentBuilder.field("enabled", enabled);
+        xContentBuilder.field("signing_key", signing_key);


Is this a hs512 key?

i think it fails in ApiTokenAuthenticator if it is anything lower than 512 bits.

I think at some point in the future we should look to support the opensearch-keystore for values referenced in the security config

nibix · 2025-06-26T06:11:00Z

src/main/java/org/opensearch/security/http/ApiTokenAuthenticator.java

+    public Boolean isRequestAllowed(final SecurityRequest request) {
+        Matcher matcher = PATTERN_PATH_PREFIX.matcher(request.path());
+        final String suffix = matcher.matches() ? matcher.group(2) : null;
+        if (isAccessToRestrictedEndpoints(request, suffix)) {
+            final OpenSearchException exception = ExceptionUtils.invalidUsageOfApiTokenException();
+            log.error(exception.toString());
+            return false;
+        }
+        return true;
+    }


This feels like a fragile way to deny certain endpoints, especially with the hardcoded path prefixes.

Also, do I understand it correctly that the goal of this code is that we cannot call the "issue API token" API with API tokens?

Shouldn't rest-admin-only restriction block access to endpoints anyway?

yea the purpose is to prevent using API Tokens to issue API Tokens. Similar to how OBO Tokens cannot be used to issue new OBO Tokens. Any ideas on how to refactor this?

DarshitChanpura

Thank you @cwperks for taking this over. Left some comments around testing and general usage.

src/main/java/org/opensearch/security/action/apitokens/ApiToken.java

src/main/java/org/opensearch/security/action/apitokens/ApiTokenAction.java

src/main/java/org/opensearch/security/action/apitokens/ApiToken.java

DarshitChanpura · 2025-06-26T18:52:53Z

src/main/java/org/opensearch/security/action/apitokens/ApiTokenRepository.java

+    private final List<TokenListener> tokenListener = new ArrayList<>();
+    private static final Logger log = LogManager.getLogger(ApiTokenRepository.class);
+
+    private final Map<String, RoleV7> jtis = new ConcurrentHashMap<>();


+1. It might not be straightforward since JTIs seem to be populated on getTokenMetadata request and there doesn't seem to be a way to pass flattenedActionGroups to that call. I may be seeing the complete picture here, but something like updateJTIs(FlattenedAGs) from PrivilegeEvaluator to update jtis and then use that to populate pluginIdToActionPrivileges which will then be used to create PrivilegeEvalContext?

src/integrationTest/java/org/opensearch/security/privileges/ApiTokenTest.java

DarshitChanpura · 2025-06-26T20:11:48Z

src/integrationTest/java/org/opensearch/test/framework/ApiTokenConfig.java

+    public XContentBuilder toXContent(XContentBuilder xContentBuilder, Params params) throws IOException {
+        xContentBuilder.startObject();
+        xContentBuilder.field("enabled", enabled);
+        xContentBuilder.field("signing_key", signing_key);


i think it fails in ApiTokenAuthenticator if it is anything lower than 512 bits.

src/test/java/org/opensearch/security/identity/SecurityTokenManagerTest.java

src/test/java/org/opensearch/security/authtoken/jwt/JwtVendorTest.java

src/integrationTest/java/org/opensearch/security/privileges/ApiTokenTest.java

Signed-off-by: Craig Perkins <cwperx@amazon.com>

ztomaszewska · 2025-11-21T09:41:27Z

Hi @cwperks ! First I wanted to thank you for your work on this PR. Really appreciate effort you put it in to move forward with this feature! Just wanted to ask whether is there any plan to soon merge your changes? Is there any blocking issues/ any help you would need? It would be really cool to have API tokens feature working :)

cwperks · 2025-11-25T20:43:19Z

I'll resolve the conflicts ASAP and try to push this forward for V1.

Its important to know that this PR will still have a lot of limitations, but paves the way for expansion. For instance, in this PR only the admin can issue tokens.

Signed-off-by: Craig Perkins <cwperx@amazon.com>

DarshitChanpura

thank you for picking this up @cwperks . Left a few comments. Main comment is addition of e2e test which checks authn+authz with API token,

src/main/java/org/opensearch/security/action/apitokens/ApiTokenAction.java

DarshitChanpura · 2025-11-26T20:31:32Z

src/main/java/org/opensearch/security/action/apitokens/ApiTokenAction.java

+                    updateRequest,
+                    ActionListener.wrap(
+                        updateResponse -> listener.onResponse(response),
+                        exception -> listener.onFailure(new ApiTokenException("Failed to refresh cache", exception))


this message should be "Failed to update API token"

CHANGELOG.md

src/main/java/org/opensearch/security/identity/SecurityTokenManager.java

src/integrationTest/java/org/opensearch/security/privileges/ApiTokenTest.java

Signed-off-by: Craig Perkins <cwperx@amazon.com>

derek-ho and others added 23 commits November 14, 2024 10:47

Merge branch 'main' of github.com:opensearch-project/security into HEAD

6ae7090

Scaffolding for POST/DELETE/GET api tokens calls (opensearch-project#…

3177c34

…4921) Signed-off-by: Derek Ho <dxho@amazon.com>

Adds JTI and expiration field support for API Tokens (opensearch-proj…

dacdae5

…ect#4967) Signed-off-by: Derek Ho <dxho@amazon.com>

Merge branch 'main' of github.com:opensearch-project/security into HEAD

e255e14

Signed-off-by: Derek Ho <dxho@amazon.com>

Merge branch 'main' of github.com:opensearch-project/security into HEAD

190bfec

Api token authc/z implementation with Cache (opensearch-project#4992)

79f0c46

Signed-off-by: Derek Ho <dxho@amazon.com>

Subset of permissions check on creation (opensearch-project#5012)

a8b4ac1

Signed-off-by: Derek Ho <dxho@amazon.com>

Merge branch 'main' of github.com:opensearch-project/security into HEAD

3776667

Change API token index actions to use action listeners and limit to 1…

8750e8b

…00 tokens outstanding (opensearch-project#5147) Signed-off-by: Derek Ho <dxho@amazon.com>

Merge branch 'main' into feature/api-tokens-cwperx

7b8b069

Fix naming

12c0f9c

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Use one PrivilegesEvaluatorContext

896e9e2

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Handle authz

97db90d

Signed-off-by: Craig Perkins <cwperx@amazon.com>

fix unit tests

362e67f

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Fix tests

fa98ae2

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Merge branch 'main' into feature/api-tokens-cwperx

f3cd485

Add integrationTests for API Token

68107ff

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Add more integration tests

f5b965a

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Add token prefix

ba93aa3

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Merge branch 'main' into feature/api-tokens-cwperx

e35d3ef

Rebase with main

c028420

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Merge branch 'main' into feature/api-tokens-cwperx

109c1ef

Fix compilation issues

3a71078

Signed-off-by: Craig Perkins <cwperx@amazon.com>

cwperks requested review from DarshitChanpura, RyanL1997, derek-ho, nibix, peternied, reta and shikharj05 as code owners June 25, 2025 17:56

cwperks requested a review from willyborankin as a code owner June 25, 2025 17:56

cwperks changed the title ~~Feature/api tokens cwperx~~ Introduce API Tokens with cluster_permissions and index_permissions directly associated with the token Jun 25, 2025

Add to CHANGELOG

dad7551

Signed-off-by: Craig Perkins <cwperx@amazon.com>

nibix reviewed Jun 26, 2025

View reviewed changes

DarshitChanpura mentioned this pull request Jun 26, 2025

Feature/api tokens #5225

Closed

5 tasks

DarshitChanpura reviewed Jun 26, 2025

View reviewed changes

nibix mentioned this pull request Jul 17, 2025

Introduced setting for disabling denormalized privilege data structures #5465

Merged

3 tasks

cwperks added 3 commits July 20, 2025 20:37

Merge branch 'main' into feature/api-tokens-cwperx

30f4f6f

Merge branch 'main' into feature/api-tokens-cwperx

3f11a61

Address PR feedback

eb512e3

Signed-off-by: Craig Perkins <cwperx@amazon.com>

cwperks added 2 commits November 25, 2025 17:12

Attempt to resolve conflicts

93a0e4f

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Fix unit test

bdd7d7f

Signed-off-by: Craig Perkins <cwperx@amazon.com>

DarshitChanpura reviewed Nov 26, 2025

View reviewed changes

cwperks added 10 commits March 16, 2026 21:15

Merge branch 'main' into feature/api-tokens-cwperx

6c45459

Address PR feedback

51c086f

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Address comments

cd47c36

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Use XContent parsing

2780872

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Address comments

720f72f

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Fix test

c86b305

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Make API Tokens Opaque Strings

64bee5f

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Update delete

a0876e4

Signed-off-by: Craig Perkins <cwperx@amazon.com>

Merge branch 'main' into feature/api-tokens-cwperx

78b5cbb

Implement soft-delete for token revocation

0d867ed

Signed-off-by: Craig Perkins <cwperx@amazon.com>

cwperks requested a review from finnegancarroll as a code owner March 25, 2026 19:10

spotlessApply

eacd135

Signed-off-by: Craig Perkins <cwperx@amazon.com>


		private final Map<String, RoleV7> jtis = new ConcurrentHashMap<>();

		void reloadApiTokensFromIndex(ActionListener<Void> listener) {

Conversation

cwperks commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How it works

Revocation model

API Reference

Create API Token

List API Tokens

Revoke API Token

Using a Token

Issues Resolved

Check List

Uh oh!

codecov bot commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nibix left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DarshitChanpura left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ztomaszewska commented Nov 21, 2025

Uh oh!

cwperks commented Nov 25, 2025

cwperks commented Jun 25, 2025 •

edited

Loading

codecov bot commented Jun 25, 2025 •

edited

Loading