Reconsider default behavior for DNFOF in 3.0.0

[cwperks]

Coming from conversation on opensearch-project/OpenSearch#17650

When a user is mapped to a role that has full access to a subset of indices in a cluster then by default _cat/indices will fail with a Forbidden error instead of returning the indices that are visible to the user. As a workaround, we instruct cluster administrators to consider toggling do_not_fail_on_forbidden to true.

With this issue, I propose changing the default behavior (at least for _cat/indices) to always return the indices visible to the user regardless if a cluster has DNFOF set to true.

I believe this is possible by updating this line to return true if the action is indices:monitor/settings/get

What do the maintainers think of this change in default behavior for 3.0.0 release?

[nibix]

Answering your question from opensearch-project/OpenSearch#17650 (comment) here:

I do believe that when using Dashboards one should have DNFOF enabled, yes.

Still, DNFOF has design flaws. Thus, I am not sure if having it enabled always makes sense.

Actually, I wrote down an RFC on DNFOF a while ago:

#3905

[cwperks]

Actually, I wrote down an RFC on DNFOF a while ago:

#3905

This would be a good change. I think this would be a good improvement to the system that reduces the level of complexity in grokking how security works in opensearch.

[wheresNasha]

Hello @nibix @cwperks 👋

I’ve read through the DNFOF RFC (#3905) and the discussion in this issue and related threads, and I agree with the core observation:
_cat/indices is a discovery-style operation, and failing it by default when users have access to only a subset of indices creates confusion and poor UX (especially for Dashboards and ad-hoc exploration).

From the discussion, it seems there is alignment around improving the default behavior at least for _cat/indices, without necessarily enabling DNFOF globally or changing its semantics for all actions.

Proposed approach:
Instead of toggling do_not_fail_on_forbidden globally or changing defaults cluster-wide, I’d like to propose overriding DNFOF applicability at the point of evaluation for _cat/indices only. Concretely, this would treat the action
indices:monitor/settings/get as DNFOF-eligible regardless of the global DNFOF setting, while leaving all other actions unchanged.

Following is the code changes that I plan in the file : https://github.com/opensearch-project/security/blob/main/src/main/java/org/opensearch/security/privileges/PrivilegesEvaluatorImpl.java

boolean isCatIndices =
    action0.equals("indices:monitor/settings/get");

boolean dnfofPossible =
    DNFOF_MATCHER.test(action0) &&
    (dnfofEnabled || isCatIndices);

This approach aims to :

1. Keeps DNFOF semantics unchanged for all other APIs
2. Avoids enabling DNFOF globally or altering config defaults
3. Solves the _cat/indices discovery problem directly
4. Makes the behavior explicit, local, and easy to reason about
5. Aligns with the “at least for _cat/indices” scope discussed in this issue

Can you help me answer few of the following questions :

• Does this approach align with the intended direction for 3.0.0?
• Is overriding DNFOF eligibility for a single action at evaluation time acceptable to maintainers?
• If yes, I’d be happy to pick this up and open a PR with tests.

Thanks for the thoughtful RFC and discussion, it really helps to understand deeply the tradeoffs.

[cwperks]

@wheresNasha FYI #3905 is actually a work in progress and nearly completed.

I'll reply back if there's something we can do about _cat/indices on the 3.x line. The only issue I foresee is that anything in this area is inherently breaking. While I don't think it makes sense that a user should have indices:monitor/settings/get on all indices in order to use _cat/indices, I'm not entirely sure of what the alternative should be. However, I do think treating this as simply a cluster permission (i.e. a binary of you can or cannot list all indices) is overly simplified.

Ideally, I think there's a URL param that can be passed into the request to specify dnfof (or similar) on the request-level.

[wheresNasha]

Thanks for the clarification @cwperks , that makes sense!
I completely agree that baking DNFOF behavior implicitly into _cat/indices could be problematic while #3905 is still evolving, especially given the breaking-change implications.
I really like the idea of request-scoped DNFOF. Conceptually, something like:
GET /_cat/indices?dnfof=true
or an equivalent internal flag feels much clearer than relying on a cluster-level toggle. It makes the partial-visibility behavior explicit.

Feel free to pull me in if theres anything I can potentially help or make #3905 lands closer to completion!!

[nibix]

@wheresNasha

Instead of toggling do_not_fail_on_forbidden globally or changing defaults cluster-wide, I’d like to propose overriding DNFOF applicability at the point of evaluation for _cat/indices only. Concretely, this would treat the action
indices:monitor/settings/get as DNFOF-eligible regardless of the global DNFOF setting, while leaving all other actions unchanged.

@cwperks

Ideally, I think there's a URL param that can be passed into the request to specify dnfof (or similar) on the request-level.

To a certain extent, we will get that in #5827

This will have the following behavior for /_cat/indices:

• /_cat/indices called by an admin user will list all indices
• /_cat/indices called by a user with limited privileges will only list the indices the user has privileges for
• /_cat/indices/my_index1,my_index2,other_index called by a user who has only privileges for my_index*, but not for other_index* will fail with a 403 error.
• /_cat/indices/my_index1,my_index2,other_index?ignore_unavailable=true called by a user who has only privileges for my_index*, but not for other_index* will return information for my_index1 and my_index2. other_index will be missing from the result.

The basic rule is: Wildcards can only match on indices the user has privilges for. If they would also match on other indices, limited users would often run into 403 errors.

If an index expression references an index directly, without wildcard, we wont silently ignore that index. Only if the user specifies ignore_unavailable, we assume that they want to have such indices silently ignored.

[wheresNasha]

@nibix
Thanks for the detailed explanation, this makes a lot of sense and honestly feels like the clean behavior model.

I really like the distinction between wildcard expansion vs explicitly referenced indices, and how ignore_unavailable becomes the explicit signal for partial visibility. That keeps the semantics predictable while avoiding silent failures.

I’m happy to help move this forward if there are pieces I can pick up whether that’s helping with tests, documentation, or validating edge cases once #5827 lands (especially around mixed wildcard + explicit index expressions).

One small thought: it might be worth documenting this behavior clearly in the _cat/indices API docs once implemented, since the privilege-based wildcard behavior is subtle but important for users to understand.

Please feel free to loop me in if there’s anything useful I can contribute 🙌