Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
de3bb1d
adding azdo test suit and preview module
SebastianClaesson May 11, 2025
90404c8
Adding readme
SebastianClaesson May 11, 2025
8d9f3c2
adding updated readme
SebastianClaesson May 11, 2025
aa8b936
Adding Azdo tests
SebastianClaesson Dec 9, 2025
8d7318a
Updating readme and setting functions in the correct folder
SebastianClaesson Dec 9, 2025
94d7523
Merge branch 'maester365:main' into feature/azdo
SebastianClaesson Dec 9, 2025
fc83a14
Relocating parent folder
SebastianClaesson Dec 9, 2025
13c55d8
Merge branch 'feature/azdo' of https://github.com/SebastianClaesson/m…
SebastianClaesson Dec 9, 2025
86125a8
updating readmes and powershell scripts
SebastianClaesson Dec 11, 2025
0ba07d8
Fix typos and improve formatting in documentation
SamErde Jan 21, 2026
c12ab8b
Refine grammar in Test-AzdoArtifactsExternalPackageProtectionToken.md
SamErde Jan 21, 2026
bdfa2ff
Fix typo in description of security policy check
SamErde Jan 21, 2026
f4f68c7
Fix grammar and punctuation in Test-AzdoAuditStream.md
SamErde Jan 21, 2026
ca1d7da
Refine description for Test-AzdoAuditStream function
SamErde Jan 21, 2026
6005abd
Fix grammar and clarity in Test-AzdoEnforceAADConditionalAccess.md
SamErde Jan 21, 2026
0c4b342
Update README.md with clearer test descriptions
SamErde Jan 21, 2026
25dcaff
Update powershell/public/maester/azdo/Test-AzdoOrganizationLimitJobAu…
SamErde Jan 21, 2026
9d40147
Merge branch 'maester365:main' into feature/azdo
SebastianClaesson Jan 23, 2026
acd7ecc
Renaming Azdo to AzureDevOps
SebastianClaesson Jan 23, 2026
2fa5cb3
Fixed typo and information
SebastianClaesson Jan 23, 2026
046bfea
Updated description
SebastianClaesson Jan 23, 2026
223fa56
Removing dot sourcing as the test cmdlets are to be part of the module
SebastianClaesson Jan 23, 2026
5adefc1
Merge branch 'feature/azdo' of https://github.com/SebastianClaesson/m…
SebastianClaesson Jan 23, 2026
0a4eb89
Fix typo in SSH authentication documentation
SamErde Jan 31, 2026
09caf7f
Fix typo in Test-AzdoOrganizationStageChooser.md
SamErde Jan 31, 2026
04fd334
Update powershell/public/maester/azuredevops/Test-AzdoOrganizationLim…
SamErde Jan 31, 2026
a7e76b6
Merge branch 'maester365:main' into feature/azdo
SebastianClaesson Feb 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion powershell/Maester.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,38 @@
'Test-MtXspmCriticalCredsOnDevicesWithNonCriticalAccounts',
'Test-MtXspmPublicRemotelyExploitableHighExposureDevices',
'Test-MtXspmCriticalCredentialsOnNonTpmProtectedDevices',
'Test-MtXspmCriticalCredentialsOnNonCredGuardProtectedDevices'
'Test-MtXspmCriticalCredentialsOnNonCredGuardProtectedDevices',
'Test-AzdoAllowRequestAccessToken',
'Test-AzdoAllowTeamAdminsInvitationsAccessToken',
'Test-AzdoArtifactsExternalPackageProtectionToken',
'Test-AzdoAuditStream',
'Test-AzdoEnforceAADConditionalAccess',
'Test-AzdoExternalGuestAccess',
'Test-AzdoFeedbackCollection',
'Test-AzdoLogAuditEvent',
'Test-AzdoOrganizationAutomaticEnrollmentAdvancedSecurityNewProject',
'Test-AzdoOrganizationBadgesArePrivate',
'Test-AzdoOrganizationCreationClassicBuildPipeline',
'Test-AzdoOrganizationCreationClassicReleasePipeline',
'Test-AzdoOrganizationLimitJobAuthorizationScopeNonReleasePipeline',
'Test-AzdoOrganizationLimitJobAuthorizationScopeReleasePipeline',
'Test-AzdoOrganizationLimitVariablesAtQueueTime',
'Test-AzdoOrganizationOwner',
'Test-AzdoOrganizationProtectAccessToRepository',
'Test-AzdoOrganizationRepositorySettingsDisableCreationTFVCRepo',
'Test-AzdoOrganizationRepositorySettingsGravatarImage',
'Test-AzdoOrganizationStageChooser',
'Test-AzdoOrganizationStorageUsage',
'Test-AzdoOrganizationTaskRestrictionsDisableMarketplaceTask',
'Test-AzdoOrganizationTaskRestrictionsDisableNode6Task',
'Test-AzdoOrganizationTaskRestrictionsShellTaskArgumentValidation',
'Test-AzdoOrganizationTriggerPullRequestGitHubRepository',
'Test-AzdoProjectCollectionAdministrator',
'Test-AzdoPublicProject',
'Test-AzdoResourceUsageProject',
'Test-AzdoResourceUsageWorkItemTag',
'Test-AzdoSSHAuthentication',
'Test-AzdoThirdPartyAccessViaOauth'

# Cmdlets to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no cmdlets to export.
CmdletsToExport = @()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Request access to Azure DevOps by e-mail notifications to administrators SHOULD BE disabled.

Rationale: Access control to Azure DevOps is to be a controlled process where access is granted and tracked.

#### Remediation action:
Disable the policy to stop these requests and notifications.
1. Sign in to your organization
2. Choose Organization settings.
3. Select Policies, locate the Request Access policy and toggle it to off.
4. Provide the URL to your internal process for gaining access. Users see this URL in the error report when they try to access the organization or a project within the organization that they don't have permission to access.

**Results:**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the detailed 401 go to users in the organization and the 404 go to users not in the organization?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do agree!
Seems the experience has changed since;

MicrosoftDocs/azure-devops-docs@91c4410#diff-a28e8dd823f1493651d0c6322e6b2dd976bccab79e279de5aea876ce20272729R44

I'll update with the new information the article.

When users try to access a project without the required permissions, the error message includes the request access URL. This link is shown on the error page to maintain confidentiality, regardless of whether the project exists.

#### Related links

* [Azure DevOps Security - Disable your organization's Request Access policy](https://go.microsoft.com/fwlink/?linkid=2113172)
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
<#
.SYNOPSIS
Returns a boolean depending on the configuration.

.DESCRIPTION
Checks the status of the 'Request Access' policy in Azure DevOps to prevent users from requesting access to your organization or projects.
When this policy is enabled, users can request access, and administrators receive email notifications for review and approval.
Disabling the policy stops these requests and notifications, helping you control access more tightly.

https://go.microsoft.com/fwlink/?linkid=2113172

.EXAMPLE
```
Test-AzdoAllowRequestAccessToken
```

Returns a boolean depending on the configuration.

.LINK
https://maester.dev/docs/commands/Test-AzdoAllowRequestAccessToken
#>

function Test-AzdoAllowRequestAccessToken {
[CmdletBinding()]
[OutputType([bool])]
param()

if ($null -eq (Get-ADOPSConnection)['Organization']) {
Write-verbose 'Not connected to Azure DevOps'
Add-MtTestResultDetail -SkippedBecause Custom -SkippedCustomReason 'Not connected to Azure DevOps'
return $null
}

$UserPolicies = Get-ADOPSOrganizationPolicy -PolicyCategory 'User'
$Policy = $UserPolicies.policy | where-object -property name -eq 'Policy.AllowRequestAccessToken'
$result = $Policy.effectiveValue
if ($result) {
$resultMarkdown = "When enabled, this policy allows users to request access, triggering email notifications to administrators for review and approval."
}
else {
$resultMarkdown = "Well done. Disabling the policy stops these requests and notifications."
}

Add-MtTestResultDetail -Result $resultMarkdown -Severity 'High'

return $result
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Access to Azure DevOps SHOULD BE a controlled process provided by the IAM team or relevant Azure DevOps administrators roles.

Rationale: By default, all administrators can invite new users to their Azure DevOps organization.
Disabling this policy prevents Team and Project Administrators from inviting new users.
However, Project Collection Administrators (PCAs) can still add new users to the organization regardless of the policy status.
Additionally, if a user is already a member of the organization, Project and Team Administrators can add that user to specific projects.

#### Remediation action:
Disable the policy to stop these invitations.
1. Sign in to your organization
2. Choose Organization settings.
3. Select Policies, locate the **Allow team and project administrators to invite new users** policy and toggle it to off.
4. Now, only Project Collection Administrators can invite new users to Azure DevOps.

> Project and Team Administrators can directly add users to their projects through the permissions blade. However, if they attempt to add users through the Add Users button located in the Organization settings > Users section, it's not visible to them. Adding a user directly through Project settings > Permissions doesn't result in the user appearing automatically in the Organization settings > Users list. For the user to be reflected in the Users list, they must sign in to the system.

#### Related links

* [Azure DevOps Security - Restrict administrators from inviting new users](https://aka.ms/azure-devops-invitations-policy)
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<#
.SYNOPSIS
Returns a boolean depending on the configuration.

.DESCRIPTION
By default, all administrators can invite new users to their Azure DevOps organization.
Disabling this policy prevents Team and Project Administrators from inviting new users or adding Entra groups.
However, Project Collection Administrators (PCAs) can still add new users and Entra groups to the organization regardless of the policy status.
Additionally, if a user is already a member of the organization, Project and Team Administrators can add that user to specific projects.

https://aka.ms/azure-devops-invitations-policy

.EXAMPLE
```
Test-AzdoAllowTeamAdminsInvitationsAccessToken
```

Returns a boolean depending on the configuration.

.LINK
https://maester.dev/docs/commands/Test-AzdoAllowTeamAdminsInvitationsAccessToken
#>

function Test-AzdoAllowTeamAdminsInvitationsAccessToken {
[CmdletBinding()]
[OutputType([bool])]
param()

if ($null -eq (Get-ADOPSConnection)['Organization']) {
Write-verbose 'Not connected to Azure DevOps'
Add-MtTestResultDetail -SkippedBecause Custom -SkippedCustomReason 'Not connected to Azure DevOps'
return $null
}

$PrivacyPolicies = Get-ADOPSOrganizationPolicy -PolicyCategory 'User'
$Policy = $PrivacyPolicies.policy | where-object -property name -eq 'Policy.AllowTeamAdminsInvitationsAccessToken'
$result = $Policy.effectiveValue
if ($result) {
$resultMarkdown = "Team and project administrators is allowed to invite new users"
}
else {
$resultMarkdown = "Well done. Enrolling to your Azure DevOps organization should be a controlled process."
}

Add-MtTestResultDetail -Result $resultMarkdown -Severity 'High'

return $result
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Externally sourced package versions SHOULD BE manually approved for internal use to prevent malicious packages from a public registry being inadvertently consumed.

Rationale: Previously, Azure Artifacts feeds presented package versions from all of its upstream sources. This includes package versions that were originally pushed to an Azure Artifacts feed (internally sourced) and package versions from common public repositories like npmjs.com, NuGet.org, Maven Central, and PyPI (externally sourced).

Configure a policy for additional security for your private feeds by limiting access to externally sourced packages when internally sources packages are already present. This provides a new layer of security, which prevents malicious packages from a public registry being inadvertently consumed. These changes will not affect any package versions that are already in use or cached in your feed.

#### Remediation action:

Enable the policy to opt-in for additional protective behavior.

1. Sign in to your organization
2. Choose Organization settings.
3. Click on policies under the security section
4. In the security policies section, toggle on ‘Additional protections when using public package registries’

**Results:**
The security behavior applies:
when an internally sourced version is already in your feed, or
when consuming a package from your feed for the first time (i.e. it is not yet in your feed), and at least one of the versions available from an upstream is internally sourced.
With the new behavior, any versions from the public registry will be blocked and not made available to download. You are able to configure the upstream behavior to allow externally sourced package versions if you choose to.

#### Related links

* [Microsoft Devblogs - Changes to Azure Artifacts Upstream Behavior](https://devblogs.microsoft.com/devops/changes-to-azure-artifact-upstream-behavior/)
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
<#
.SYNOPSIS
Returns a boolean depending on the configuration.

.DESCRIPTION
Checks the policy for additional security for your private feeds by limiting access to externally sourced packages when internally sourced packages are already present.
This provides a new layer of security, which prevents malicious packages from a public registry being inadvertently consumed.
These changes will not affect any package versions that are already in use or cached in your feed.

https://devblogs.microsoft.com/devops/changes-to-azure-artifact-upstream-behavior

.EXAMPLE
```
Test-AzdoArtifactsExternalPackageProtectionToken
```

Returns a boolean depending on the configuration.

.LINK
https://maester.dev/docs/commands/Test-AzdoArtifactsExternalPackageProtectionToken
#>

function Test-AzdoArtifactsExternalPackageProtectionToken {
[CmdletBinding()]
[OutputType([bool])]
param()

if ($null -eq (Get-ADOPSConnection)['Organization']) {
Write-verbose 'Not connected to Azure DevOps'
Add-MtTestResultDetail -SkippedBecause Custom -SkippedCustomReason 'Not connected to Azure DevOps'
return $null
}

$SecurityPolicies = Get-ADOPSOrganizationPolicy -PolicyCategory 'Security'
$Policy = $SecurityPolicies.policy | where-object -property name -eq 'Policy.ArtifactsExternalPackageProtectionToken'
$result = $Policy.effectiveValue
if ($result) {
$resultMarkdown = "Well done. Your Azure DevOps tenant limits access to externally sourced packages when internally sources packages are already present."
}
else {
$resultMarkdown = "Your tenant should prefer to use internal source packages when present"
}



Add-MtTestResultDetail -Result $resultMarkdown -Severity 'High'

return $result
}
24 changes: 24 additions & 0 deletions powershell/public/maester/azuredevops/Test-AzdoAuditStream.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Audit logs SHOULD BE retained according to your organization's needs, and protected from purging.

Rationale: Send auditing data to other Security Incident and Event Management (SIEM) tools and open new possibilities, such as the ability to trigger alerts for specific events, create views on auditing data, and perform anomaly detection. Setting up a stream also allows you to store more than 90-days of auditing data, which is the maximum amount of data that Azure DevOps keeps for your organizations.

#### Remediation action:

Create an audit stream, which sends data to other locations for further processing.

1. Sign in to your organization.
2. Choose Organization settings.
3. Select Auditing.
> If you don't see Auditing in Organization Settings, then auditing is not currently enabled for your organization. Someone in the organization owner or Project Collection Administrators (PCAs) group must enable Auditing in Organization Policies. You will then be able to see events on the Auditing page if you have the appropriate permissions.
1. Go to the Streams tab, and then select New stream.
2. Select the stream target that you want to configure, and then select from the following instructions to set up your stream target type.
1. Splunk
2. Event Grid
3. Azure Monitor Log

**Results:**
Audit streams represent a pipeline that flows audit events from your Azure DevOps organization to a stream target. Every half hour or less, new audit events are bundled and streamed to your targets.

#### Related links

* [Azure DevOps Security - Create audit streaming](https://learn.microsoft.com/en-us/azure/devops/organizations/audit/auditing-streaming?view=azure-devops)
57 changes: 57 additions & 0 deletions powershell/public/maester/azuredevops/Test-AzdoAuditStream.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
<#
.SYNOPSIS
Returns a boolean depending on the configuration.

.DESCRIPTION
Sends auditing data to Security Incident and Event Management (SIEM) tools and open new possibilities,
such as the ability to trigger alerts for specific events, create views on auditing data, and perform
anomaly detection. Setting up a stream also allows you to store more than 90-days of auditing data,
which is the maximum amount of data that Azure DevOps keeps for your organizations.

https://learn.microsoft.com/en-us/azure/devops/organizations/audit/auditing-streaming?view=azure-devops

.EXAMPLE
```
Test-AzdoAuditStream
```

Returns a boolean depending on the configuration.

.LINK
https://maester.dev/docs/commands/Test-AzdoAuditStream
#>

function Test-AzdoAuditStream {
[CmdletBinding()]
[OutputType([bool])]
param()

if ($null -eq (Get-ADOPSConnection)['Organization']) {
Write-verbose 'Not connected to Azure DevOps'
Add-MtTestResultDetail -SkippedBecause Custom -SkippedCustomReason 'Not connected to Azure DevOps'
return $null
}

$AuditStreams = Get-ADOPSAuditStreams

if ($AuditStreams) {
if ('Enabled' -in $AuditStreams.status) {
$resultMarkdown = "Well done. Audit logs have been configured for long-term storage and purge protection."
$result = $true
}
else {
$resultMarkdown = "Audit Streams have been configured for long-term storage and purge protection but is not enabled."
$result = $false
}
}
else {
$resultMarkdown = "Audit Streams have not been configured for long-term storage and purge protection."
$result = $false
}



Add-MtTestResultDetail -Result $resultMarkdown -Severity 'Critical'

return $result
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
Conditional Access Policies SHOULD BE configured for Microsoft Entra ID-backed organizations.

Rationale: When you sign in to the web portal of a Microsoft Entra ID-backed organization, Microsoft Entra ID always performs validation for any Conditional Access Policies (CAPs) set by tenant administrators.

#### Remediation action:
Disable the policy to stop these requests and notifications.
1. Sign in to your organization
2. Choose Organization settings.
3. Select Policies, and then toggle your policy to on or off as needed.
Comment on lines +6 to +9
Copy link

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remediation step says "Disable the policy to stop these requests and notifications", but this test and the rest of the document argue that enforcing Conditional Access is desired. To avoid confusing readers, this should instruct users to enable the relevant Conditional Access policy in Azure DevOps/Entra ID rather than disable it.

Suggested change
Disable the policy to stop these requests and notifications.
1. Sign in to your organization
2. Choose Organization settings.
3. Select Policies, and then toggle your policy to on or off as needed.
Enable and enforce the relevant Conditional Access policy in Microsoft Entra ID/Azure DevOps so that these requests are evaluated according to your security requirements.
1. Sign in to your organization
2. Choose Organization settings.
3. Select Policies, and then toggle the relevant Conditional Access policy to **On** to enforce it.

Copilot uses AI. Check for mistakes.
1. If the “Enable IP Conditional Access policy Validation” organization policy is enabled, we check IP fencing policies on both web and non-interactive flows, such as non-Microsoft client flows (e.g., using a PAT with git operations).
2. Sign-in policies might be enforced for PATs as well. Using PATs to make Microsoft Entra ID calls requires adherence to any sign-in policies that are set. For example, if a sign-in policy requires that a user sign in every seven days, you must also sign in every seven days to continue using PATs for Microsoft Entra ID requests.
> We support MFA policies on web flows only. For non-interactive flows, if they don't satisfy the conditional access policy, the user isn't prompted for MFA and gets blocked instead.
> We support IP-fencing conditional access policies (CAPs) for both IPv4 and IPv6 addresses. If your IPv6 address is being blocked, ensure that the tenant administrator configured CAPs to allow your IPv6 address. Additionally, consider including the IPv4-mapped address for any default IPv6 address in all CAP conditions.

**Results:**
When you sign in to the web portal of a Microsoft Entra ID-backed organization, Microsoft Entra ID always performs validation for any Conditional Access Policies (CAPs) set by tenant administrators.

#### Related links

* [Azure DevOps Security - Conditional Access Policies support on Azure DevOps](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/change-application-access-policies?view=azure-devops#cap-support-on-azure-devops)
Loading