Skip to content

feat: created script for local keycloak setup #1037

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alpakhomov wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: created script for local keycloak setup #1037

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Now services are running, but to start using BadgerDoc, some additional configur

## Keycloak local configuration

_It's a good idea to automate this section_
_You can run [configure_local_keycloak](scripts/configure_local_keycloak.sh) script which will update local Keycloak config (steps 2 to 10) for you. Note: you should have `jq` installed!_

Important! This is not secure configuration, follow [KeyCloak best practices](https://www.keycloak.org/server/configuration-production) to setup on production environment

Expand Down Expand Up @@ -164,7 +164,7 @@ Airflow runs using its own resources (PostgreSQL, Redis, Flower) without sharing
cp airflow/.env.example airflow/.env
```

To setup service account you need to configure Keycloak for BadgerDoc first.
To setup service account you need to configure Keycloak for BadgerDoc first. _If you used [configure_local_keycloak](scripts/configure_local_keycloak.sh) script to setup local Keycloak - steps 2 to 4 should already be done_

2. Setup service account. Login into Keycloak using url http://127.0.0.1:8082/auth and `admin:admin` as credentials. Select Clients -> badgerdoc-internal -> Service Accounts Roles -> Find Service Account User and click "service-account-badgerdoc-internal". Then select Attributes tab and add `tenants:local` attribute like you did it for `admin`.

Expand Down
308 changes: 308 additions & 0 deletions scripts/configure_local_keycloak.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,308 @@
#!/bin/bash

KEYCLOAK_USER="${KEYCLOAK_USER:=admin}"
KEYCLOAK_PASSWORD="${KEYCLOAK_PASSWORD:=admin}"
KEYCLOAK_URL="${KEYCLOAK_URL:=http://localhost:8082/auth}"

ACCESS_TOKEN=$(curl -s -X POST ${KEYCLOAK_URL}/realms/master/protocol/openid-connect/token \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "username=${KEYCLOAK_USER}" \
-d "password=${KEYCLOAK_PASSWORD}" \
-d "grant_type=password" \
-d "client_id=admin-cli" | jq -r '.access_token')

# Go to Realm Settings -> Keys and disable RSA-OAEP algorithm. It will help to avoid issue explainded here https://github.com/jpadilla/pyjwt/issues/722
RSA_OAEP_COMPONENT_ID=$(curl -s -X GET ${KEYCLOAK_URL}/admin/realms/master/components?type=org.keycloak.keys.KeyProvider \
-H "Authorization: Bearer ${ACCESS_TOKEN}" | jq -r '.[] | select(.config.algorithm[]? == "RSA-OAEP") | .id')

curl -s -X PUT ${KEYCLOAK_URL}/admin/realms/master/components/${RSA_OAEP_COMPONENT_ID} \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"id": "${RSA_OAEP_COMPONENT_ID}",
"name": "rsa-enc-generated",
"providerId": "rsa-enc-generated",
"providerType": "org.keycloak.keys.KeyProvider",
"config": {
"priority": ["100"],
"enabled": ["false"],
"active": ["false"],
"algorithm": ["RSA-OAEP"],
"keySize": ["2048"]
}
}'

if [ $? -ne 0 ]; then
echo -e "\nFailed to disable RSA-OAEP key provider."
exit $?
else
echo -e "\nRSA-OAEP key provider disabled successfully."
fi

# # Add tenant attribute to admin user, go to Users -> select admin -> go to Attributes -> create attribute tenants:local, and save
KEYCLOAK_USER_ID=$(curl -s -X GET ${KEYCLOAK_URL}/admin/realms/master/users?username=${KEYCLOAK_USER} \
-H "Authorization: Bearer ${ACCESS_TOKEN}" | jq -r '.[0].id')
curl -s -X PUT ${KEYCLOAK_URL}/admin/realms/master/users/${KEYCLOAK_USER_ID}/ \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"attributes": {
"tenants": ["local"]
}
}'
if [ $? -ne 0 ]; then
echo -e "\nFailed to add tenant attribute to admin user."
exit $?
else
echo -e "\nTenant attribute added to admin user successfully."
fi

# # Go to Clients -> admin-cli -> Mappers -> Create and fill form with following values:
# # Param Value
# # Protocol openid-connect
# # Name tenants
# # Mapper Type User Attribute
# # User Attribute tenants
# # Token Claim Name tenants
# # Claim JSON Type string
# # Add to ID token On
# # Add to access token On
# # Add to userinfo On
# # Multivalued On
# # Aggregate attribute values On
CLIENT_ID=$(curl -s -X GET ${KEYCLOAK_URL}/admin/realms/master/clients?clientId=admin-cli \
-H "Authorization: Bearer ${ACCESS_TOKEN}" | jq -r '.[0].id')
curl -s -X POST ${KEYCLOAK_URL}/admin/realms/master/clients/${CLIENT_ID}/protocol-mappers/models \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"name": "tenants",
"protocol": "openid-connect",
"protocolMapper": "oidc-usermodel-attribute-mapper",
"consentRequired": false,
"config": {
"user.attribute": "tenants",
"claim.name": "tenants",
"jsonType.label": "String",
"access.token.claim": true,
"id.token.claim": true,
"userinfo.token.claim": true,
"multivalued": true,
"aggregate.attrs": true
}
}'
if [ $? -ne 0 ]; then
echo -e "\nFailed to create tenants mapper."
exit $?
else
echo -e "\nTenants mapper created successfully."
fi

# # Go to Client Scopes -> Find roles -> Scope and select admin in list to add to Assigned Roles, then go to Mappers and ensure that only 2 mappers exists: realm roles and client roles. Delete all other mappers
CLIENT_SCOPE_ID=$(curl -s -X GET ${KEYCLOAK_URL}/admin/realms/master/client-scopes \
-H "Authorization: Bearer ${ACCESS_TOKEN}" | jq -r '.[] | select(.name == "roles") | .id')
ADMIN_ROLE_ID=$(curl -s -X GET "${KEYCLOAK_URL}/admin/realms/master/roles" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" | jq -r '.[] | select(.name == "admin") | .id')
curl -s -X POST "${KEYCLOAK_URL}/admin/realms/master/client-scopes/$CLIENT_SCOPE_ID/scope-mappings/realm" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '[
{
"id": "'$ADMIN_ROLE_ID'",
"name": "admin"
}
]'

curl -s -X GET "${KEYCLOAK_URL}/admin/realms/master/client-scopes/$CLIENT_SCOPE_ID/protocol-mappers/models" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" | \
jq -r '.[] | select(.name != "realm roles" and .name != "client roles") | .id' | \
while read MAPPER_ID; do
echo -e "Deleting mapper: $MAPPER_ID"
curl -s -X DELETE "${KEYCLOAK_URL}/admin/realms/master/client-scopes/$CLIENT_SCOPE_ID/protocol-mappers/models/$MAPPER_ID" \
-H "Authorization: Bearer ${ACCESS_TOKEN}"
done
if [ $? -ne 0 ]; then
echo -e "\nFailed to create admin mapper in Find roles client scope."
exit $?
else
echo -e "\nAdmin mapper created successfully in Find roles client scope."
fi

# # Go to Clients -> Create -> Fill form and save

# # Param Value
# # Client ID badgerdoc-internal
# # Client Protocol openid-connect
curl -s -X POST ${KEYCLOAK_URL}/admin/realms/master/clients \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"clientId": "badgerdoc-internal",
"protocol": "openid-connect",
"enabled": true,
"publicClient": false,
"serviceAccountsEnabled": true,
"authorizationServicesEnabled": true,
"redirectUris": ["_"],
"webOrigins": ["_"]
}'
if [ $? -ne 0 ]; then
echo -e "\nFailed to create badgerdoc-internal client."
exit $?
else
echo -e "\nBadgerdoc-internal client created successfully."
fi

# Now you can Credentials tab, open it and copy Secret
# Then Client ID and Secret must be set to .env as KEYCLOAK_SYSTEM_USER_CLIENT=badgerdoc-internal and KEYCLOAK_SYSTEM_USER_SECRET to copied key
CLIENT_UUID=$(curl -s -X GET ${KEYCLOAK_URL}/admin/realms/master/clients \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" | jq -r '.[] | select(.clientId == "badgerdoc-internal") | .id')

CLIENT_SECRET=$(curl -s -X GET ${KEYCLOAK_URL}/admin/realms/master/clients/${CLIENT_UUID}/client-secret \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" | jq -r '.value')

# Go to Clients -> Find badgerdoc-internal -> Service Account Roles -> Client Roles -> master-realm -> Find view-users and view-identity-providers in Available Roles and add to Assigned Roles

SERVICE_ACCOUNT_USER_ID=$(curl -s -X GET "${KEYCLOAK_URL}/admin/realms/master/clients/$CLIENT_UUID/service-account-user" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" | jq -r '.id')

MASTER_REALM_CLIENT_ID=$(curl -s -X GET "${KEYCLOAK_URL}/admin/realms/master/clients" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" | jq -r '.[] | select(.clientId == "master-realm") | .id')

ROLES_JSON=$(curl -s -X GET "${KEYCLOAK_URL}/admin/realms/master/clients/$MASTER_REALM_CLIENT_ID/roles" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" | \
jq '[.[] | select(.name == "view-users" or .name == "view-identity-providers")]')


curl -s -X POST "${KEYCLOAK_URL}/admin/realms/master/users/$SERVICE_ACCOUNT_USER_ID/role-mappings/clients/$MASTER_REALM_CLIENT_ID" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d "$ROLES_JSON"

if [ $? -ne 0 ]; then
echo -e "\nFailed to assign roles to badgerdoc-internal service account."
exit $?
else
echo -e "\nRoles assigned to badgerdoc-internal service account successfully."
fi

# Go to Roles -> add roles: presenter, manager, role-annotator, annotator, engineer. Open admin role, go to Composite Roles -> Realm Roles and add all these roles

ROLES=("presenter" "manager" "role-annotator" "annotator" "engineer")

for role in "${ROLES[@]}"; do
echo -e "\nCreating role: $role"
curl -s -X POST "${KEYCLOAK_URL}/admin/realms/master/roles" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"name\": \"$role\", \"description\": \"$role role\"}"
done

ROLES_JSON=$(curl -s -X GET "${KEYCLOAK_URL}/admin/realms/master/roles" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" | \
jq '[.[] | select(.name == "presenter" or .name == "manager" or .name == "role-annotator" or .name == "annotator" or .name == "engineer") | {id: .id, name: .name}]')


curl -s -X POST "${KEYCLOAK_URL}/admin/realms/master/roles/admin/composites" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d "$ROLES_JSON"

if [ $? -ne 0 ]; then
echo -e "\nFailed to assign roles to badgerdoc-internal service account."
exit $?
else
echo -e "\nRoles assigned to badgerdoc-internal service account successfully."
fi

# Go to Realm Settings -> Tokens -> Find Access Token Lifespan and set 1 Days
curl -s -X PUT "${KEYCLOAK_URL}/admin/realms/master" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"accessTokenLifespan": 86400
}'

# Set up Airflow as a pipeline service in local mode
# Setup service account. Login into Keycloak using url http://127.0.0.1:8082/auth and admin:admin as credentials. Select Clients -> badgerdoc-internal -> Service Accounts Roles -> Find Service Account User and click "service-account-badgerdoc-internal". Then select Attributes tab and add tenants:local attribute like you did it for admin.
curl -s -X PUT ${KEYCLOAK_URL}/admin/realms/master/users/${SERVICE_ACCOUNT_USER_ID}/ \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"attributes": {
"tenants": ["local"]
}
}'
if [ $? -ne 0 ]; then
echo -e "\nFailed to add tenant attribute to badgerdoc-internal service account user."
exit $?
else
echo -e "\nTenant attribute added to badgerdoc-internal service account user successfully."
fi

# Go to Role Mappings and assign admin and default-roles-master
DEFAULT_ROLES_ID=$(curl -s -X GET "${KEYCLOAK_URL}/admin/realms/master/roles/default-roles-master" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" | jq -r '.id')

# Assign roles
curl -s -X POST "${KEYCLOAK_URL}/admin/realms/master/users/$SERVICE_ACCOUNT_USER_ID/role-mappings/realm" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '[
{"id": "'$ADMIN_ROLE_ID'", "name": "admin"},
{"id": "'$DEFAULT_ROLES_ID'", "name": "default-roles-master"}
]'
if [ $? -ne 0 ]; then
echo -e "\nFailed to assign roles to badgerdoc-internal service account user."
exit $?
else
echo -e "\nRoles assigned to badgerdoc-internal service account user successfully."
fi

# Go to Clients -> badgerdoc-internal -> Mappers -> Create and fill form:

# Param Value
# Protocol openid-connect
# Name tenants
# Mapper Type User Attribute
# User Attribute tenants
# Token Claim Name tenants
# Claim JSON Type string
# Add to ID token On
# Add to access token On
# Add to userinfo On
# Multivalued On
# Aggregate attribute values On
curl -s -X POST ${KEYCLOAK_URL}/admin/realms/master/clients/${CLIENT_UUID}/protocol-mappers/models \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"name": "tenants",
"protocol": "openid-connect",
"protocolMapper": "oidc-usermodel-attribute-mapper",
"consentRequired": false,
"config": {
"user.attribute": "tenants",
"claim.name": "tenants",
"jsonType.label": "String",
"access.token.claim": true,
"id.token.claim": true,
"userinfo.token.claim": true,
"multivalued": true,
"aggregate.attrs": true
}
}'
if [ $? -ne 0 ]; then
echo -e "\nFailed to create tenants mapper."
exit $?
else
echo -e "\nTenants mapper created successfully."
fi

COLOR=$'\e[0;31m'
echo -e "Change value of KEYCLOAK_SYSTEM_USER_SECRET in Badgerdoc .env and Airflow .env to ${COLOR}${CLIENT_SECRET}${COLOR}"