THREESCALE-11081 The ability to set the application_id & application_key in the application CRD creation

[tkan145]

What

THREESCALE-11081

Close #1042

Verification steps

• Checkout this branch
• Prepare the cluster

make cluster/prepare/local

• Install APIM

cat << EOF | oc create -f -
kind: Secret
apiVersion: v1
metadata:
  name: s3-credentials
  namespace: $NAMESPACE
data:
  AWS_ACCESS_KEY_ID: c29tZXRoaW5nCg==
  AWS_BUCKET: c29tZXRoaW5nCg==
  AWS_REGION: dXMtd2VzdC0xCg==
  AWS_SECRET_ACCESS_KEY: c29tZXRoaW5nCg==
type: Opaque
EOF

DOMAIN=$(oc get routes console -n openshift-console -o json | jq -r '.status.ingress[0].routerCanonicalHostname' | sed 's/router-default.//')
cat << EOF | oc create -f -
kind: APIManager
apiVersion: apps.3scale.net/v1alpha1
metadata:
  name: 3scale
  namespace: $NAMESPACE
spec:
  wildcardDomain: $DOMAIN
  apicast:
    stagingSpec:
      replicas: 0
    productionSpec:
      replicas: 0
  system:
    fileStorage:
      simpleStorageService:
        configurationSecretRef:
          name: s3-credentials
  externalComponents:
    backend:
      redis: true
    system:
      database: true
      redis: true
EOF

• Start the operator

make run

• Wait for the installation to finish
• Setup product/backend

Details

# Get the ADMIN_URL and ADMIN_ACCESS_TOKEN from apimanager and system-seed secret
DOMAIN=$(oc get routes console -n openshift-console -o json | jq -r '.status.ingress[0].routerCanonicalHostname' | sed 's/router-default.//')
ADMIN_ACCESS_TOKEN=$(oc get secret system-seed -n 3scale-test -o jsonpath="{.data.ADMIN_ACCESS_TOKEN}"| base64 --decode)
oc project 3scale-test
# Create the required secrets for Accounts, products and backends. 
oc apply -f - <<EOF
---
apiVersion: v1
kind: Secret
metadata:
  name: mytenant
type: Opaque
stringData:
  adminURL: https://3scale-admin.$DOMAIN
  token: $ADMIN_ACCESS_TOKEN
EOF
# user secret
oc apply -f - <<EOF
---
apiVersion: v1
kind: Secret
metadata:
  name: myusername01
stringData:
  password: "123456"
EOF
# Developer User
oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: DeveloperUser
metadata:
  name: developeruser01
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  developerAccountRef:
    name: developeraccount01
  email: myusername01@example.com
  passwordCredentialsRef:
    name: myusername01
  providerAccountRef:
    name: mytenant
  role: admin
  username: myusername01
EOF
sleep 30
# TODO check for developer user completed
oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: DeveloperAccount
metadata:
  name: developeraccount01
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  orgName: 3scale-test
  providerAccountRef:
    name: mytenant
EOF
# deploy httpbin and use it as the backend
oc new-project httpbin
oc new-app quay.io/trepel/httpbin
oc get svc
oc scale deployment/httpbin --namespace httpbin --replicas=8 
oc project 3scale-test

# create backend
oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: Backend
metadata:
  name: backend1-cr
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  mappingRules:
    - httpMethod: GET
      increment: 1
      last: true
      metricMethodRef: GET_one
      pattern: /
    - httpMethod: POST
      increment: 1
      metricMethodRef: hits
      pattern: /
  methods:
    GET_one:
      description: Number of API hits
      friendlyName: GET_one
  metrics:
    hits:
      description: Number of API hits
      friendlyName: Hits
      unit: hits
  name: backend1
  privateBaseURL: 'http://httpbin.httpbin.svc:8080'
  systemName: backend1
EOF
# Product
oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: Product
metadata:
  name: product1-cr
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  applicationPlans:
    plan01:
      name: "My Plan 01"
    plan02:
      name: "My Plan 02"
  deployment:
    apicastHosted:
      authentication:
        userkey:
          authUserKey: token
  name: product1
  backendUsages:
    backend1:
      path: /
  mappingRules:
    - httpMethod: GET
      pattern : "/"
      metricMethodRef: hits
      increment: 1
    - httpMethod: POST
      pattern : "/"
      metricMethodRef: hits
      increment: 1    
EOF
oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: Product
metadata:
  name: product2-cr
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  applicationPlans:
    plan01:
      name: "My Plan 01"
    plan02:
      name: "My Plan 02"
  deployment:
    apicastHosted:
      authentication:
        appKeyAppID:
          appID: token
  name: product2
  backendUsages:
    backend1:
      path: /
  mappingRules:
    - httpMethod: GET
      pattern : "/"
      metricMethodRef: hits
      increment: 1
    - httpMethod: POST
      pattern : "/"
      metricMethodRef: hits
      increment: 1    
EOF
# application
oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: Application
metadata:
  name: application-cr
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  accountCR: 
    name: developeraccount01
  applicationPlanName: plan01
  productCR: 
    name: product1-cr
  name: testApp
  description: further testing
EOF

# TODO proxy promote
oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: ProxyConfigPromote
metadata:
  name: product1-v1-production
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  productCRName: product1-cr
  production: true
  deleteCR: true
EOF

sleep 30
echo Product Route: 
echo "https://$(oc get routes | grep product1 |grep production| awk '{print $2}')" 
echo
echo User_key: 
echo $(curl -s -X 'GET' "https://3scale-admin.$DOMAIN/admin/api/applications.xml?access_token=$ADMIN_ACCESS_TOKEN&page=1&per_page=500&service_id=3" -H 'accept: */*' | grep -oP '<user_key>\K[^<]+' | sed 's/\s//g')

• Wait for the script to finish and login into the admin portal and confirm that a product is setup with an application called testApp
• Setup secret

cat << EOF | oc create -f -
kind: Secret
apiVersion: v1
metadata:
  name: auth-secret
  namespace: 3scale-test
data:
  ApplicationID: 'Zm9vCg=='
  ApplicationKey: 'Zm9vCg=='
  UserKey: >-
    dGVzdGtleQo=
type: Opaque
EOF

• Setup Application with user_key

oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: Application
metadata:
  name: application-cr2
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  accountCR: 
    name: developeraccount01
  applicationPlanName: plan01
  authSecretRef:
    name: auth-secret
  productCR: 
    name: product1-cr
  name: testApp2
  description: further testing
EOF

• Check in Admin portal, you should see a new testApp2 application under product1 with UserKey set to testkey
• Setup Application with app_id/app_key

oc apply -f - <<EOF
---
apiVersion: capabilities.3scale.net/v1beta1
kind: Application
metadata:
  name: application-cr3
  namespace: 3scale-test
  annotations:
    "insecure_skip_verify": "true"
spec:
  accountCR: 
    name: developeraccount01
  applicationPlanName: plan01
  authSecretRef:
    name: auth-secret
  productCR: 
    name: product2-cr
  name: testApp3
  description: further testing
EOF

• Check in Admin portal, you should see a new testApp3 application with product2 with Application ID/Application Key set to something

[briangallagher]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scanner	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[tkan145]

@dcheng1248 would you able to give this PR a quick look and let me know if it meets your requirements?

[dcheng1248]

I think the error message should say appid/appkey as authentication mode (not user-key), and the lacking key should be appid/appkey instead of userkey. Same issue for OIDC validation.

[tkan145]

Fixed. Thanks for the review, much appreciated!

[dcheng1248]

@tkan145 I cannot test the operator directly on our cluster, but based on the code it would indeed solve our issue, thanks for the work!
I left a comment on where I think might be a typo.

[eguzki]

LGTM

Just for cuiosity, if I change the secret, the 3scale application gets updated as well?

[tkan145]

The application ID is a one-time assignment. Once generated, it cannot be modified; you must delete and then recreate the application to change it.

If you need to change the key, then we have ApplicationAuth CR for that.

If you change the secret, the operator will still read the secret but won't do anything unless:

• it's a new application. Code
• Product reference has change. Code

[eguzki]

The application ID is a one-time assignment. Once generated, it cannot be modified; you must delete and then recreate the application to change it.

If you need to change the key, then we have ApplicationAuth CR for that.

If you change the secret, the operator will still read the secret but won't do anything unless:

* it's a new application. [Code](https://github.com/3scale/3scale-operator/blob/2f7b415c8a5c100405c5b1151fc1c70939bb21cc/controllers/capabilities/application_threescale_reconciler.go#L113)

* Product reference has change. [Code](https://github.com/3scale/3scale-operator/blob/2f7b415c8a5c100405c5b1151fc1c70939bb21cc/controllers/capabilities/application_threescale_reconciler.go#L131)

Makes sense, thanks for the explanation