Using ACI or MMS Group-based Policies for Kubernetes Environment

Description

Neither the network nor the application people are familiar with the way network policies are defined in Kubernetes. This project implements the popular group-based network policy model in a CNI environment that supports k8s's network policy API. For automation systems using Group-based Policy (e.g., Multi-cloud-network Middleware Systems, MMS), by defining a unified template that can be defined once and reused multiple times, you can automate and unify multi-cloud policy deployment without having to learn multi-cloud network configuration one by one.

Functions

• The basic logic of Group-based Policy is to connect the cloud workload to the network providing the service (provider network) and the network using the service (consumer network) independent of addresses, infrastructure elements (e.g. VLANs, etc.). Communication can only occur between networks that have directly established a relationship for service provisioning or consuming, and the policies to establish such mutual relationship is called a contract.

• The service providers are generally considered to have a higher security level, and they have no restrictions on access to the consumers, while consumers can only access the protocols and ports opened by the providers. Of course this default rule can be modified so that the security level of either the consumer or the provider is equivalent, and the provider does not have direct access to protocols and ports that the consumer does not authorize. In this case it is only necessary to fine-tune this code so that only the provider has a rule that opens the protocols and port numbers on the local side, so that communication only occurs on the protocols and ports given by the provider (note that k8s' Network Policy is stateful, it allows traffic back to the consumer.)

• For a contract, a network can be both a provider and a consumer, thus establishing mutual communication with other parties while keeping itself secure.

• Both the service provider network and the service consumer network are protected externally, i.e. other pods cannot access them directly

• If networks need to publish services externally, group policies can specify which ports these networks can declare open to be accessed externally and from which addresses they can allow access. This will involve source address identification, and since different service publishing methods, proxy modes and settings of service.spec.externalTrafficPolicy will affect source addresses, it is recommended not to use overly precise source address matching, but to consider these access requirements as North-South traffic and provide further security in conjunction with other methods.

• This tool is equivalent to designing a baseline for initial security. Since the networkpolicies resources generated by the tool are standard, they can be customized manually at any time according to subsequent security needs. But some of the manually customized rules should be used with caution, for example, egress rules will affect DNS services

Limitations

Due to the limitations of the Kubernetes Network Policy feature, the Group Policy capabilities that can be implemented are not exactly equivalent to Group Policy features such as ACI EPG.

• Only 'permit' statement is currently supported (whitelist mechanism)

• The source port number in the contract is ignored

• The 'established' setting in the contract is ignored

Environment

• Python 3+

• Kubernetes Client Library for python

Usage

aci2k8s.py

Push the Group-based Policies to Kubernetes cluster (make sure the kubeconfig is in ~/.kube directory with the proper authorization). Different network policies can be implemented by modifying the JSON file input_data.json, which also can be extracted from MMS or other orchestrators. You can choose whether or not to actually post the policy to the target cluster by setting the DRY_RUN variable at the beginning of the program.

For example, we want to express the following group-based network policy as follows:

This policy is expressed as a JSON file whose contents will look like this:

{
  "namespaces": [
    "mms"
  ],
  "contracts": {
    "hangwe_inst_constract01": {
      "provide_networks": [
        "hangwe_inst_network02"
      ],
      "consume_networks": [
        "hangwe_inst_network01"
      ],
      "ports": [
        {
          "protocol": "TCP",
          "port": "23"
        },
        {
          "protocol": "TCP",
          "port": "3306"
        }
      ]
    },
    "hangwe_inst_constract02": {
      "provide_networks": [
        "hangwe_inst_network01"
      ],
      "consume_networks": [
        "hangwe_inst_network03"
      ],
      "ports": [
        {
          "protocol": "TCP",
          "port": "23"
        }
      ]
    }
  },
  "expose": [
    {
      "network": "hangwe_inst_network01",
      "cidr": "0.0.0.0/0",
      "except": [],
      "ports": [
        {
          "protocol": "TCP",
          "port": "80"
        }
      ]
    }
  ]
}

This application generates the following networkpolicies objects after deployment to a Kubernetes cluster:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: hangwe-inst-network01
spec:
  podSelector:
    matchLabels:
      mms_network_tag: hangwe-inst-network01

  policyTypes:
  - Ingress

  ingress:
  - from:
    - namespaceSelector: {}
      podSelector:
        matchLabels:
          mms_network_tag: hangwe-inst-network01

  - from:
    - namespaceSelector: {}
      podSelector:
        matchExpressions:
          - {key: mms_network_tag, operator: In, values: [hangwe-inst-network02]}

  - from:
    - namespaceSelector: {}
      podSelector:
        matchExpressions:
          - {key: mms_network_tag, operator: In, values: [hangwe-inst-network03]}
    ports:
    - protocol: TCP
      port: 23

  - from:
    - ipBlock:
        cidr: 0.0.0.0/0
    ports:
    - protocol: TCP
      port: 80


apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: hangwe-inst-network02
spec:
  podSelector:
    matchLabels:
      mms_network_tag: hangwe-inst-network02

  policyTypes:
  - Ingress

  ingress:
  - from:
    - namespaceSelector: {}
      podSelector:
        matchLabels:
          mms_network_tag: hangwe-inst-network02

  - from:
    - namespaceSelector: {}
      podSelector:
        matchExpressions:
          - {key: mms_network_tag, operator: In, values: [hangwe-inst-network01]}
    ports:
    - protocol: TCP
      port: 23

    - protocol: TCP
      port: 3306


apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: hangwe-inst-network03
spec:
  podSelector:
    matchLabels:
      mms_network_tag: hangwe-inst-network03

  policyTypes:
  - Ingress

  ingress:
  - from:
    - namespaceSelector: {}
      podSelector:
        matchLabels:
          mms_network_tag: hangwe-inst-network03

  - from:
    - namespaceSelector: {}
      podSelector:
        matchExpressions:
          - {key: mms_network_tag, operator: In, values: [hangwe-inst-network01]}

data_input.py and input_data.json

The main program ai2k8s.py will call the function inside the module data_input.py, which reads the data from the file input_data.json.

create_pods.py

Any Kubernetes resource with the label 'mms_network_tag=' can join the network specified by the name. This script automatically creates a number of test pods that are tagged to join all the networks involved in the network policy, and automatically generates a pod without a tag for comparison in the test. Make sure to use the same DATA data as the previous scripts. You can choose whether or not to actually post the policy to the target cluster by setting the DRY_RUN variable at the beginning of the program.

clear_pods_np.py

Clear the network policies and test pods generated by the first two scripts. Make sure to use the same DATA data as the first two scripts. You can choose whether to remove test pods or network policies by setting the Clear_Pods and Clear_NetworkPolicies variables at the beginning of the program.