Skip to main content
Version: 3.16-dev

Install Capsule

This article explains how the KubeRocketCI leverages Capsule capabilities to enable isolation for both the core platform components and the business applications workload.

KubeRocketCI uses Capsule to ensure resource isolation. It is essential to define constraints through the Capsule tenant approach. This approach serves two primary objectives: limiting the resources allocated to the KubeRocketCI components and regulating the resources utilized by each deployed environment.

To ensure isolation for the core platform components, create the krci namespace under the Capsule tenant. Use the template provided in this instruction to create the Capsule tenant specifically for the core components.

Integration​

The following diagram outlines the general steps involved in configuring Capsule for seamless integration with KubeRocketCI. This process ensures efficient resource isolation, allowing for the deployment and management of both KubeRocketCI core platform components and business application workloads. Follow the sequential flow of the diagram to successfully integrate Capsule with the KubeRocketCI platform:

  1. Capsule Installation - This initial step involves setting up Capsule on your environment.

  2. Tenant Configuration - Once Capsule is installed, the next critical step is configuring the Capsule tenant. This involves defining specific parameters and constraints to regulate the allocation of resources to KubeRocketCI components.

  3. Impersonation - Impersonation plays a role in managing user identities and permissions within the Capsule environment. This step ensures secure and controlled access to resources.

  4. Create Namespace - The creation of a dedicated namespace under Capsule is crucial for isolating and managing the core components of the platform. This step establishes the environment where KubeRocketCI will be deployed (krci by default).

  5. Deploy KubeRocketCI - The final step involves deploying the platform within the configured Capsule environment.

Installation​

The installation procedure consists of three steps:

  1. Define the namespace where KubeRocketCI will be installed in Capsule values:

    values.yaml
    manager:
      options:
        # -- Boolean, enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash
        forceTenantPrefix: true
        # enable capsule for KubeRocketCI tenant and cd-pipeline-operator
        # NOTE: Capsule cannot manage cluster system namespaces
        users:
          # Capsule 0.14 default group, kept because this list replaces the chart default
          - kind: Group
            name: projectcapsule.dev
          - kind: Group
            name: capsule.clastix.io
          # enable for cd-pipeline-operator https://github.com/epam/edp-cd-pipeline-operator/blob/release/2.17/deploy-templates/values.yaml#L10
          - kind: Group
            name: system:serviceaccounts:krci  # krci - Namespace where KubeRocketCI will be installed.
          - kind: Group
            name: masters                      # Kubernetes group that manages the KubeRocketCI tenant.
          # uncomment if Argo CD manage main KubeRocketCI tenant
          #- kind: Group
          #  name: system:serviceaccounts:argocd
          # uncomment if edp-oidc-admins group manage main KubeRocketCI tenant
          #- kind: Group
          #  name: edp-oidc-admins

  2. If you run Capsule 0.14 or later, keep its metadata webhook away from objects that are already being deleted:

    values.yaml
    webhooks:
      hooks:
        metadata:
          matchConditions:
            # The first two conditions are the chart defaults. Helm replaces lists, so keep them.
            - name: ignore-subresources
              expression: '!has(request.subResource) || request.subResource == ""'
            - name: ignore-events
              expression: 'request.resource.resource != "events"'
            - name: skip-objects-being-deleted
              expression: 'request.operation != "UPDATE" || !has(object.metadata.deletionTimestamp)'
    warning

    Without the last condition, a Role or RoleBinding created before Capsule 0.14 can get stuck in the Terminating state when it is deleted, together with the Argo CD Application that owns it.

    Capsule 0.14 adds a webhook that labels every object created or updated in a tenant namespace. When such an object is deleted with foreground propagation, the Kubernetes garbage collector finishes the deletion with an update that only removes a finalizer. The webhook adds its labels to that update, so Kubernetes no longer treats it as a finalizer-only change, checks the garbage collector's RBAC permissions, and rejects the request.

    Argo CD uses foreground propagation by default when it prunes resources and when it deletes an Application. This covers KubeRocketCI stage deployments in Capsule tenants, deleting an environment, and removing a Role from an application chart. Deletions made with kubectl delete or Helm use background propagation and are not affected.

  3. To install the Capsule tool, use the Cluster Add-Ons approach. The Capsule add-on already includes the condition from the previous step. For more details, please refer to the Capsule official page.

Configuration​

After installing Capsule, the next crucial step is configuration. Follow the guidelines provided below to configure Capsule and ensure seamless integration with your system:

  1. Specify the Capsule tenant configuration:

    krci-tenant-example.yaml
    apiVersion: capsule.clastix.io/v1beta2
    kind: Tenant
    metadata:
      name: krci-tenant
    spec:
      ...
      owners:
        - clusterRoles:
            - admin
            - capsule-namespace-manager
          kind: Group
          name: system:masters
        #
        # Uncomment if Argo CD manage KubeRocketCI tenant
        #- clusterRoles:
        #    - admin
        #    - capsule-namespace-manager
        #  kind: ServiceAccount
        #  name: system:serviceaccount:argocd:argocd-application-controller
        #
        # Uncomment if edp-oidc-admins group manage KubeRocketCI tenant
        #- clusterRoles:
        #    - admin
        #    - capsule-namespace-manager
        #  kind: Group
        #  name: edp-oidc-admins
      ...

    system:masters Group that manages the KubeRocketCI tenant. Must be declared in the users list of the Capsule values above.

    The following example includes additional specifications that can be used to manage the KubeRocketCI tenant:

    Tenant configuration example
    apiVersion: capsule.clastix.io/v1beta2
    kind: Tenant
    metadata:
      name: krci-tenant
    spec:
      ingressOptions:
        allowWildcardHostnames: false
        allowedHostnames:
          allowedRegex: ^.*example.com$ # DNSwildcard for KubeRocketCI usage
        hostnameCollisionScope: Tenant
      limitRanges:
        items:
          - limits:
                # The default limits apply to each container unless otherwise specified by default
              - default:
                  cpu: 768m
                  memory: 768Mi
                # The default requests apply to each container unless otherwise specified by default
                defaultRequest:
                  cpu: 256m
                  memory: 512Mi
                type: Container
          - limits:
              # In case Tekton pipelines need to use volume workspaces. If use emptydir please set it to 0.
              - max:
                  storage: 3Gi
                min:
                  storage: 3Gi
                type: PersistentVolumeClaim
      # Since EDKubeRocketCIP uses one namespace, the namespace quota is set to 1
      namespaceOptions:
        quota: 1
      networkPolicies:
        items:
          - ingress:
              - from:
                  - namespaceSelector:
                      matchLabels:
                        capsule.clastix.io/tenant: krci-tenant
                  - podSelector: {}
                  - ipBlock:
                      cidr: 172.32.0.0/16
            podSelector: {}
            policyTypes:
              - Ingress
      # Default KubeRocketCI admins group to make admin users tenant owners
      owners:
        - clusterRoles:
            - admin
            - capsule-namespace-manager
          kind: Group
          name: system:masters
        #
        # Uncomment if Argo CD manage main KubeRocketCI tenant
        #- clusterRoles:
        #    - admin
        #    - capsule-namespace-manager
        #  kind: ServiceAccount
        #  name: system:serviceaccount:argocd:argocd-application-controller
        #
        # Uncomment if edp-oidc-admins group manage main KubeRocketCI tenant
        #- clusterRoles:
        #    - admin
        #    - capsule-namespace-manager
        #  kind: Group
        #  name: sandbox-oidc-cluster-admins
      resourceQuotas:
        items:
            # The maximum CPU and Memory capacity for the KubeRocketCI tenant
          - hard:
              limits.cpu: '2'
              limits.memory: 2Gi
            # The maximum number of pods that can be deployed within a namespace
          - hard:
              pods: '15'
        scope: Tenant
      serviceOptions:
        # Enable the capabilities to create ClusterIP service types only
        allowedServices:
          externalName: false
          loadBalancer: false
          nodePort: false

  2. To install the Capsule tenant, use the Cluster Add-Ons approach.

  3. Create namespace for the KubeRocketCI tenant:

    1. Log in as a system:masters member and create a namespace for platform deployment under the Capsule tenant using the following command:

      kubectl create namespace krci

    2. To ensure that the krci namespace is successfully created under the Capsule tenant, you can verify the status of the Capsule tenant using the following command:

      kubectl get tenant krci-tenant -o yaml

      Expected output:

      ...
      status:
        namespaces:
        - krci
        size: 1
        state: Active

  4. Enable Capsule in the values.yaml file to make it possible to manage deployable environments:

    values.yaml
    cd-pipeline-operator:
      tenancyEngine: "capsule"

By following these instructions, Capsule policies are utilized to manage the KubeRocketCI core components and deployable environments. By adhering to these guidelines, you should successfully install KubeRocketCI with Capsule enabled as the tenancyEngine.

Troubleshooting​

Roles or RoleBindings Stuck in Terminating After a Capsule Upgrade​

An Argo CD Application stays in deletion, or a stage cannot be redeployed, while Roles or RoleBindings in a tenant namespace keep a deletion timestamp. List them with the following command:

kubectl get roles,rolebindings -A \
  -o jsonpath='{range .items[?(@.metadata.deletionTimestamp)]}{.metadata.namespace}/{.kind}/{.metadata.name}{"\n"}{end}'

Add the skip-objects-being-deleted condition described in the Installation section and apply the Capsule values. Objects that are already stuck do not need manual action: the garbage collector retries them and completes the deletion within about 17 minutes. To release an object immediately, remove its finalizer as a cluster administrator:

kubectl patch rolebinding <name> -n <namespace> --type=merge -p '{"metadata":{"finalizers":null}}'
Visit Main Site