diff --git a/doc/source/admintasks/kubernetes/figures/interrogation-services.png b/doc/source/admintasks/kubernetes/figures/interrogation-services.png new file mode 100644 index 000000000..2d85c01f0 Binary files /dev/null and b/doc/source/admintasks/kubernetes/figures/interrogation-services.png differ diff --git a/doc/source/admintasks/kubernetes/figures/library-harbor.png b/doc/source/admintasks/kubernetes/figures/library-harbor.png new file mode 100644 index 000000000..5b2a82fda Binary files /dev/null and b/doc/source/admintasks/kubernetes/figures/library-harbor.png differ diff --git a/doc/source/admintasks/kubernetes/figures/new-replication-rule.png b/doc/source/admintasks/kubernetes/figures/new-replication-rule.png new file mode 100644 index 000000000..0458e5fc5 Binary files /dev/null and b/doc/source/admintasks/kubernetes/figures/new-replication-rule.png differ diff --git a/doc/source/admintasks/kubernetes/harbor-as-system-app-1d1e3ec59823.rst b/doc/source/admintasks/kubernetes/harbor-as-system-app-1d1e3ec59823.rst new file mode 100644 index 000000000..77222d8dc --- /dev/null +++ b/doc/source/admintasks/kubernetes/harbor-as-system-app-1d1e3ec59823.rst @@ -0,0 +1,590 @@ +.. _harbor-as-system-app-1d1e3ec59823: + +============================ +Harbor as System Application +============================ + +.. rubric:: |context| + +Harbor is an open-source registry that secures artifacts with policies and +role-based access control, ensures images are scanned and free from +vulnerabilities, and signs images as trusted. Harbor has been evolved to a +complete |OCI| compliant cloud-native artifact registry. + +With Harbor V2.0, users can manage images, manifest lists, Helm charts, +|CNABs|, |OPAs| among others which all adhere to the |OCI| image specification. +It also allows for pulling, pushing, deleting, tagging, replicating, and +scanning such kinds of artifacts. Signing images and manifest list are also +possible now. + +Harbor supports replication of images between registries, and offers advanced +security features such as user management, access control and activity +auditing. + +See https://goharbor.io/docs/2.0.0/ for more details on Harbor. + +------------------- +Harbor Installation +------------------- + +.. rubric:: |prereq| + +- CephFS backed |PVCs| are recommended for Harbor such that when configuring + multiple replicas, for |AIO-DX| or Standard configurations, both Harbor + replicas can read and write to the registry. + +- Create a secret as described below: + + - Generate certificates and create secret. + + A |CA| cert and server cert creation procedure using cert-manager is + specified below: + + Create the certificate for Harbor using Cert-Manager and using the + local |CA|, system-local-ca, as the issuer. Note that the certificate + should be created in the ``harbor-tls SECRET`` in the Harbor + ``NAMESPACE``. + + For example: + + #. Create the following Harbor certificate yaml configuration file: + + .. code-block:: none + + ~(keystone_admin)]$ cat < harbor-certificate.yaml + --- + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: harbor-certificate + namespace: harbor + spec: + secretName: harbor-tls + issuerRef: + name: system-local-ca + kind: ClusterIssuer + duration: 2160h # 90 days + renewBefore: 360h # 15 days + commonName: < oam floating IP Address or FQDN > + subject: + organizations: + - ABC-Company + organizationalUnits: + - StarlingX-harbor + ipAddresses: + - < oam floating IP address > + dnsNames: + - < harbor dns> # e.g. harbor.yourdomian.com + - < notary dns > # optional, required only if exposed on ingress e.g. notary.yourdomian.com + EOF + + #. Apply the configuration: + + .. code-block:: none + + ~(keystone_admin)]$ kubectl apply -f harbor-certificate.yaml + + #. Verify the configuration: + + .. code-block:: none + + ~(keystone_admin)]$ kubectl get certificate harbor-certificate -n harbor + + After successful configuration, the certificate's Ready status + will be True. + + - nodePort + + #. Create Harbor using NodePort to expose the service + + .. note:: + + The instructions below assume that the NodePorts 30102, 30103 + and 30104 are available; i.e. not used by any other + applications. + + #. Locate the Harbor system application tarball in + ``/usr/local/share/applications/helm``. + + For example: + + .. code-block:: none + + /usr/local/share/applications/helm/harbor-.tgz + + #. Upload the Harbor application. + + .. code-block:: none + + ~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/harbor-.tgz + + #. Configure the Helm Overrides for Harbor. + + Below values need to be configured for nodePort: + + .. code-block:: none + + + expose: + + type: nodePort # Type should be nodeport + tls: + enabled: true + certSource: secret + secret: # Certificate Source is secret + secretName: "harbor-tls" # A secret containing tls.crt and tls.key + notarySecretName: "harbor-tls" # A secret containing tls.crt and tls.key + + nodePort: + # The name of NodePort service + name: harbor + ports: + http: + # The service port Harbor listens on when serving HTTP + port: 80 + # The node port Harbor listens on when serving HTTP + nodePort: 30002 + https: + # The service port Harbor listens on when serving HTTPS + port: 443 + # The node port Harbor listens on when serving HTTPS + nodePort: 30003 + # Only needed when notary.enabled is set to true + notary: + # The service port Notary listens on + port: 4443 + # The node port Notary listens on + nodePort: 30004 + + + externalURL: https://harbor.yourdomian.com:30003 # URL of harbor listing on 30003 port + + For |AIO-DX| and standard setup, add below ``storageClass`` and + ``accessModes`` override. + + Underlying PVCs pre-requisistes: ``Harbor-Jobservice`` and + ``Harbor-Registry`` microservice. + + For example: + + .. code-block:: none + + persistence: + enabled: true + resourcePolicy: "keep" + persistentVolumeClaim: + registry: + existingClaim: "" + storageClass: "cephfs" + subPath: "" + accessMode: ReadWriteMany + size: 5Gi + annotations: {} + jobservice: + jobLog: + existingClaim: "" + storageClass: "cephfs" + subPath: "" + accessMode: ReadWriteMany + size: 1Gi + annotations: {} + + #. Execute Helm overrides. + + .. code-block:: none + + ~(keystone_admin)]$ system helm-override-update harbor harbor harbor --values values.yaml + + #. Apply/Create the Harbor system application. + + .. code-block:: none + + ~(keystone_admin)]$ system application-apply harbor + + - Ingress + + Create Harbor using Ingress to expose the service. + + .. note:: + + The instructions below assume that the URL + ``harbor.yourdomain.com`` has been configured in the |DNS| server + owning ``yourdomain.com`` as the ``OAM FLOATING IP Address`` of + |prod|. + + #. Locate the Harbor system application tarball in + ``/usr/local/share/applications/helm``. + + For example: + + .. code-block:: none + + /usr/local/share/applications/helm/harbor-.tgz + + #. Upload the Harbor application. + + .. code-block:: none + + ~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/harbor-.tgz + + #. Configure the Helm overrides for Harbor configuration. + + The values below need to be configured for ingress in the + ``values.yaml`` file. + + .. code-block:: none + + expose: + type: ingress. # Type should be ingress + tls: + enabled: true + certSource: secret + secret: # Certificate Source is secret + secretName: "harbor-tls" # Above created secret name + notarySecretName: "harbor-tls" # Above created secret name + ingress: + hosts: + core: harbor.yourdomian.com # Harbor Domain name + notary: notary.yourdomian.com # Notary Domain name + annotations: + kubernetes.io/ingress.class: nginx. # Add ingressclass name. It would be # "nginx" if you are using default ingress # controller. + nginx.org/client-max-body-size: "0". # Add this notation for nginx otherwise nginx # will reject the image pull & push + externalURL: https://harbor.yourdomian.com # URL of harbor + + + For |AIO-DX| and standard setup, add below ``storageClass`` and + ``accessModes`` override for |PVC| used for ``Harbor-Jobservice`` + and ``Harbor-Registry`` microservice. + + For example: + + .. code-block:: none + + persistence: + enabled: true + resourcePolicy: "keep" + persistentVolumeClaim: + registry: + existingClaim: "" + storageClass: "cephfs" + subPath: "" + accessMode: ReadWriteMany + size: 5Gi + annotations: {} + jobservice: + jobLog: + existingClaim: "" + storageClass: "cephfs" + subPath: "" + accessMode: ReadWriteMany + size: 1Gi + annotations: {} + + Update the Helm overrides. + + .. code-block:: none + + ~(keystone_admin)]$ system helm-override-update harbor harbor harbor --values values.yaml + + #. Apply/Create the Harbor system application. + + .. code-block:: none + + ~(keystone_admin)]$ system application-apply harbor + + +------------------------------------------------- +Configure LDAP Authentication for Harbor Registry +------------------------------------------------- + +To configure Harbor to use |prod| Local |LDAP| for authentication, follow the +instructions in `Configure LDAP/Active Directory Authentication +`__ +with the following values: + +For |prod| local |LDAP|: + +.. code-block:: none + + LDP URL: ldap://controller + + LDAP search DN: cn=ldapadmin,dc=cgcs,dc=local + + LDAP Search Password: + + LDAP Base DN: dc=cgcs,dc=local + + LDAP UID: cn + +-------------------------------------- +Push an Image to a in Harbor +-------------------------------------- + +#. Run :command:`sudo su` before Docker login. + +#. Docker Login. + + .. code-block:: none + + docker login -u + + .. note:: + + Replace ```` with actual harborURL and replace + ```` with your actual username. + +#. Tag the image. + + .. code-block:: none + + docker tag redis:latest //redis:latest + +#. Push the image. + + .. code-block:: none + + docker push //redis:latest + +------------------------- +Pull an Image from Harbor +------------------------- + +Use command to pull an image: + +.. code-block:: none + + docker pull // redis:latest + +Where ```` is either: + +- for ``'Ingress' expose: harbor.yourdomian.com`` + +- for ``'NodePort' expose: https:// :30103`` + +---------------------------------- +Push a Helm Chart as an OCI Object +---------------------------------- + +.. code-block:: none + + helm package + + helm push oci://// + +----------------- +Pull a Helm Chart +----------------- + +.. code-block:: none + + helm pull oci:///harbor + + +------------------ +Project Management +------------------ + +A project in Harbor contains all the repositories (images) of a particular +application; e.g. the |prod| project in Harbor would contain all the +images from |prod|. + +For more details on creating and configuring a project see: `Create Projects +`__ and +`Project Configuration +`__. + + +--------------------------------- +Use Images from a Remote Registry +--------------------------------- + +A project in a local Harbor registry can be used as a pull-through cache for a +remote registry. For example, a Harbor registry project on a |prod| Subcloud +can be used as a pull-through cache of a Harbor registry project on its |prod| +system controller. This can be achieved using a proxy project which acts as a +proxy to the remote registry, or using a replication of the remote registry. + +.. rubric:: Add a proxy project + +Proxy cache allows you to use Harbor to proxy and cache images from a target +public or private registry. It does not allow to push the image by the user. +Below are the steps to create a proxy project: + +#. Create a registry endpoint. See `Creating Replication Endpoints + `__. + +#. Create a new project using the above created registry endpoint. See + `Configure Proxy Cache + `__. + +.. rubric:: Add Replication Rule + +Setting up pull based replication replicates the images from the remote +registry based on a trigger. Trigger can be Manual, scheduled, or event based +as shown below. + +.. image:: figures/new-replication-rule.png + +#. Create a replication endpoint. See `Creating Replication Endpoints + `__. + +#. Create a new replication rule with replication mode as ``pull based`` as + describe in `Creating a Replication Rule + `__. + + +------------------------ +Add Users to the Project +------------------------ + +You can add individual users to a project, for more details see `Assign Users +to a Project +`__. + +- |LDAP|/AD users can be added to the project. + +- |LDAP|/AD groups to projects and assign a role to the group. + +- You can see the various roles and user permission associated with the roles + in `User Permissions By Role + `__. + + +------------------------------- +Configure Signed Images Support +------------------------------- + +For more details see: `Implementing Content Trust +`__. + +#. Select cosign or notary for content trust. Harbor will then + only allow verified images to be pulled from the project. + + .. image:: figures/library-harbor.png + +#. Enable content trust by setting the following environment variables on the + machine on which you run the Docker client. + + .. code-block:: none + + export DOCKER_CONTENT_TRUST=1 + + export DOCKER_CONTENT_TRUST_SERVER=https://: + + If you push the image for the first time, you will be asked to enter the + root key passphrase. + +------------------------------ +Configure Trivy Scanner Plugin +------------------------------ + +Trivy is installed and configured as a default scanner. + +.. image:: figures/interrogation-services.png + :width: 800 + +----------------------------- +Configure Size of Registry DB +----------------------------- + +Registry DB size can be configured by setting following in ``values.yaml`` +under: + +.. code-block:: none + + persistence: + registry: + size: 5Gi + jobservice: + jobLog: + size: 1Gi + +Use :command:`system helm-override` command to set the value (Default set to +5Gi). + +------------------------------------------------------ +Enforcement of Image Security Policies Using Portieris +------------------------------------------------------ + +Portieris allows to configure trust policies for an individual namespace or +cluster-wide and checks the image against a signed image list on a specified +notary server to enforce the configured image policies. An Administrator can +enforce |prod| image security policies using the Portieris admission +controller. + +It is required to pull from a registry using a ``docker-registry`` secret. +Enforcing trust for anonymous image pulls is not supported. + +To use portieris, an administrator needs to follow below steps: + +#. Install portieris as specified in :ref:`install-portieris`. + +#. Create a ``docker-registry`` secret. + + .. code-block:: none + + kubectl create secret docker-registry \ + -n harbor harbor-registry-secret \ + --docker-server=:port \ + --docker-username=admin \ + --docker-password=Test@123 + + .. note:: + + If the pod creation with the above secret fails, the user should try + with new secret with ``--docker-server`` as ````. + +#. Configure image policy to allow images from Harbor registry + notary as + specified + :ref:`portieris-clusterimagepolicy-and-imagepolicy-configuration`. Below + example shows the policy allowing image from Harbor registry. + + .. code-block:: none + + apiVersion: portieris.cloud.ibm.com/v1 + kind: ImagePolicy + metadata: + name: allow-custom + + .. code-block:: none + + namespace: harbor + spec: + repositories: + - name: ":30003/*" + policy: + trust: + enabled: true + trustServer: "https://:30004" # Optional, custom trust server for repository + +#. Pull a signed image from Harbor registry in a pod using ``harbor-secret`` + created above. Please note that image policy and pod should be created in + the same namespace. + + .. code-block:: none + + apiVersion: v1 + kind: Pod + metadata: + name: test-pod-public + spec: + containers: + - command: + - sleep + - '3600' + image: :30003/public-demo/redis:latest + imagePullPolicy: Always + name: test-pod + tolerations: + - key: "node-role.kubernetes.io/master" + operator: "Exists" + effect: "NoSchedule" + imagePullSecrets: + - name: harbor-registry-secret + +---------- +Limitation +---------- + +Harbor application cannot be deployed during bootstrap due to the bootstrap +deployment dependencies such as early availability of storage class. diff --git a/doc/source/admintasks/kubernetes/index-admintasks-kub-ebc55fefc368.rst b/doc/source/admintasks/kubernetes/index-admintasks-kub-ebc55fefc368.rst index 184893a06..c561038f9 100644 --- a/doc/source/admintasks/kubernetes/index-admintasks-kub-ebc55fefc368.rst +++ b/doc/source/admintasks/kubernetes/index-admintasks-kub-ebc55fefc368.rst @@ -88,6 +88,15 @@ O-RAN O2 Interface oran-o2-application-b50a0c899e66 +-------------------- +Harbor as System App +-------------------- + +.. toctree:: + :maxdepth: 1 + + harbor-as-system-app-1d1e3ec59823 + -------------------------------------- Technology Preview - Istio Application -------------------------------------- diff --git a/doc/source/shared/abbrevs.txt b/doc/source/shared/abbrevs.txt index 8d465aa67..3e4d7c19e 100755 --- a/doc/source/shared/abbrevs.txt +++ b/doc/source/shared/abbrevs.txt @@ -27,6 +27,8 @@ .. |CDI| replace:: :abbr:`CDI (Containerized Data Importer)` .. |CLI| replace:: :abbr:`CLI (Command Line Interface)` .. |CLIs| replace:: :abbr:`CLIs (Command Line Interfaces)` +.. |CNAB| replace:: :abbr:`CNAB (Cloud Native Application Bundle)` +.. |CNABs| replace:: :abbr:`CNABs (Cloud Native Application Bundles)` .. |CNI| replace:: :abbr:`CNI (Container Networking Interface)` .. |CNIs| replace:: :abbr:`CNIs (Container Networking Interfaces)` .. |CoW| replace:: :abbr:`CoW (Copy on Write)` @@ -121,9 +123,12 @@ .. |OAM| replace:: :abbr:`OAM (Operations, administration and management)` .. |OEM| replace:: :abbr:`OEM (Original Equipment Manufacturer)` .. |OC| replace:: :abbr:`OC (Ordinary Clock)` +.. |OCI| replace:: :abbr:`OCI (Open Container Initiative)` .. |OID| replace:: :abbr:`OID (Object Identifier)` .. |OIDC| replace:: :abbr:`OIDC (OpenID Connect)` .. |ONAP| replace:: :abbr:`ONAP (Open Network Automation Program)` +.. |OPA| replace:: :abbr:`OPA (Open Policy Agent)` +.. |OPAs| replace:: :abbr:`OPAs (Open Policy Agents)` .. |OVS| replace:: :abbr:`OVS (Open Virtual Switch)` .. |OSD| replace:: :abbr:`OSD (Object Storage Daemons)` .. |OSDs| replace:: :abbr:`OSDs (Object Storage Daemons)`