Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions modules/authentication-troubleshooting-issues.adoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
:_mod-docs-content-type: CONCEPT
:_mod-docs-content-type: PROCEDURE
[id="authentication-troubleshooting-issues"]
= Troubleshooting {productname} authentication and authorization issues for specific users

Use the following procedure to troubleshoot authentication and authorization issues for specific users.
[role="_abstract"]
To troubleshoot {productname} authentication and authorization issues for specific users, you can exec into the {productname} pod or container and query the `federatedlogin` and `user` tables to verify user records.

.Procedure

Expand All @@ -16,7 +17,6 @@ quay=# select * from federatedlogin;
----
+
.Example output
+
[source,terminal]
----
id | user_id | service_id | service_ident | metadata_json
Expand All @@ -38,7 +38,6 @@ quay=# select username, email from "user";
----
+
.Example output
+
[source,terminal]
----
username | email
Expand Down
3 changes: 2 additions & 1 deletion modules/authentication-troubleshooting.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
[id="authentication-troubleshooting"]
= Troubleshooting {productname} authentication

Authentication and authorization is crucial for secure access to {productname}. Together, they safeguard sensitive container images, verify user identities, enforce access controls, facilitate auditing and accountability, and enable seamless integration with external identity providers. By prioritizing authentication, organizations can bolster the overall security and integrity of their container registry environment.
[role="_abstract"]
Authentication and authorization secure access to {productname} and safeguard container images, verify identities, and enforce access controls.

The following authentication methods are supported by {productname}:

Expand Down
22 changes: 22 additions & 0 deletions modules/check-backend-storage-running.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
:_mod-docs-content-type: CONCEPT
[id="check-backend-storage-running"]
= Checking the status of your backend storage

[role="_abstract"]
To check the status of your {product-title} backend storage and verify access, you can use provider dashboards and CLIs for AWS, GCS, NooBaa, ODF, Ceph, Azure, or OpenStack Swift. Ensure all {product-title} instances have access to all S3 storage backends.

* *Amazon Web Service Storage (AWS)*. Check the AWS S3 service health status on the link:https://health.aws.amazon.com/health/status[AWS Service Health Dashboard]. Validate your access to S3 by listing objects in a known bucket using the `aws` CLI or SDKs.

* *Google Cloud Storage (GCS)*. Check the link:https://status.cloud.google.com/[Google Cloud Status Dashboard] for the status of the GCS service. Verify your access to GCS by listing objects in a known bucket using the Google Cloud SDK or GCS client libraries.

* *NooBaa*. Check the NooBaa management console or administrative interface for any health or status indicators. Ensure that the NooBaa services and related components are running and accessible. Verify access to NooBaa by listing objects in a known bucket using the NooBaa CLI or SDK.

* **{odf}**. Check the {ocp} Console or management interface for the status of the {odf} components. Verify the availability of {odf} S3 interface and services. Ensure that the {odf} services are running and accessible. Validate access to {odf} S3 by listing objects in a known bucket using the appropriate S3-compatible SDK or CLI.

* **Ceph**. Check the status of Ceph services, including Ceph monitors, OSDs, and RGWs. Validate that the Ceph cluster is healthy and operational. Verify access to Ceph object storage by listing objects in a known bucket using the appropriate Ceph object storage API or CLI.

* **Azure Blob Storage**. Check the link:https://azure.status.microsoft/en-us/status[Azure Status Dashboard] to see the health status of the Azure Blob Storage service. Validate your access to Azure Blob Storage by listing containers or objects using the Azure CLI or Azure SDKs.

* **OpenStack Swift**. Check the link:https://www.ibm.com/docs/ro/cmwo/4.3.0.0?topic=services-checking-status[OpenStack Status] page to verify the status of the OpenStack Swift service. Ensure that the Swift services, like the proxy server, container servers, object servers, are running and accessible. Validate your access to Swift by listing containers or objects using the appropriate Swift CLI or SDK.

After checking the status of your backend storage, ensure that all {productname} instances have access to all s3 storage backends.
26 changes: 26 additions & 0 deletions modules/check-data-replication.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
:_mod-docs-content-type: PROCEDURE
[id="check-data-replication"]
= Checking data replication in backend buckets

[role="_abstract"]
To ensure that your {product-title} data is properly replicated in all backend buckets, you can use the `aws CLI` to list objects in the bucket. Run `aws s3 ls` with `--recursive`, `--human-readable`, and `--summarize` to verify replication.

.Prerequisites

* You have installed the `aws` CLI.

.Procedure

. Enter the following command to ensure that your data is replicated in all backend buckets:
+
[source,terminal]
----
$ aws --profile quay_prod_s3 --endpoint=http://10.0.x.x:port s3 ls ocp-quay --recursive --human-readable --summarize
----
+
.Example output
[source,terminal]
----
Total Objects: 17996
Total Size: 514.4 GiB
----
57 changes: 57 additions & 0 deletions modules/check-resource-allocation.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
:_mod-docs-content-type: PROCEDURE
[id="check-resource-allocation"]
= Checking resource allocation

[role="_abstract"]
To check resource allocation for your {productname} deployment and monitor disk, CPU, and memory usage, you can use `oc exec` or `podman exec` for disk usage and `oc adm top pods` or `podman stats` for other resources.

.Procedure

. Obtain a list of running containers.

. Monitor disk usage of your {productname} deployment.

.. If you are using the {productname} Operator on {ocp}, enter the following command:
+
[source,terminal]
----
$ oc exec -it <quay_database_pod_name> -- df -ah
----

.. If you are using a standalone deployment of {productname}, enter the following command:
+
[source,terminal]
----
$ podman exec -it <quay_database_conatiner_name> df -ah
----

. Monitor other resource usage.

.. Enter the following command to check resource allocation on a {productname} Operator deployment:
+
[source,terminal]
----
$ oc adm top pods
----

.. Enter the following command to check the status of a specific pod on a standalone deployment of {productname}:
+
[source,terminal]
----
$ podman pod stats <pod_name>
----

.. Enter the following command to check the status of a specific container on a standalone deployment of {productname}:
+
[source,terminal]
----
$ podman stats <container_name>
----
+
The following information is returned:
+
* *CPU %*. The percentage of CPU usage by the container since the last measurement. This value represents the container's share of the available CPU resources.
* *MEM USAGE / LIMIT*. The current memory usage of the container followed by its memory limit. The values are displayed in the format `current_usage / memory_limit`. For example, `300.4MiB / 7.795GiB` indicates that the container is currently using 300.4 megabytes of memory out of a limit of 7.795 gigabytes.
* *MEM %*. The percentage of memory usage by the container in relation to its memory limit.
* *NET I/O*. The network I/O (input/output) statistics of the container. It displays the amount of data transmitted and received by the container over the network. The values are displayed in the format: `transmitted_bytes / received_bytes`.
* *BLOCK I/O*. The block I/O (input/output) statistics of the container. It represents the amount of data read from and written to the block devices (for example, disks) used by the container. The values are displayed in the format `read_bytes / written_bytes`.
38 changes: 38 additions & 0 deletions modules/clair-dependencies-update.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
:_mod-docs-content-type: PROCEDURE

Check warning on line 1 in modules/clair-dependencies-update.adoc

View workflow job for this annotation

GitHub Actions / vale

Vale: AsciiDocDITA.DocumentTitle

The document title (a level 0 heading) is missing.
[id="clair-dependencies-update"]
== Updating Clair scanner and its dependencies

Check failure on line 3 in modules/clair-dependencies-update.adoc

View workflow job for this annotation

GitHub Actions / vale

Vale: AsciiDocDITA.TaskSection

Sections are not allowed in DITA tasks.

[role="_abstract"]
To ensure your Clair scanner is up to date and avoid issues with newer image formats in {product-title}, you can check your Clair version using oc logs for OpenShift or podman logs for standalone deployments. Use the latest Clair version for support for newer formats and vulnerability database updates.

[NOTE]
====
Checking Clair logs can also be used to check if there are any errors from the updaters microservice in your Clair logs. By default, Clair updates the vulnerability database every 30 minutes.
====

.Procedure

. Check your version of Clair.

.. If you are running Clair on {productname-ocp}, enter the following command:
+
[source,terminal]
----
$ oc logs clair-pod
----

.. If you are running a standalone deployment of {productname} and using a Clair container, enter the following command:
+
[source,terminal]
----
$ podman logs clair-container
----
+
.Example output

Check warning on line 31 in modules/clair-dependencies-update.adoc

View workflow job for this annotation

GitHub Actions / vale

Vale: AsciiDocDITA.BlockTitle

Block titles can only be assigned to examples, figures, and tables in DITA.
+
[source,terminal]
----
"level":"info",
"component":"main",
"version":"v4.5.1",
----
153 changes: 4 additions & 149 deletions modules/clair-troubleshooting-issues.adoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
:_mod-docs-content-type: PROCEDURE
:_mod-docs-content-type: CONCEPT
[id="clair-troubleshooting-issues"]
= Troubleshooting Clair issue

Use the following procedures to troubleshoot Clair.
[role="_abstract"]
To troubleshoot Clair vulnerability scanning in {product-title} and resolve scan issues, you can verify image compatibility, allowlist Clair updaters when using a proxy, check the Clair config.yaml, and inspect image metadata.

[id="verify-image-compatibility"]
== Verifying image compatibility
Expand All @@ -16,101 +17,6 @@ For more information, see link:https://access.redhat.com/documentation/en-us/red

If you are using Clair behind a proxy configuration, you must allowlist the updaters in your proxy or firewall configuration. For more information about updater URLs, see link:https://access.redhat.com/documentation/en-us/red_hat_quay/{producty}/html/vulnerability_reporting_with_clair_on_red_hat_quay/clair-concepts#clair-updater-urls[Clair updater URLs].

[id="clair-dependencies-update"]
== Updating Clair scanner and its dependencies

Ensure that you are using the latest version of Clair security scanner. Outdated versions might lack support for newer image formats, or might have known issues.

Use the following procedure to check your version of Clair.

[NOTE]
====
Checking Clair logs can also be used to check if there are any errors from the updaters microservice in your Clair logs. By default, Clair updates the vulnerability database every 30 minutes.
====

.Procedure

. Check your version of Clair.

.. If you are running Clair on {productname-ocp}, enter the following command:
+
[source,terminal]
----
$ oc logs clair-pod
----

.. If you are running a standalone deployment of {productname} and using a Clair container, enter the following command:
+
[source,terminal]
----
$ podman logs clair-container
----
+
.Example output
+
[source,terminal]
----
"level":"info",
"component":"main",
"version":"v4.5.1",
----

[id="enabling-debug-mode-clair"]
== Enabling debug mode for Clair

By default, debug mode for Clair is disabled. You can enable debug mode for Clair by updating your `clair-config.yaml` file.

.Prerequisites

* For Clair on {productname-ocp} deployments, you must link:https://docs.redhat.com/en/documentation/red_hat_quay/3.12/html-single/vulnerability_reporting_with_clair_on_red_hat_quay/index#custom-clair-configuration-managed-database[Running a custom Clair configuration with a managed Clair database].

Use the following procedure to enable debug mode for Clair.

.Procedure

. Update your `clair-config.yaml` file to include the debug option.

.. On standalone {productname} deployments:

... Add the following configuration field to your `clair-config.yaml` file:
+
[source,yaml]
----
log_level: debug
----

... Restart your Clair deployment by entering the following command:
+
[source,terminal]
----
$ podman restart <clair_container_name>
----

.. On {productname-ocp} deployments:

... On the {ocp} web console, click *Operators* -> *Installed Operators* -> *Quay Registry*.

... Click the name of your registry, for example, *Example Registry*. You are redirected to the *Details* page of your registry.

... Click the Config Bundle Secret, for example, *example-registry-config-bundle-xncls*.

... Confirm that you are running a custom Clair configuration by looking for the `clair-config.yaml` file under the *Data* section of the *Details* page of your secret.

... If you have a `clair-config.yaml` file, click *Actions* -> *Edit Secret*. If you do not, see "Running a custom Clair configuration with a managed Clair database".

... Update your `clair-config.yaml` file to include the `log_level: debug` configuration variable. For example:
+
[source,yaml]
----
log_level: debug
----

... Click *Save*.

... You can check the status of your Clair deployment by clicking *Workloads* -> *Pods*. The `clair-app` pod should report `1/1` under the *Ready* category.

... You can confirm that Clair is returning debugging information by clicking the *clair-app* pod that is ready -> *Logs*.

[id="checking-clair-configuration"]
== Checking Clair configuration

Expand All @@ -119,55 +25,4 @@ Check your Clair `config.yaml` file to ensure that there are no misconfiguration
[id="inspect-image-metadata"]
== Inspect image metadata

In some cases, you might receive an *Unsupported* message. This might indicate that the scanner is unable to extract the necessary metadata from the image. Check if the image metadata is properly formatted and accessible.

[role="_additional-resources"]
.Additional resources

For more information, see link:https://access.redhat.com/articles/7018077[Troubleshooting Clair].


////
[id="check-logs-updaters-errors"]
== Checking logs for updaters errors

Check if there are any errors from the updaters microservice in your Clair logs. By default, Clair updates the vulnerability database every 30 minutes.

Use the following procedure to check your Clair logs.

.Procedure

. Check your Clair logs.

.. If you are running Clair on the {productname} Operator, enter the following command:
+
[source,terminal]
----
$ oc logs clair-pod
----

.. If you are running a standalone deployment of {productname} and using a Clair container, enter the following command:
+
[source,terminal]
----
$ podman logs clair-container
----


[id="updating-cve-database"]
== Updating the CVE database

Updating the CVE database can be a memory and CPU intensive task, especially if there are several CVEs that must be parsed. If the resources are exhausted during this process, the system kernel can stop the offending process. This should be visible in Docker logs, Podman logs, or in the system journal. For example:

[source,terminal]
----
May 14 21:48:14 vm-mtr3-live-k8s-00-ranchernode-4 kernel: [36611.338195] [26556] 0 26556 734467 386889 4165632 0 937 clair

May 14 21:48:14 vm-mtr3-live-k8s-00-ranchernode-4 kernel: [36611.338227] Memory cgroup out of memory: Kill process 26556 (clair) score 1922 or sacrifice child

May 14 21:48:14 vm-mtr3-live-k8s-00-ranchernode-4 kernel: [36611.339573] Killed process 26556 (clair) total-vm:2937868kB, anon-rss:1536364kB, file-rss:11192kB, shmem-rss:0kB

May 14 21:48:14 vm-mtr3-live-k8s-00-ranchernode-4 kernel: [36611.396171] oom_reaper: reaped process 26556 (clair), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
----
////

In some cases, you might receive an *Unsupported* message. This might indicate that the scanner is unable to extract the necessary metadata from the image. Check if the image metadata is properly formatted and accessible.
24 changes: 24 additions & 0 deletions modules/connectivity-networking.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
:_mod-docs-content-type: PROCEDURE
[id="connectivity-networking"]
= Checking the connectivity between {productname} and the database pod

[role="_abstract"]
To check the connectivity between your {productname} instance and the database pod and troubleshoot connection issues, you can use `oc exec` for Operator deployments or `podman exec` for standalone deployments.

.Procedure

. Check the connectivity between {productname} and the database pod.

.. If you are using the {productname} Operator on {ocp}, enter the following command:
+
[source,terminal]
----
$ oc exec -it _quay_pod_name_ -- curl -v telnet://<database_pod_name>:5432
----

.. If you are using a standalone deployment of {productname}, enter the following command:
+
[source,terminal]
----
$ podman exec -it <quay_container_name >curl -v telnet://<database_container_name>:5432
----
Loading
Loading