https://www.skills.google/games/7172/labs/44418
✅ test result of nov05_gsp514.sh Commit bf14ba6
Welcome to Cloud Shell! Type "help" to get started, or type "gemini" to try prompting with Gemini CLI.
Your Cloud Platform project in this session is set to qwiklabs-gcp-02-5a604ab89977.
Use `gcloud config set project [PROJECT_ID]` to change to a different project.
student_01_9c7c7dab05ca@cloudshell:~ (qwiklabs-gcp-02-5a604ab89977)$ rm -f nov05_gsp514.sh
curl -LO https://raw.githubusercontent.com/nov05/gcp-skills-boost/refs/heads/dev/GSP514%20Build%20a%20Data%20Mesh%20with%20Knowledge%20Catalog%3A%20Challenge%20Lab/nov05_gsp514.sh
chmod +x nov05_gsp514.sh
./nov05_gsp514.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8715 100 8715 0 0 43397 0 --:--:-- --:--:-- --:--:-- 43358
👉 Enter username 2: student-03-58c03c8e7fdd@qwiklabs.net
Your active configuration is: [cloudshell-4345]
Updated property [compute/region].
🔹 Project ID: qwiklabs-gcp-02-5a604ab89977
🔹 Project number: 1081421419800
🔹 Region: us-west1
🔹 Zone: us-west1-b
🔹 User: student_01_9c7c7dab05ca
🔹 Username 2: student-03-58c03c8e7fdd@qwiklabs.net
🔹 Bukect: qwiklabs-gcp-02-5a604ab89977-customer-online-sessions
========================================================
Task 1. Create a Knowledge Catalog lake with two zones and two assets
========================================================
Operation "operations/acat.p2-1081421419800-5fda20c5-7689-4aa9-a253-e2a33e7497df" finished successfully.
Waiting for [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/operations/operation-1778669079973-651b0aaaf4f27-1f47b88f-a039190e] to finish...done.
Created [sales-lake] Lake created in [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1].
Waiting for [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/operations/operation-1778669446308-651b0c085222b-73ff754b-ddadec5b] to finish...done.
Created [raw-customer-zone] Zone created in [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/lakes/sales-lake].
Waiting for [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/operations/operation-1778669455328-651b0c10ec3cf-8b439405-75489aaa] to finish...done.
Created [curated-customer-zone] Zone created in [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/lakes/sales-lake].
Waiting for [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/operations/operation-1778669464299-651b0c197a775-e0d8d2d4-bcff2130] to finish...done.
Created [customer-engagements] Asset created in [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/lakes/sales-lake/zones/raw-customer-zone].
Waiting for [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/operations/operation-1778669506366-651b0c4198a56-72d17563-e68ac7cb] to finish...done.
Created [customer-orders] Asset created in [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/lakes/sales-lake/zones/curated-customer-zone].
========================================================
Task 2. Create an aspect type and add an aspect to a zone
========================================================
https://docs.cloud.google.com/dataplex/docs/enrich-entries-metadata#gcloud
Waiting for [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/operations/operation-1778669548236-651b0c6986d79-1867247f-bc617db0] to finish...done.
Created [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/aspectTypes/protected-customer-data-aspect] Aspect Type created in project [qwiklabs-gcp-02-5a604ab89977] with location [us-west1].
authorization: {}
createTime: '2026-05-13T10:52:28.301147171Z'
displayName: Protected Customer Data Aspect
etag: YKqOVK5dHCwrKi41kHy6HOnwmzwIODSTDGOkzSIE0uc
metadataTemplate:
name: protected_customer_data_template
recordFields:
- annotations:
displayName: Raw Data Flag
enumValues:
- index: 1
name: Yes
- index: 2
name: No
index: 1
name: raw_data_flag
type: enum
- annotations:
displayName: Protected Contact Information Flag
enumValues:
- index: 1
name: Yes
- index: 2
name: No
index: 2
name: protected_contact_information_flag
type: enum
type: record
name: projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/aspectTypes/protected-customer-data-aspect
uid: 8bdb41d2-12d4-4b17-8e53-387363867d62
updateTime: '2026-05-13T10:52:29.734197799Z'
👉 Check entry list:
ENTRY_ID: @dataplex_entry
SOURCE_LOCATION: us-west1
DISPLAY_NAME:
DESCRIPTION:
RESOURCE: projects/1081421419800/locations/us-west1/entryGroups/@dataplex
SYSTEM: Dataplex
PLATFORM: GCP
LABELS:
ANCESTORS:
ENTRY_TYPE: projects/655216118709/locations/global/entryTypes/entrygroup
ENTRY_CREATE_TIME: 2026-05-13T10:52:29.896286Z
ENTRY_UPDATE_TIME: 2026-05-13T10:52:32.294375Z
PARENT_ENTRY:
FULLY_QUALIFIED_NAME:
SOURCE_CREATE_TIME: 2026-05-13T10:52:31.032715Z
SOURCE_UPDATE_TIME: 2026-05-13T10:52:31.032715Z
ENTRY_ID: protected-customer-data-aspect_aspectType
SOURCE_LOCATION: us-west1
DISPLAY_NAME: Protected Customer Data Aspect
DESCRIPTION:
RESOURCE: projects/1081421419800/locations/us-west1/aspectTypes/protected-customer-data-aspect
SYSTEM: Dataplex
PLATFORM: GCP
LABELS:
ANCESTORS:
ENTRY_TYPE: projects/655216118709/locations/global/entryTypes/aspecttype
ENTRY_CREATE_TIME: 2026-05-13T10:52:29.955656Z
ENTRY_UPDATE_TIME: 2026-05-13T10:52:29.955656Z
PARENT_ENTRY:
FULLY_QUALIFIED_NAME:
SOURCE_CREATE_TIME: 2026-05-13T10:52:29.706042Z
SOURCE_UPDATE_TIME: 2026-05-13T10:52:29.706042Z
👉 Check entry protected-customer-data-aspect_aspectType:
aspects:
655216118709.global.aspecttype-aspect:
aspectSource: {}
aspectType: projects/655216118709/locations/global/aspectTypes/aspecttype-aspect
createTime: '2026-05-13T10:52:29.955656Z'
data: {}
updateTime: '2026-05-13T10:52:29.955656Z'
createTime: '2026-05-13T10:52:29.955656Z'
entrySource:
createTime: '2026-05-13T10:52:29.706042Z'
displayName: Protected Customer Data Aspect
location: us-west1
platform: GCP
resource: projects/1081421419800/locations/us-west1/aspectTypes/protected-customer-data-aspect
system: Dataplex
updateTime: '2026-05-13T10:52:29.706042Z'
entryType: projects/655216118709/locations/global/entryTypes/aspecttype
name: projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/entryGroups/@dataplex/entries/protected-customer-data-aspect_aspectType
updateTime: '2026-05-13T10:52:29.955656Z'
👉 Check aspect-patch.json:
{
"qwiklabs-gcp-02-5a604ab89977.us-west1.protected-customer-data-aspect": {
"data": {
"raw_data_flag": "Yes",
"protected_contact_information_flag": "Yes",
},
},
}
Updated entry [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/qwiklabs-gcp-02-5a604ab89977/datasets/raw_customer_zone].
========================================================
Task 3. Assign a Knowledge Catalog IAM role to another user
========================================================
bindings:
- members:
- user:student-03-58c03c8e7fdd@qwiklabs.net
role: roles/dataplex.dataReader
etag: BwZRsMd9qsw=
version: 1
bindings:
- members:
- user:student-03-58c03c8e7fdd@qwiklabs.net
role: roles/dataplex.dataReader
- members:
- user:student-03-58c03c8e7fdd@qwiklabs.net
role: roles/dataplex.dataWriter
etag: BwZRsMed3dY=
version: 1
========================================================
Task 4. Create and upload a data quality specification file to Cloud Storage
========================================================
Copying file://dq-customer-orders.yaml to gs://qwiklabs-gcp-02-5a604ab89977-dq-config/dq-customer-orders.yaml
Completed files 1/1 | 309.0B/309.0B
========================================================
Task 5. Define and run an auto data quality job in Knowledge Catalog
========================================================
Waiting for [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/operations/operation-1778669570342-651b0c7e9bd23-763db296-15d9f40e] to finish...done.
Created [projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/dataScans/customer-orders-data-quality-job] Data quality scan created in project [qwiklabs-gcp-02-5a604ab89977] with location [us-west1].
createTime: '2026-05-13T10:52:51.490111408Z'
data:
resource: //bigquery.googleapis.com/projects/qwiklabs-gcp-02-5a604ab89977/datasets/customer_orders/tables/ordered_items
executionSpec:
trigger:
onDemand: {}
executionStatus: {}
name: projects/qwiklabs-gcp-02-5a604ab89977/locations/us-west1/dataScans/customer-orders-data-quality-job
state: ACTIVE
type: DATA_QUALITY
uid: a9f03590-6c73-4111-9977-5353c69df788
updateTime: '2026-05-13T10:54:05.406931496Z'
job:
createTime: '1970-01-01T00:00:00Z'
dataQualitySpec:
postScanActions:
bigqueryExport:
resultsTable: projects/qwiklabs-gcp-02-5a604ab89977/datasets/orders_dq_dataset/tables/results
rules:
- column: user_id
dimension: COMPLETENESS
nonNullExpectation: {}
threshold: 1.0
- column: order_id
dimension: COMPLETENESS
nonNullExpectation: {}
threshold: 1.0
name: projects/1081421419800/locations/us-west1/dataScans/customer-orders-data-quality-job/jobs/f9ca0607-7196-4bb2-a70f-856cde23273a
state: PENDING
type: DATA_QUALITY
uid: f9ca0607-7196-4bb2-a70f-856cde23273a
👉 Running customer-orders-data-quality-job...
Job status: PENDING
Job status: PENDING
Job status: PENDING
Job status: RUNNING
Job status: RUNNING
Job status: RUNNING
Job status: RUNNING
Job status: RUNNING
Job status: RUNNING
Job status: RUNNING
Job status: SUCCEEDED
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------+---------------------+---------------------+-----------------------------------------------------------------------------------+----------------------------------------------+------------------+-----------+------------------+------------+----------------------+-------------+----------------+------------------------+-----------------+-------------+---------------------+------------------+--------------------------+----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+---------------------+--------------------------+---------------+-----------------+-------------+----------------+
| data_quality_scan | data_source | data_quality_job_id | data_quality_job_configuration | job_labels | job_start_time | job_end_time | job_quality_result | job_dimension_result | job_rows_scanned | rule_name | rule_description | rule_type | rule_evaluation_type | rule_column | rule_dimension | rule_threshold_percent | rule_parameters | rule_passed | rule_rows_evaluated | rule_rows_passed | rule_rows_passed_percent | rule_rows_null | rule_failed_records_query | created_on | last_updated | rule_assertion_row_count | debug_queries | rule_attributes | rule_source | rule_suspended |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------+---------------------+---------------------+-----------------------------------------------------------------------------------+----------------------------------------------+------------------+-----------+------------------+------------+----------------------+-------------+----------------+------------------------+-----------------+-------------+---------------------+------------------+--------------------------+----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+---------------------+--------------------------+---------------+-----------------+-------------+----------------+
| {"resource_name":"//dataplex.googleapis.com/projects/1081421419800/locations/us-west1/dataScans/customer-orders-data-quality-job","project_id":"qwiklabs-gcp-02-5a604ab89977","location":"us-west1","data_scan_id":"customer-orders-data-quality-job","display_name":null} | {"resource_name":"//bigquery.googleapis.com/projects/qwiklabs-gcp-02-5a604ab89977/datasets/customer_orders/tables/ordered_items","dataplex_entity_project_id":null,"dataplex_entity_project_number":null,"dataplex_lake_id":null,"dataplex_zone_id":null,"dataplex_entity_id":null,"table_project_id":"qwiklabs-gcp-02-5a604ab89977","table_project_number":"1081421419800","dataset_id":"customer_orders","table_id":"ordered_items","catalog_id":null,"namespace_id":null} | f9ca0607-7196-4bb2-a70f-856cde23273a | {"enable_catalog_based_rules":false,"increment_column":"","incremental":false,"row_filter":"","sampling_percent":100,"trigger":"on_demand"} | NULL | 2026-05-13 10:54:53 | 2026-05-13 10:55:25 | {"passed":"true","score":"100.0","incremental_start":null,"incremental_end":null} | {"COMPLETENESS":{"passed":true,"score":100}} | 1000 | | | NULL check | Per row | user_id | COMPLETENESS | 100.0 | NULL | true | 1000 | 1000 | 100.0 | 0 | WITH `f9ca0607-7196-4bb2-a70f-856cde23273a` AS (SELECT * FROM `qwiklabs-gcp-02-5a604ab89977.customer_orders.ordered_items` ) SELECT * FROM `f9ca0607-7196-4bb2-a70f-856cde23273a` WHERE (`user_id` IS NULL); | 2026-05-13 10:54:03 | 2026-05-13 10:54:03 | NULL | [] | NULL | NULL | false |
| {"resource_name":"//dataplex.googleapis.com/projects/1081421419800/locations/us-west1/dataScans/customer-orders-data-quality-job","project_id":"qwiklabs-gcp-02-5a604ab89977","location":"us-west1","data_scan_id":"customer-orders-data-quality-job","display_name":null} | {"resource_name":"//bigquery.googleapis.com/projects/qwiklabs-gcp-02-5a604ab89977/datasets/customer_orders/tables/ordered_items","dataplex_entity_project_id":null,"dataplex_entity_project_number":null,"dataplex_lake_id":null,"dataplex_zone_id":null,"dataplex_entity_id":null,"table_project_id":"qwiklabs-gcp-02-5a604ab89977","table_project_number":"1081421419800","dataset_id":"customer_orders","table_id":"ordered_items","catalog_id":null,"namespace_id":null} | f9ca0607-7196-4bb2-a70f-856cde23273a | {"enable_catalog_based_rules":false,"increment_column":"","incremental":false,"row_filter":"","sampling_percent":100,"trigger":"on_demand"} | NULL | 2026-05-13 10:54:53 | 2026-05-13 10:55:25 | {"passed":"true","score":"100.0","incremental_start":null,"incremental_end":null} | {"COMPLETENESS":{"passed":true,"score":100}} | 1000 | | | NULL check | Per row | order_id | COMPLETENESS | 100.0 | NULL | true | 1000 | 1000 | 100.0 | 0 | WITH `f9ca0607-7196-4bb2-a70f-856cde23273a` AS (SELECT * FROM `qwiklabs-gcp-02-5a604ab89977.customer_orders.ordered_items` ) SELECT * FROM `f9ca0607-7196-4bb2-a70f-856cde23273a` WHERE (`order_id` IS NULL); | 2026-05-13 10:54:03 | 2026-05-13 10:54:03 | NULL | [] | NULL | NULL | false |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------+---------------------+---------------------+-----------------------------------------------------------------------------------+----------------------------------------------+------------------+-----------+------------------+------------+----------------------+-------------+----------------+------------------------+-----------------+-------------+---------------------+------------------+--------------------------+----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+---------------------+--------------------------+---------------+-----------------+-------------+----------------+
✅ All done
student_01_9c7c7dab05ca@cloudshell:~ (qwiklabs-gcp-02-5a604ab89977)$