Level | Statements | Strongly Agree | Agree | Neutral | Disagree | Strongly Disagree |
---|---|---|---|---|---|---|
AWARENESS | -I can easily recall basic technical terminology and recognize the main tools required in my field. -I am familiar with the standard procedures used in data processing and analysis. |
[ ] | [ ] | [ ] | [ ] | [ ] |
ACQUISITION | -I understand core technical concepts and can explain how standard methods work.-I can interp |
| Level | Technical/Functional Competencies | Managerial Competencies | Human Competencies | Conceptual Competencies | |-------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------
Domain | Sub-domain | Questions | |
---|---|---|---|
Operational excellence | Infrastructure as code | How do you automate the deployment of databases, data pipelines, and ETL processes? | |
Monitoring and observability | What practices do you use for logging, monitoring, and alerting for data operations | ||
How do you monitor data quality metrics (freshness, completeness, anomalies, etc.) | |||
Incident & change management | How do you track and manage data schema change, pipeline modifications | ||
How do you document data pipeline incidents and troubleshooting | |||
Automation & orchestration | How do you automate ingestion, cleansing, and transformation processes | ||
Do you use any orchestration tools? | |||
Security | Encryption & data protection | Do you use data encryption at rest (storage) and in transit (ETL, ingestion) | |
What methods do you use to secure sensitive data fields? |
' |
' |
' |
tool,tool category,link,integration,dbt Cloud/dbt Core (use with caution) | |
Sisu,AI/ML,https://sisudata.com/,...you can define your metrics in dbt and then use them in Sisu for one-click analyses.,dbt Cloud | |
Continual,AI/ML,https://continual.ai/,"Continual integrates with dbt by allowing dbt users to define entities, feature sets, and predictive models directly from their existing dbt models.",dbt Cloud | |
Holistics ,BI,https://www.holistics.io/,"Holistics fully integrates with your dbt project, allows you to perform data modeling and transformation at dbt layer, and push those definitions to Holistics BI layer",dbt Cloud/dbt Core | |
mode,BI,https://mode.com/,Mode customers can now get better views on data freshness with our dbt integration.,dbt Cloud | |
thoughtspot,BI,https://www.thoughtspot.com/,"ThoughtSpot’s dbt integration allows you to easily provide your existing dbt models and automatically create ThoughtSpot Worksheets, which you can use to search your data.",dbt Cloud | |
Transform,BI,https://transform.co/, |
-! 🚨 WARNING 🚨 !-
You probably do not want to do this because dbt Cloud will not be able to drop the relevant schema
upon PR merge / close so you will end up with clutter if you are not on top of this.
The following is the default behaviour of [dbt Cloud CI runs][1] when:
AWS Glue Studio | AWS Glue DataBrew | ||
---|---|---|---|
Source | -S3 -AWS Glue Data catalog (S3, RDS, Redshift, etc.) -Streaming (AWS Kinesis Data Streams, Kafka) | -Manual upload -Direct connection using JDBC -AWS Glue Data catalog (S3, Redshift, RDS) -Amazon Appflow -AWS Data Exchange -Snowflake | |
Algorithm | No information but as per https://www.acf.hhs.gov/sites/default/files/documents/opre/opre-understanding_effect_opioid_epidemic_child_maltreatment-jan2022.pdf, k-mean clustering is used | No information |
' |