Last active
July 8, 2023 11:00
Revisions
-
xirkus revised this gist
Jan 20, 2021 . 1 changed file with 1 addition and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -56,6 +56,7 @@ Caveats: - Also requires duplicate config per deployment environment which seems to defeat the purpose of _DRY_. - [The Role-Profiles Pattern Across Infrastructure as Code](https://medium.com/cloud-technology-solutions/the-role-profiles-pattern-across-infrastructure-as-code-3d8910dcd3c9) - interesting perspective with regards to desired structure of infrastructure code, but again relies on Terragrunt. - [Advantages and Pitfalls of your Infra-as-Code Repo Strategy](https://medium.com/swlh/advantages-and-pitfalls-of-your-infra-as-code-repo-strategy-214c1cb45612) - Another article with some interesting perspectives, but again recommends structuring a multi-tenant repo with directories instead of Workspaces. Workspaces maintain the state of a _specific_ deployment while providing access control on who can make the changes. Relegating this to a git repo's access control seems like a step backwards to me. > ***These perspectives of advanced Terraform users suggest that _STRUCTURE_ is a fundamental complexity/issue introduced by Terraform. Can these be addressed using _JUST_ Terraform features and providing a "Convention over Configuration" solution?*** -
xirkus revised this gist
Jan 19, 2021 . 1 changed file with 3 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -507,8 +507,8 @@ Now that you've set up your project repository, Workspaces, environment variable Here are a few suggestions: 1. **There should be 3 branches in your Terraform project's git repo which correspond to each of the different deployments/Workspaces - `dev`, `stage`, and `prod`.** Having different branches isolates changes from propagating automatically to QA/production environments. ***[Optional - You can use the `main` (historically called `master` branch of your git repo as `prod`].*** In any case, pushing Terraform changes to `stage` or `prod` should be limited to automated/audited processes only. This allows changes to be gated on assurance tests before becoming live on production. 1. **Developers making changes to the infrastructure should _ONLY_ use feature branches off of the `dev` branch.** When the Terraform change is ready, it is merged into the `dev` branch. This is the only branch developer can commit changes to directly and should be enforced in the Terraform git repo. 1. **When changes are made to `dev`, they are pulled into `stage` branch for testing.** Ideally, there are automated tests which verify that there are no breaking changes and that the deployed application passes the required assurance tests/processes before being promoted to production. 1. **Finally, when changes to `stage` have passed, the Terraform changes are merged into `prod`.** This step should be _fully_ automated using a CI/CD system of your choice. @@ -517,6 +517,7 @@ Note that: - **For each stage, a set of corresponding `.tfvars` is used.** For example, to perform a developer deployment, the `dev.tfvars` are used as an argument to the Terraform operation (i.e. `terraform plan -var-file=./env/dev.tfvars`). - **It's also recommended that different IAM accounts are associated with each stage.** Following AWS IAM guidelines, there should be a IAM group for developers which give them permissions for creating/deploying development _ONLY_ infrastructure. Each developer should belong to the developer IAM group. For users who are responsible for `stage` environments, another group can be created with the requiste permissions, or alternatively an assumable role. For production deploy, the CI system should be configured with a limited set of credentials to pull the change in the Terraform repository and deploy on production infrastructure. - **Each deployment environment will have it's own resources (meaning that all assets created within the deployment environment will be _UNIQUE_ to it).** This ensures that environments deployed by users will not accidentally use privileges associated with these environments to update the wrong environment. - **State files should be stored remotely using Terraform Cloud.** This ensures that there is an audited trail of state changes which can potentially be used to restore previous state if a rollback is required. - **There's a cost to maintaining different deployment environments due to the duplication of resources.** _The benefit is the clear separation of deployment assets and the privileges required to access/deploy them._ # Miscellaneous -
xirkus revised this gist
Jan 19, 2021 . 1 changed file with 8 additions and 8 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -507,17 +507,17 @@ Now that you've set up your project repository, Workspaces, environment variable Here are a few suggestions: 1. **There should be 3 branches in your Terraform project's git repo which correspond to each of the different deployments/Workspaces - `dev`, `stage`, and `prod`.** Having different branches isolates changes from propagating automatically to QA/production environments. [Optional - You can use the `main` (historically called `master` branch of your git repo as `prod`]. In any case, changes should be pushing Terraform changes to `stage` or `prod` should be limited to automated processes only. This allows changes to be gated on assurance tests before becoming live on production. 1. **Developers making changes to the infrastructure should use feature branches off of the `dev` branch.** When the Terraform change is ready, it is merged into the `dev` branch. This is the only branch developer can commit changes to directly and should be enforced in the Terraform git repo. 1. **When changes are made to `dev`, they are pulled into `stage` branch for testing.** Ideally, there are automated tests which verify that there are no breaking changes and that the deployed application passes the required assurance tests/processes before being promoted to production. 1. **Finally, when changes to `stage` have passed, the Terraform changes are merged into `prod`.** This step should be _fully_ automated using a CI/CD system of your choice. Note that: - **For each stage, a set of corresponding `.tfvars` is used.** For example, to perform a developer deployment, the `dev.tfvars` are used as an argument to the Terraform operation (i.e. `terraform plan -var-file=./env/dev.tfvars`). - **It's also recommended that different IAM accounts are associated with each stage.** Following AWS IAM guidelines, there should be a IAM group for developers which give them permissions for creating/deploying development _ONLY_ infrastructure. Each developer should belong to the developer IAM group. For users who are responsible for `stage` environments, another group can be created with the requiste permissions, or alternatively an assumable role. For production deploy, the CI system should be configured with a limited set of credentials to pull the change in the Terraform repository and deploy on production infrastructure. - **Each deployment environment will have it's own resources (meaning that all assets created within the deployment environment will be _UNIQUE_ to it).** This ensures that environments deployed by users will not accidentally use privileges associated with these environments to update the wrong environment. - **There's a cost to maintaining different deployment environments due to the duplication of resources.** _The benefit is the clear separation of deployment assets and the privileges required to access/deploy them._ # Miscellaneous -
xirkus revised this gist
Jan 19, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -18,7 +18,7 @@ Infrastructure-as-Code is a principal that drives modern DevOps practice. I disc - [Use Officially Support Terraform Registry Modules](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#use-officially-supported-terraform-registry-modules) - [Terraform Workspaces](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#terraform-workspaces) - [Use Terraform Cloud to structure Workspaces and save deployment state](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#use-terraform-cloud-to-structure-workspaces-and-save-deployment-state) - [DevOps Workflow](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#devops-workflow) - [Miscellaneous](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#miscellaneous) - [Conclusion](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#conclusion) - [TODO](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#todo) -
xirkus revised this gist
Jan 19, 2021 . 1 changed file with 19 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -18,6 +18,7 @@ Infrastructure-as-Code is a principal that drives modern DevOps practice. I disc - [Use Officially Support Terraform Registry Modules](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#use-officially-supported-terraform-registry-modules) - [Terraform Workspaces](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#terraform-workspaces) - [Use Terraform Cloud to structure Workspaces and save deployment state](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#use-terraform-cloud-to-structure-workspaces-and-save-deployment-state) - [DevOps Workflow]() - [Miscellaneous](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#miscellaneous) - [Conclusion](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#conclusion) - [TODO](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#todo) @@ -500,6 +501,24 @@ To configure Terraform Cloud as a remote backend with multiple Workspaces, you n > ``` > Ideally, a more informative error message would point you at the issue :( # DevOps Workflow Now that you've set up your project repository, Workspaces, environment variables, how do you incorporate this into your DevOps pipeline? The following guidance assumes an AWS provider, but should apply to any supported cloud provider. Here are a few suggestions: 1. There should be 3 branches in your Terraform project's git repo which correspond to each of the different deployments/Workspaces - `dev`, `stage`, and `prod`. Having different branches isolates changes from propagating automatically to QA/production environments. [Optional - You can use the `main` (historically called `master` branch of your git repo as `prod`]. In any case, changes should be pushing Terraform changes to `stage` or `prod` should be limited to automated processes only. This allows changes to be gated on assurance tests before becoming live on production. 1. Developers making changes to the infrastructure should use feature branches off of the `dev` branch. When the Terraform change is ready, it is merged into the `dev` branch. 1. When changes are made to `dev`, they are pulled into `stage` branch for testing. Ideally, there are automated tests which verify that there are no breaking changes and that the deployed application passes the required assurance tests/processes before being promoted to production. 1. Finally, when changes to `stage` have passed, the Terraform changes are merged into `prod`. This step should be _fully_ automated. Note that: - For each stage, a set of corresponding `.tfvars` is used. For example, to perform a developer deployment, the `dev.tfvars` are used as an argument to the Terraform operation (i.e. `terraform plan -var-file=./env/dev.tfvars`). - It's also recommended that different IAM accounts are associated with each stage. Following AWS IAM guidelines, there should be a IAM group for developers which give them permissions for creating/deploying development _ONLY_ infrastructure. Each developer should belong to the developer IAM group. For users who are responsible for `stage` environments, another group can be created with the requiste permissions, or alternatively an assumable role. For production deploy, the CI system should be configured with a limited set of credentials to pull the change in the Terraform repository and deploy on production infrastructure. - Each deployment environment will have it's own resources (meaning that all assets created within the deployment environment will be _UNIQUE_ to it). This ensures that environments deployed by users will not accidentally use privileges associated with these environments to update the wrong environment. - There's a cost to maintaining different deployment environments due to the duplication of resources. The benefit is the clear separation of deployment assets and the privileges required to access/deploy them. # Miscellaneous ## Why is there a [Keybase.io](https://keybase.io) dependency for the AWS Lambda Module? -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -519,4 +519,4 @@ Comments/feedback are welcome! # TODO - [ ] Multi-provider single monolith Terraform repos? Is the ultimate redundancy/resiliency solution to deploy your cloud infrastructure to multiple providers? If so, who is doing this? -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 4 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -411,10 +411,10 @@ Terraform Cloud is a remote backend state storage service. If either no remote ` To configure Terraform Cloud as a remote backend with multiple Workspaces, you need to: 1. **Create a Terraform Cloud account.** 1. **Create a Workspace per deployment environment by specifying a Workspace `prefix` as a convention.** For example, you could provision separate workspaces for dev, stage, and prod using the following Workspace naming convention (where `org-` is the organizational prefix): - `org-dev` - `org-stage` - `org-prod` 1. **Configure the `terraform` block in your `${TERRAFORM_ROOT}/main.tf` with a remote backend**: ``` backend "remote" { -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -399,12 +399,10 @@ An additional challenge is that state files contain sensitive information (such Any solution requires both separation of deployment states as well as the ability to share/update state with access control privileges. Tracing of changes to infrastructure can help remediate breaking changes to specific changes introduced by individual devs. _Enter [Terraform Cloud](https://www.terraform.io/cloud)._ #### Use Terraform Cloud to structure Workspaces and save deployment state Terraform Cloud is a remote backend state storage service. If either no remote `backend` or Workspace is specified, the following will occur: - Terraform will use the `default` Workspace. -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -148,7 +148,6 @@ A few key points regarding the Module: 1. ***Modules do _NOT_ provide inheritance visibility to specifications between parent/child relationships which a few notable exceptions (i.e. `provider`).*** Information that needs to be passed between parent/child/sibling Modules needs to be _specified explicitly_. 1. ***Modules can consist of more than a single `.tf` file.*** 1. ***Modules can be user-defined or reference external/public modules such as those available in the Terraform Registry.*** If you think of the Module as analogous to a programming language method or function, then an obvious usage structure emerges regarding the organization of Terraform assets. @@ -250,6 +249,7 @@ The following guidance is a combination of project organization and usage which 1. **The goal of this structure is to provide a single-source-of-truth per provider feature, adhere to the Single Responsibility principle, and avoid duplication.** The consequence of this is that features which require multiple configuration (i.e. account/permission configuration as well as asset configuration) will have it's configuration spread across multiple modules. This is intentional; where configuration is dependent on other provider functionality, dependencies can be easily specified between the modules (for example ordering of configuration). This is _much_ more difficult to accomplish if the asset creation mixes a bunch of responsibilities together. 1. **Each user-defined Module in this structure is meant to be reusable (as per the DRY principle).** In other words, required parameters should be passed in as Input variables instead of hard-coded within a Module. These variables _MUST_ be passed into individual `module` blocks and defined within the provider-specific Module's `variables.tf` file. Each Module should be able to be invoked multiple times with configuration specific parameters passed in through Input variables. 1. **Leverage built-in Terraform variables and reduce duplicate interpolation through the use of `locals`.** 1. **Document _EACH_ module with a `README.md` to communicate configuration context to future users that _cannot_ be captured in the configuration files themelves (i.e. reasons why a particular work-around was used).** Optionally, add a link to the official documentation. 1. **Use the Terraform Registry `module` provisioning specification.** Each Module should specify `Provisioning Instructions`. For example, here's the [AWS IAM Module](https://registry.terraform.io/modules/terraform-aws-modules/iam/aws/latest). Use indivudal `resource` directives as a _last resort_. 1. **Run `terraform init` to download the Terraform Registry provided Modules**. These are stored in the `$TERRAFORM_ROOT/.terraform` directory. The underlying structure is a reflection of the organization of your project's `module` structure. The importance of the `.terraform` directory _CANNOT be UNDERESTIMATED_. For each Terraform Registry Module, the complete implementation as well as _EXAMPLES_ are provided. **Use the provided examples to understand how to configure the Module.** -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 2 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -35,7 +35,7 @@ To make sense of these guidelines, you should be aware of the following: - Infrastructure as Code implies that programmatic structure should be applied to the organization of the code, not simply that configuration files should be version controlled. - Programmatic structure implies the DRY (Don't Repeat Yourself) principle. - Programmatic structure implies the Single Responsibility principle. - Programmatic structure implies that code will be literate and self-documenting. - Programming idioms are used to structure the understanding of how to best leverage Terraform and it's capabilities. - We adhere to the KISS Principle and avoid adding unnecessary complexity. @@ -148,6 +148,7 @@ A few key points regarding the Module: 1. ***Modules do _NOT_ provide inheritance visibility to specifications between parent/child relationships which a few notable exceptions (i.e. `provider`).*** Information that needs to be passed between parent/child/sibling Modules needs to be _specified explicitly_. 1. ***Modules can consist of more than a single `.tf` file.*** 1. ***Modules can be user-defined or reference external/public modules such as those available in the Terraform Registry.*** 1. ***Document _EACH_ module with a `README.md` to communicate configuration context to future users that _cannot_ be captured in the configuration files themelves (i.e. reasons why a particular work-around was used). Optionally, add a link to the official documentation. If you think of the Module as analogous to a programming language method or function, then an obvious usage structure emerges regarding the organization of Terraform assets. -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 5 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -455,12 +455,14 @@ To configure Terraform Cloud as a remote backend with multiple Workspaces, you n > `shared_credentials_file` parameter is set in your root Module's `provider` block or if you try setting the ENV_VAR in the > Terraform Cloud UI for the Workspace._ > > ***The work-around is to go to each Workspace's > `Settings -> General Settings` and change the Execution Mode to `Local`.*** > > **Terraform Cloud Workspace Settings:** > >  > > **Execution Mode Default Setting:** > >  > > _The Workstate's state file will be stored on > Terraform Cloud, but the `terraform plan` will run locally_. -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -253,6 +253,7 @@ The following guidance is a combination of project organization and usage which 1. **Run `terraform init` to download the Terraform Registry provided Modules**. These are stored in the `$TERRAFORM_ROOT/.terraform` directory. The underlying structure is a reflection of the organization of your project's `module` structure. The importance of the `.terraform` directory _CANNOT be UNDERESTIMATED_. For each Terraform Registry Module, the complete implementation as well as _EXAMPLES_ are provided. **Use the provided examples to understand how to configure the Module.** **Example `.terraform` directory**  > **[Recommendation] The root `main.tf` should have `module` references to user-defined implementation details for specific -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 9 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -251,6 +251,9 @@ The following guidance is a combination of project organization and usage which 1. **Leverage built-in Terraform variables and reduce duplicate interpolation through the use of `locals`.** 1. **Use the Terraform Registry `module` provisioning specification.** Each Module should specify `Provisioning Instructions`. For example, here's the [AWS IAM Module](https://registry.terraform.io/modules/terraform-aws-modules/iam/aws/latest). Use indivudal `resource` directives as a _last resort_. 1. **Run `terraform init` to download the Terraform Registry provided Modules**. These are stored in the `$TERRAFORM_ROOT/.terraform` directory. The underlying structure is a reflection of the organization of your project's `module` structure. The importance of the `.terraform` directory _CANNOT be UNDERESTIMATED_. For each Terraform Registry Module, the complete implementation as well as _EXAMPLES_ are provided. **Use the provided examples to understand how to configure the Module.** **Example `.terraform` directory**  > **[Recommendation] The root `main.tf` should have `module` references to user-defined implementation details for specific > provider features** @@ -453,6 +456,12 @@ To configure Terraform Cloud as a remote backend with multiple Workspaces, you n > > **The work-around is to go to each Workspace's > `Settings -> General Settings` and change the Execution Mode to `Local`.** > > **Terraform Cloud Workspace Settings** >  > > **Execution Mode Default Setting** >  > > _The Workstate's state file will be stored on > Terraform Cloud, but the `terraform plan` will run locally_. > > Here's an example of the output when `Remote` execution is set for the Terraform Cloud Workspace, but fails: -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 5 additions and 5 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -11,15 +11,15 @@ Infrastructure-as-Code is a principal that drives modern DevOps practice. I disc - [Repository Structure](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#repository-structure) - [Basics](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#basics) - [Directives](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#directives) - [Environment Variables](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#environment-variables) - [Two Stage Deployment](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#two-stage-deployment) - [The Module](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#the-module) - [Runtime variables with `.tfvars`](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#runtime-variables-in-tfvars) - [Use Officially Support Terraform Registry Modules](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#use-officially-supported-terraform-registry-modules) - [Terraform Workspaces](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#terraform-workspaces) - [Use Terraform Cloud to structure Workspaces and save deployment state](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#use-terraform-cloud-to-structure-workspaces-and-save-deployment-state) - [Miscellaneous](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#miscellaneous) - [Conclusion](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#conclusion) - [TODO](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#todo) ## Rationale -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -122,7 +122,7 @@ Terraform configuration consists of files (which end with the `.tf` extension) w The above directives are _not_ exhaustive, but the ones mentioned will be the most commonly used to set up some basic configuration. ### Environment Variables **[Assigning values to root module variables](https://www.terraform.io/docs/configuration/variables.html#assigning-values-to-root-module-variables)** -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -122,7 +122,9 @@ Terraform configuration consists of files (which end with the `.tf` extension) w The above directives are _not_ exhaustive, but the ones mentioned will be the most commonly used to set up some basic configuration. ### [Environment Variables] **[Assigning values to root module variables](https://www.terraform.io/docs/configuration/variables.html#assigning-values-to-root-module-variables)** Terraform provides a number of mechanisms to provide input into the executing `terraform` operation. These inputs can be provided through the following methods (listed in order of precedence): -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -7,10 +7,10 @@ Infrastructure-as-Code is a principal that drives modern DevOps practice. I disc # Tables of Contents - [Rationale](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#rationale) - [Background](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#background) - [Terraform Documentation Gotcha's](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#terraform-documentation-gotchas) - [Repository Structure](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#repository-structure) - [Basics](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#basics) - [Directives](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#directives) - Environment Variables - [Two Stage Deployment](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#two-stage-deployment) - [The Module](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#the-module) -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -58,7 +58,9 @@ Caveats: > ***These perspectives of advanced Terraform users suggest that _STRUCTURE_ is a fundamental complexity/issue introduced by Terraform. Can these be addressed using _JUST_ Terraform features and providing a "Convention over Configuration" solution?*** ## Terraform Documentation Gotcha's **[Official Terraform Documentation](https://www.terraform.io/docs/index.html)** There's a wealth of documentation on Terraform provided by Hashicorp, and most users would assume (like myself) that this is good place to start. But here's the problem - the shear _amount of documentation_ does very little to help determine _WHERE_ to get started. -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 9 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -105,7 +105,9 @@ variables.tf ## Basics ### Directives [Official Configuration Documentation](https://www.terraform.io/docs/configuration/index.html) Terraform configuration consists of files (which end with the `.tf` extension) which contain directives. These directives are used to accomplish the following things: @@ -223,7 +225,9 @@ These types of configuration vary in the asset specifics, but not necessarily in > 1. ***Terraform provides the `local.workspace` variable*** to reference the current Workspace within configuration files. If > you use the Workspace feature, there is no need to create a user defined `workspace` variable. #### Use officially supported Terraform Registry Modules **[Terraform Registry](https://registry.terraform.io)** It's tempting to begin using Terraform by slapping together some quick `resource` directives, but this is the path to despair. You'll quickly realize that maintaining configuration this way easily leads to duplication and spaghetti. @@ -388,7 +392,9 @@ Any solution requires both separation of deployment states as well as the abilit _Enter Terraform Cloud._ #### Use Terraform Cloud to structure Workspaces and save deployment state [Terraform Cloud](https://www.terraform.io/cloud) Terraform Cloud is a remote backend state storage service. If either no remote `backend` or Workspace is specified, the following will occur: -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 7 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -483,6 +483,13 @@ To configure Terraform Cloud as a remote backend with multiple Workspaces, you n ## Why is there a [Keybase.io](https://keybase.io) dependency for the AWS Lambda Module? If you look closely at the Terraform Registry AWS Lambda Module, you'll note that configuration requires a Keybase.io user. The purpose of this dependency is to be able to use a public PGP key for the purpose of encrypting credentials for the Lambda service. Unfortunately, there doesn't seem to be an alternative (such as having a private/public PGP key to point the configuration towards or other public keyservers which might already contain a public PGP key). To be able to fulfill this dependency requires the following steps: 1. Create a PGP key pair. 1. Create a Keybase.io account. 1. Upload the public key to your Keybase.io account. This will require verification using the private key generated. 1. Once complete, you should be able to refer to the keybase PGP key using the `keybase:username` directive. This will resolve a public Keybase URL (https://keybase.io/username/pgp_keys.asc) where the public PGP can be downloaded. # Conclusion Hopefully, this give you a roadmap to get started on your Terraform journey. There should be enough to avoid the common Terraform pitfalls and provide you a scalable/extensible architecture for your Terraform project. -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 6 additions and 6 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -392,17 +392,17 @@ _Enter Terraform Cloud._ Terraform Cloud is a remote backend state storage service. If either no remote `backend` or Workspace is specified, the following will occur: - Terraform will use the `default` Workspace. - Terraform will use a local state backend and store `.tfstate` in the `${TERRAFORM_ROOT}` directory on your machine. To configure Terraform Cloud as a remote backend with multiple Workspaces, you need to: 1. **Create a Terraform Cloud account.** 1. **Create a Workspace per deployment environment by specifying a Workspace `prefix` as a convention.** For example, you could provision separate workspaces for dev, stage, and prod using the following Workspace naming convention (where `app-` is the organizational prefix): - org-dev - org-stage - org-prod 1. **Configure the `terraform` block in your `${TERRAFORM_ROOT}/main.tf` with a remote backend**: ``` backend "remote" { hostname = "app.terraform.io" @@ -413,7 +413,7 @@ To configure Terraform Cloud as a remote backend with multiple Workspaces, you n } } ``` 1. **Run `terraform init` to pick up the changes.** You should see something similar in the output: ``` Initializing the backend... -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -477,7 +477,7 @@ To configure Terraform Cloud as a remote backend with multiple Workspaces, you n > % terraform workspace list > workspaces not supported > ``` > Ideally, a more informative error message would point you at the issue :( # Miscellaneous -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 7 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -18,6 +18,8 @@ Infrastructure-as-Code is a principal that drives modern DevOps practice. I disc - [Use Officially Support Terraform Registry Modules](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#use-officially-supported-terraform-registry-modules) - Terraform Workspaces - Terraform Cloud - [Miscellaneous]() - [Conclusion]() - [TODO](https://gist.github.com/xirkus/e57cc20fe7fc95694e302887948e9b12#todo) ## Rationale @@ -481,6 +483,11 @@ To configure Terraform Cloud as a remote backend with multiple Workspaces, you n ## Why is there a [Keybase.io](https://keybase.io) dependency for the AWS Lambda Module? # Conclusion Hopefully, this give you a roadmap to get started on your Terraform journey. There should be enough to avoid the common Terraform pitfalls and provide you a scalable/extensible architecture for your Terraform project. Comments/feedback are welcome! # TODO -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -468,7 +468,7 @@ To configure Terraform Cloud as a remote backend with multiple Workspaces, you n > 17: provider "aws" { > ``` > **[BUG]: Misleading error message on misconfiguration.** > If you forget to specify or use the `workspace.prefix` directive in your configuration, `terraform` operations will fail > opaquely with the following message: > ``` -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -391,7 +391,7 @@ _Enter Terraform Cloud._ Terraform Cloud is a remote backend state storage service. If either no remote `backend` or Workspace is specified, the following will occur: 1. Terraform will use the `default` Workspace. 1. Terraform will use a local state backend and store `.tfstate` in the `${TERRAFORM_ROOT}` directory on your machine. To configure Terraform Cloud as a remote backend with multiple Workspaces, you need to: -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -240,7 +240,7 @@ The following guidance is a combination of project organization and usage which 1. **Each user-defined Module in this structure is meant to be reusable (as per the DRY principle).** In other words, required parameters should be passed in as Input variables instead of hard-coded within a Module. These variables _MUST_ be passed into individual `module` blocks and defined within the provider-specific Module's `variables.tf` file. Each Module should be able to be invoked multiple times with configuration specific parameters passed in through Input variables. 1. **Leverage built-in Terraform variables and reduce duplicate interpolation through the use of `locals`.** 1. **Use the Terraform Registry `module` provisioning specification.** Each Module should specify `Provisioning Instructions`. For example, here's the [AWS IAM Module](https://registry.terraform.io/modules/terraform-aws-modules/iam/aws/latest). Use indivudal `resource` directives as a _last resort_. 1. **Run `terraform init` to download the Terraform Registry provided Modules**. These are stored in the `$TERRAFORM_ROOT/.terraform` directory. The underlying structure is a reflection of the organization of your project's `module` structure. The importance of the `.terraform` directory _CANNOT be UNDERESTIMATED_. For each Terraform Registry Module, the complete implementation as well as _EXAMPLES_ are provided. **Use the provided examples to understand how to configure the Module.** > **[Recommendation] The root `main.tf` should have `module` references to user-defined implementation details for specific > provider features** -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -239,7 +239,7 @@ The following guidance is a combination of project organization and usage which 1. **The goal of this structure is to provide a single-source-of-truth per provider feature, adhere to the Single Responsibility principle, and avoid duplication.** The consequence of this is that features which require multiple configuration (i.e. account/permission configuration as well as asset configuration) will have it's configuration spread across multiple modules. This is intentional; where configuration is dependent on other provider functionality, dependencies can be easily specified between the modules (for example ordering of configuration). This is _much_ more difficult to accomplish if the asset creation mixes a bunch of responsibilities together. 1. **Each user-defined Module in this structure is meant to be reusable (as per the DRY principle).** In other words, required parameters should be passed in as Input variables instead of hard-coded within a Module. These variables _MUST_ be passed into individual `module` blocks and defined within the provider-specific Module's `variables.tf` file. Each Module should be able to be invoked multiple times with configuration specific parameters passed in through Input variables. 1. **Leverage built-in Terraform variables and reduce duplicate interpolation through the use of `locals`.** 1. **Use the Terraform Registry `module` provisioning specification.** Each Module should specify `Provisioning Instructions`. For example, here's the [AWS IAM Module](https://registry.terraform.io/modules/terraform-aws-modules/iam/aws/latest). Use indivudal `resource` directives as a _last resort_. 1. **Run `terraform init` to download the Terraform Registry provided Modules**. These are stored in the `$TERRAFORM_ROOT/.terraform` directory. The underlying structure is a reflection of the organization of your project's `module` structure. The importance of the `.terraform` directory _CANNOT be UNDERESTIMATED_. For each Terraform Registry Module, the complete implementation as well as _EXAMPLES_ are provided. **Use the provided examples to understand the to configure the Module.** > **[Recommendation] The root `main.tf` should have `module` references to user-defined implementation details for specific -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -75,7 +75,7 @@ This list is far from exhaustive, but it gives you a sense of the number of shar ***TL;DR*** - here's the recommended Terraform organizational structure. What follows is a discussion of basic Terraform concepts and the justifictions as to _WHY_ this structure makes the most sense. Throughout, I make a number of **[Recommendation]s** as well as point out some **[ANTI-PATTERN]s**, **[WARNING]s**, and **[BUG]s**. ``` . -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -75,7 +75,7 @@ This list is far from exhaustive, but it gives you a sense of the number of shar ***TL;DR*** - here's the recommended Terraform organizational structure. What follows is a discussion of basic Terraform concepts and the justifictions as to _WHY_ this structure makes the most sense. Throughout, I make a number of **[Recommendation]s** as well as point out some **[ANTI-PATTERN]s**,**[WARNING]s**, and **[BUG]s**. ``` . -
xirkus revised this gist
Jan 18, 2021 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -60,7 +60,7 @@ Caveats: There's a wealth of documentation on Terraform provided by Hashicorp, and most users would assume (like myself) that this is good place to start. But here's the problem - the shear _amount of documentation_ does very little to help determine _WHERE_ to get started. > **Terraform's documentation is like learning to communicate in a foreign language using only a dicitionary as a guide.** While the specifics are well documented, it's difficult to understand the _CONTEXT_ of how to apply that information without significant experimentation. In other words, the tool ergonomics _SUCK_. Here's a list of missing features that would go a long way to improving the Terraform experience
NewerOlder