If you use Rancher you will probably be using RKEv2 on your clusters for your kubernetes distribution.
kube2iam adds AWS IAM integration to your kubernetes.
You can already add IAM permissions via an EC2 Instance Profile (and its attached IAM Role) to you kubernetes roles, but it is open any pod using those permissions.
kube2iam solves this issue by intercepting calls to the ec2 metadata service. If the pod has an annotation to assume an IAM role, then kube2iam will attempt to provide credentials to just that role.
In order to do this the Instance Profile role needs to have permission to assume this other "pod" role.
So you're pod role may look like this... It should have a Trust Policy of :
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::${var.environment.account_id}:role/my_instance_profile_role_name"
]
},
"Action": "sts:AssumeRole"
}
]
}Where "my_instance_profile_role_name" is the name of the role on the Node Instance Profile. This allows the node to assume our "Pod" role.
The "pod" role can have a policy attached like this example :
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "route53:GetChange",
"Resource": "arn:aws:route53:::change/*"
},
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets"
],
"Resource": "arn:aws:route53:::hostedzone/*"
},
{
"Effect": "Allow",
"Action": "route53:ListHostedZonesByName",
"Resource": "*"
}
]
}The above permissions will allow the pod to manipulate a Route53 Zone.
So we now need to specify our Node Instance Profile (and its role and policy). The role will need a Trust Policy of :
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}This will allow the EC2 service to use the role as its instance profile. The role now needs a policy to allow it to assume our "pod" role:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAssumeRoles",
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::${var.environment.account_id}:role/< MY_POD_ROLE_NAME >"
}
]
}Note: we can also use wildcards on the above assume role name, to simplify maintenance of this IAM.
You may want to attach other policies to this role for general (unrestricted) node usage, such as to pull images from our ECR private repositories. Such a policy like arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
You can use their Helm chart.
You need some additional values for use on RKE, giving a chart deploy of :
helm repo add kube2iam https://jtblin.github.io/kube2iam/
helm install kube2iam/kube2iam --name my-release \
--set host.iptables=true \
--set host.interface=cali+ \
--set rbac.create=true \
--set podSecurityPolicy.enabled=true \
--set verbose=trueNote the use of host.iptables=true and host.interface=cali+. This is important for RKEv2. It uses Canal for networking (which means flannel for inter-node and calico for intra-node). The "cali+" value is referred to here.
Also rbac.create=true and podSecurityPolicy.enabled=true are needed otherwise you'll get errors like "Failed to list *v1.Namespace: namespaces is forbidden: User".
There is info on testing here. I alter it slightly, though, to get more information and prevent Pod CrashLoopBackoff spining by using a "sleep loop" as below :
apiVersion: v1
kind: Pod
metadata:
name: aws-cli
labels:
name: aws-cli
annotations:
iam.amazonaws.com/role: role-arn
iam.amazonaws.com/external-id: external-id
spec:
containers:
- image: fstab/aws-cli
command:
# Just spin & wait forever
command: [ "/bin/bash", "-c", "--" ]
args: [ "while true; do sleep 30; done;" ]
name: aws-cliOnce the pod is running you can shell exec into it and run your aws cli command like :
aws route53 list-hosted-zones-by-nameThis should succeed since you had permission for this in your "pod" role.
If it fails check the logs on the kube2iam daemonset. There should be log lines like these :
time="2022-04-12T12:59:11Z" level=debug msg="Pod OnUpdate" pod.iam.role="arn:aws:iam::597767386394:role/prpl-backbone-k8s-kube2iam-cert-manager" pod.name=aws-cli pod.namespace=default pod.status.ip=10.42.0.38 pod.status.phase=Running
time="2022-04-12T13:00:15Z" level=debug msg="Proxy ec2 metadata request" metadata.url=169.254.169.254 req.method=PUT req.path=/latest/api/token req.remote=10.42.0.35
time="2022-04-12T13:00:15Z" level=info msg="PUT /latest/api/token (403) took 2.349861 ms" req.method=PUT req.path=/latest/api/token req.remote=10.42.0.35 res.duration=2.3498609999999998 res.status=403
time="2022-04-12T13:00:15Z" level=info msg="GET /latest/meta-data/iam/security-credentials/ (200) took 0.011261 ms" req.method=GET req.path=/latest/meta-data/iam/security-credentials/ req.remote=10.42.0.35 res.duration=0.011261 res.status=200
time="2022-04-12T13:00:16Z" level=debug msg="retrieved credentials from sts endpoint: sts.amazonaws.com" ns.name=cert-manager pod.iam.role="arn:aws:iam::597767386394:role/prpl-backbone-k8s-kube2iam-cert-manager" req.method=GET req.path="/latest/meta-data/iam/security-credentials/arn:aws:iam::597767386394:role/prpl-backbone-k8s-kube2iam-cert-manager" req.remote=10.42.0.35For fun you can also test the Node role. You should by able to query the ECR repo for a list of images like :
aws ecr list-images --repository-name <MY_REPO_NAME>Hopefully it is now working and you want to use it on something useful...
cert-manager requires IAM permissions to manipulate Route53 DNS (at least for a LetsEncrypt DNS01-verified configuration).
The "pod" role we previously used, above, already contains the necessary permissions.
So to deploy cert-manager we can avoid the less secure, and cumbersome creation of an IAM user (and its specifying of the IAM access and secret keys on the ClusterIssuer). Instead it will use kube2iam and assume the "pod" role.
This helm release would look like :
# Chart from here : https://github.com/cert-manager/cert-manager/tree/master/deploy/charts/cert-manager
helm install cert-manager \
--set version=v1.8.0 \
--set installCRDs=true \
--set podDnsConfig.nameservers[0]="8.8.8.8" \
--set podDnsConfig.nameservers[1]="1.1.1.1" \
--set deploymentAnnotations.iam.amazonaws.com/role=arn:aws:iam::<MY ACCOUNT ID>:role/<MY POD ROLE NAME> \
--set podAnnotations.iam.amazonaws.com/role=arn:aws:iam::<MY ACCOUNT ID>:role/<MY POD ROLE NAME>The deploymentAnnotations and podAnnotations are the important part here for our IAM to work.
Hopefully that helps get started on kube2iam and RKE.
Also consider adding assumed-role limitations by namespace, to increase security. See Namespace Restrictions.