Skip to content

Instantly share code, notes, and snippets.

@jeesmon

jeesmon/istio-debug-cheatsheet.md

Last active March 1, 2022 14:03
Download ZIP

Revisions

  1. jeesmon revised this gist Mar 1, 2022. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions istio-debug-cheatsheet.md
    View file
    Original file line number Diff line number Diff line change
    @@ -8,6 +8,7 @@ https://github.com/istio/istio/issues/15086
    ## Docs

    * Envoy - https://www.envoyproxy.io/docs/envoy/latest/
    * Envoy Access Log: https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage

    ## Troubleshooting

  2. jeesmon revised this gist Mar 1, 2022. 1 changed file with 3 additions and 3 deletions.
    6 changes: 3 additions & 3 deletions istio-debug-cheatsheet.md
    View file
    Original file line number Diff line number Diff line change
    @@ -57,9 +57,9 @@ kubectl -n <ns> logs -f <pod> istio-proxy
    Method 2:

    istioctl dashboard controlz $(kubectl -n istio-system get pods -l app=istiod -o jsonpath='{.items[0].metadata.name}').istio-system --port 15000 --address "0.0.0.0"
    kubectl -n tenant3 exec mock-6865f9b6bf-jlpjc -c istio-proxy -- pilot-agent request POST 'logging?connection=info'
    kubectl -n tenant3 exec mock-6865f9b6bf-jlpjc -c istio-proxy -- pilot-agent request POST 'logging?level=info'
    kubectl -n tenant3 exec mock-6865f9b6bf-jlpjc -c istio-proxy -- pilot-agent request POST 'logging?rbac=debug'
    kubectl -n <ns> exec <pod> -c istio-proxy -- pilot-agent request POST 'logging?connection=info'
    kubectl -n <ns> exec <pod> -c istio-proxy -- pilot-agent request POST 'logging?level=info'
    kubectl -n <ns> exec <pod> -c istio-proxy -- pilot-agent request POST 'logging?rbac=debug'

    ```
  3. jeesmon created this gist Mar 1, 2022.
    68 changes: 68 additions & 0 deletions istio-debug-cheatsheet.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,68 @@
    # Istio Debug

    ## JWKS

    https://github.com/istio/istio/pull/17176
    https://github.com/istio/istio/issues/15086

    ## Docs

    * Envoy - https://www.envoyproxy.io/docs/envoy/latest/

    ## Troubleshooting

    * Check proxy config

    ```
    istioctl pc listener istio-ingressgateway-xxxx-yyyy.istio-system -o json
    ````
    * Need istio injection for RequestAuthentication and AuthorizationPolicy to work
    * Host in Gateway and sni in EnvoyFilter should match for listener config
    * Check envoy version included in istio
    kubectl exec -it <pod> -c istio-proxy -n <ns> pilot-agent request GET server_info
    * DENY policy is checked first
    * local_reply_config to customize 403 output is supported only from envoy v1.15+
    * No customization available for upstream errors
    * Need a route for / is needed in VirtualService for EnvoyFilter to work if there is a separate route for context path
    * Analyze
    `istioctl analyze -n <ns>`
    * Describe
    `istioctl x describe service <ns>.<service>`
    * Install manifest
    `istioctl manifest install --set values.pilot.enableProtocolSniffingForOutbound=false`
    * Set envoy log level
    ```
    Method 1:

    istioctl dashboard envoy <pod>.<ns> --address "0.0.0.0"
    curl -X POST http://localhost:15000/logging?level=info
    kubectl -n <ns> logs -f <pod> istio-proxy

    Method 2:

    istioctl dashboard controlz $(kubectl -n istio-system get pods -l app=istiod -o jsonpath='{.items[0].metadata.name}').istio-system --port 15000 --address "0.0.0.0"
    kubectl -n tenant3 exec mock-6865f9b6bf-jlpjc -c istio-proxy -- pilot-agent request POST 'logging?connection=info'
    kubectl -n tenant3 exec mock-6865f9b6bf-jlpjc -c istio-proxy -- pilot-agent request POST 'logging?level=info'
    kubectl -n tenant3 exec mock-6865f9b6bf-jlpjc -c istio-proxy -- pilot-agent request POST 'logging?rbac=debug'

    ```
    * Authorization debug
    https://istio.io/latest/docs/ops/common-problems/security-issues/#ensure-istiod-accepts-the-policies