ingress nginx All the controller files should live on a different container Controller container should have bare minimum to work just go program No file other than NGINX files should exist on this container Inside nginx container there should be a really small http listener just able Proposal to split containers This includes not mounting the service account All the NGINX files should live on one container ServiceAccount should be mounted just on controller

Answer:
# Proposal to split containers

* All the NGINX files should live on one container
  * No file other than NGINX files should exist on this container
  * This includes not mounting the service account
* All the controller files should live on a different container
  * Controller container should have bare minimum to work (just go program)
  * ServiceAccount should be mounted just on controller

* Inside nginx container, there should be a really small http listener just able
to start, stop and reload NGINX

## Roadmap (what needs to be done)
* Map what needs to be done to mount the SA just on controller container
* Map all the required files for NGINX to work
* Map all the required network calls between controller and NGINX
  * eg.: Dynamic lua reconfiguration
* Map problematic features that will need attention
  * SSLPassthrough today happens on controller process and needs to happen on NGINX

### Ports and endpoints on NGINX container
* Public HTTP/HTTPs port - 80 and 443
* Lua configuration port - 10246 (HTTP) and 10247 (Stream)
* 3333 (temp) - Dataplane controller http server
  * /reload - (POST) Reloads the configuration.
    * "config" argument is the location of temporary file that should be used / moved to nginx.conf
  * /test - (POST) Test the configuration of a given file location
    * "config" argument is the location of temporary file that should be tested

### Mounting empty SA on controller container

```yaml
kind: Pod
apiVersion: v1
metadata:
  name: test
spec:
  containers:
  - name: nginx
    image: nginx:latest
    ports:
    - containerPort: 80
  - name: othernginx
    image: alpine:latest
    command: ["/bin/sh"]
    args: ["-c", "while true; do date; sleep 3; done"]
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: emptysecret
  volumes:
  - name: emptysecret
    emptyDir:
      sizeLimit: 1Mi
```

### Mapped folders on NGINX configuration
**WARNING** We need to be aware of inter mount containers and inode problems. If we
mount a file instead of a directory, it may take time to reflect the file value on
the target container

*  "/etc/nginx/lua/?.lua;/etc/nginx/lua/vendor/?.lua;;"; - Lua scripts
* "/var/log/nginx" - NGINX logs
* "/tmp/nginx (nginx.pid)" - NGINX pid directory / file, fcgi socket, etc
* " /etc/nginx/geoip" - GeoIP database directory - OK - /etc/ingress-controller/geoip
* /etc/nginx/mime.types - Mime types
* /etc/ingress-controller/ssl - SSL directory (fake cert, auth cert)
* /etc/ingress-controller/auth - Authentication files
* /etc/nginx/modsecurity - Modsecurity configuration
* /etc/nginx/owasp-modsecurity-crs - Modsecurity rules
* /etc/nginx/tickets.key - SSL tickets - OK - /etc/ingress-controller/tickets.key
* /etc/nginx/opentelemetry.toml - OTEL config - OK - /etc/ingress-controller/telemetry
* /etc/nginx/opentracing.json - Opentracing config - OK - /etc/ingress-controller/telemetry
* /etc/nginx/modules - NGINX modules
* /etc/nginx/fastcgi_params (maybe) - fcgi params
* /etc/nginx/template - Template, may be used by controller only

##### List of modules
```
ngx_http_auth_digest_module.so    ngx_http_modsecurity_module.so
ngx_http_brotli_filter_module.so  ngx_http_opentracing_module.so
ngx_http_brotli_static_module.so  ngx_stream_geoip2_module.so
ngx_http_geoip2_module.so
```

##### List of files that may be removed
```
-rw-r--r--    1 www-data www-data      1077 Jun 23 19:44 fastcgi.conf
-rw-r--r--    1 www-data www-data      1077 Jun 23 19:44 fastcgi.conf.default
-rw-r--r--    1 www-data www-data      1007 Jun 23 19:44 fastcgi_params
-rw-r--r--    1 www-data www-data      1007 Jun 23 19:44 fastcgi_params.default
drwxr-xr-x    2 www-data www-data      4096 Jun 23 19:34 geoip
-rw-r--r--    1 www-data www-data      2837 Jun 23 19:44 koi-utf
-rw-r--r--    1 www-data www-data      2223 Jun 23 19:44 koi-win
drwxr-xr-x    6 www-data www-data      4096 Sep 19 14:13 lua
-rw-r--r--    1 www-data www-data      5349 Jun 23 19:44 mime.types
-rw-r--r--    1 www-data www-data      5349 Jun 23 19:44 mime.types.default
drwxr-xr-x    2 www-data www-data      4096 Jun 23 19:44 modsecurity
drwxr-xr-x    2 www-data www-data      4096 Jun 23 19:44 modules
-rw-r--r--    1 www-data www-data     18275 Oct  1 21:28 nginx.conf
-rw-r--r--    1 www-data www-data      2656 Jun 23 19:44 nginx.conf.default
-rwx------    1 www-data www-data       420 Oct  1 21:28 opentelemetry.toml
-rw-r--r--    1 www-data www-data         2 Oct  1 21:28 opentracing.json
drwxr-xr-x    7 www-data www-data      4096 Jun 23 19:44 owasp-modsecurity-crs
-rw-r--r--    1 www-data www-data       636 Jun 23 19:44 scgi_params
-rw-r--r--    1 www-data www-data       636 Jun 23 19:44 scgi_params.default
drwxr-xr-x    2 www-data www-data      4096 Sep 19 14:13 template
-rw-r--r--    1 www-data www-data       664 Jun 23 19:44 uwsgi_params
-rw-r--r--    1 www-data www-data       664 Jun 23 19:44 uwsgi_params.default
-rw-r--r--    1 www-data www-data      3610 Jun 23 19:44 win-utf
```

Proposal to split containers    All the NGINX files should live on one container     No file other than NGINX files should exist on this container     This includes not mounting the service account   All the controller files should live on a different container     Controller container should have bare minimum to work  just go program      ServiceAccount should be mounted just on controller    Inside nginx container  there should be a really small http listener just able  to start  stop and reload NGINX     Roadmap  what needs to be done    Map what needs to be done to mount the SA just on controller container   Map all the required files for NGINX to work   Map all the required network calls between controller and NGINX     eg   Dynamic lua reconfiguration   Map problematic features that will need attention     SSLPassthrough today happens on controller process and needs to happen on NGINX      Ports and endpoints on NGINX container   Public HTTP HTTPs port   80 and 443   Lua configuration port   10246  HTTP  and 10247  Stream    3333  temp    Dataplane controller http server      reload    POST  Reloads the configuration         config  argument is the location of temporary file that should be used   moved to nginx conf      test    POST  Test the configuration of a given file location        config  argument is the location of temporary file that should be tested      Mounting empty SA on controller container     yaml kind  Pod apiVersion  v1 metadata    name  test spec    containers      name  nginx     image  nginx latest     ports        containerPort  80     name  othernginx     image  alpine latest     command     bin sh       args     c    while true  do date  sleep 3  done       volumeMounts        mountPath   var run secrets kubernetes io serviceaccount       name  emptysecret   volumes      name  emptysecret     emptyDir        sizeLimit  1Mi          Mapped folders on NGINX configuration   WARNING   We need to be aware of inter mount containers and inode problems  If we  mount a file instead of a directory  it may take time to reflect the file value on  the target container       etc nginx lua   lua  etc nginx lua vendor   lua       Lua scripts     var log nginx    NGINX logs     tmp nginx  nginx pid     NGINX pid directory   file  fcgi socket  etc      etc nginx geoip    GeoIP database directory   OK    etc ingress controller geoip    etc nginx mime types   Mime types    etc ingress controller ssl   SSL directory  fake cert  auth cert     etc ingress controller auth   Authentication files    etc nginx modsecurity   Modsecurity configuration    etc nginx owasp modsecurity crs   Modsecurity rules    etc nginx tickets key   SSL tickets   OK    etc ingress controller tickets key    etc nginx opentelemetry toml   OTEL config   OK    etc ingress controller telemetry    etc nginx opentracing json   Opentracing config   OK    etc ingress controller telemetry    etc nginx modules   NGINX modules    etc nginx fastcgi params  maybe    fcgi params    etc nginx template   Template  may be used by controller only        List of modules     ngx http auth digest module so    ngx http modsecurity module so ngx http brotli filter module so  ngx http opentracing module so ngx http brotli static module so  ngx stream geoip2 module so ngx http geoip2 module so            List of files that may be removed      rw r  r      1 www data www data      1077 Jun 23 19 44 fastcgi conf  rw r  r      1 www data www data      1077 Jun 23 19 44 fastcgi conf default  rw r  r      1 www data www data      1007 Jun 23 19 44 fastcgi params  rw r  r      1 www data www data      1007 Jun 23 19 44 fastcgi params default drwxr xr x    2 www data www data      4096 Jun 23 19 34 geoip  rw r  r      1 www data www data      2837 Jun 23 19 44 koi utf  rw r  r      1 www data www data      2223 Jun 23 19 44 koi win drwxr xr x    6 www data www data      4096 Sep 19 14 13 lua  rw r  r      1 www data www data      5349 Jun 23 19 44 mime types  rw r  r      1 www data www data      5349 Jun 23 19 44 mime types default drwxr xr x    2 www data www data      4096 Jun 23 19 44 modsecurity drwxr xr x    2 www data www data      4096 Jun 23 19 44 modules  rw r  r      1 www data www data     18275 Oct  1 21 28 nginx conf  rw r  r      1 www data www data      2656 Jun 23 19 44 nginx conf default  rwx          1 www data www data       420 Oct  1 21 28 opentelemetry toml  rw r  r      1 www data www data         2 Oct  1 21 28 opentracing json drwxr xr x    7 www data www data      4096 Jun 23 19 44 owasp modsecurity crs  rw r  r      1 www data www data       636 Jun 23 19 44 scgi params  rw r  r      1 www data www data       636 Jun 23 19 44 scgi params default drwxr xr x    2 www data www data      4096 Sep 19 14 13 template  rw r  r      1 www data www data       664 Jun 23 19 44 uwsgi params  rw r  r      1 www data www data       664 Jun 23 19 44 uwsgi params default  rw r  r      1 www data www data      3610 Jun 23 19 44 win utf


ingress nginx oscardoe janedoe approvers title KEP Template alicedoe reviewers TBD authors

Answer:
---
title: KEP Template
authors:
  - "@janedoe"
reviewers:
  - TBD
  - "@alicedoe"
approvers:
  - TBD
  - "@oscardoe"
editor: TBD
creation-date: yyyy-mm-dd
last-updated: yyyy-mm-dd
status: provisional|implementable|implemented|deferred|rejected|withdrawn|replaced
see-also:
  - "/docs/enhancements/20190101-we-heard-you-like-keps.md"
  - "/docs/enhancements/20190102-everyone-gets-a-kep.md"
replaces:
  - "/docs/enhancements/20181231-replaced-kep.md"
superseded-by:
  - "/docs/enhancements/20190104-superseding-kep.md"
---

# Title

This is the title of the KEP.
Keep it simple and descriptive.
A good title can help communicate what the KEP is and should be considered as part of any review.

The title should be lowercased and spaces/punctuation should be replaced with `-`.

To get started with this template:

1. **Make a copy of this template.**
  Create a copy of this template and name it `YYYYMMDD-my-title.md`, where `YYYYMMDD` is the date the KEP was first drafted.
1. **Fill out the "overview" sections.**
  This includes the Summary and Motivation sections.
  These should be easy if you've preflighted the idea of the KEP in an issue.
1. **Create a PR.**
  Assign it to folks that are sponsoring this process.
1. **Create an issue**
  When filing an enhancement tracking issue, please ensure to complete all fields in the template.
1. **Merge early.**
  Avoid getting hung up on specific details and instead aim to get the goal of the KEP merged quickly.
  The best way to do this is to just start with the "Overview" sections and fill out details incrementally in follow on PRs.
  View anything marked as a `provisional` as a working document and subject to change.
  Aim for single topic PRs to keep discussions focused.
  If you disagree with what is already in a document, open a new PR with suggested changes.

The canonical place for the latest set of instructions (and the likely source of this file) is [here](YYYYMMDD-kep-template.md).

The `Metadata` section above is intended to support the creation of tooling around the KEP process.
This will be a YAML section that is fenced as a code block.
See the KEP process for details on each of these items.

## Table of Contents

A table of contents is helpful for quickly jumping to sections of a KEP and for highlighting any additional information provided beyond the standard KEP template.

Ensure the TOC is wrapped with
- [Summary](#summary)
- [Motivation](#motivation)
  - [Goals](#goals)
  - [Non-Goals](#non-goals)
- [Proposal](#proposal)
  - [User Stories [optional]](#user-stories-optional)
    - [Story 1](#story-1)
    - [Story 2](#story-2)
  - [Implementation Details/Notes/Constraints [optional]](#implementation-detailsnotesconstraints-optional)
  - [Risks and Mitigations](#risks-and-mitigations)
- [Design Details](#design-details)
  - [Test Plan](#test-plan)
    - [Removing a deprecated flag](#removing-a-deprecated-flag)
- [Implementation History](#implementation-history)
- [Drawbacks [optional]](#drawbacks-optional)
- [Alternatives [optional]](#alternatives-optional)

## Summary

The `Summary` section is incredibly important for producing high quality user-focused documentation such as release notes or a development roadmap.
It should be possible to collect this information before implementation begins in order to avoid requiring implementers to split their attention between writing release notes and implementing the feature itself.

A good summary is probably at least a paragraph in length.

## Motivation

This section is for explicitly listing the motivation, goals and non-goals of this KEP.
Describe why the change is important and the benefits to users.
The motivation section can optionally provide links to [experience reports][] to demonstrate the interest in a KEP within the wider Kubernetes community.

[experience reports]: https://github.com/golang/go/wiki/ExperienceReports

### Goals

List the specific goals of the KEP.
How will we know that this has succeeded?

### Non-Goals

What is out of scope for this KEP?
Listing non-goals helps to focus discussion and make progress.

## Proposal

This is where we get down to the nitty gritty of what the proposal actually is.

### User Stories [optional]

Detail the things that people will be able to do if this KEP is implemented.
Include as much detail as possible so that people can understand the "how" of the system.
The goal here is to make this feel real for users without getting bogged down.

#### Story 1

#### Story 2

### Implementation Details/Notes/Constraints [optional]

What are the caveats to the implementation?
What are some important details that didn't come across above.
Go in to as much detail as necessary here.
This might be a good place to talk about core concepts and how they relate.

### Risks and Mitigations

What are the risks of this proposal and how do we mitigate.
Think broadly.
For example, consider both security and how this will impact the larger kubernetes ecosystem.

How will security be reviewed and by whom?
How will UX be reviewed and by whom?

Consider including folks that also work outside project.

## Design Details

### Test Plan

**Note:** *Section not required until targeted at a release.*

Consider the following in developing a test plan for this enhancement:

- Will there be e2e and integration tests, in addition to unit tests?
- How will it be tested in isolation vs with other components?

No need to outline all of the test cases, just the general strategy.
Anything that would count as tricky in the implementation and anything particularly challenging to test should be called out.

All code is expected to have adequate tests (eventually with coverage expectations).
Please adhere to the [Kubernetes testing guidelines][testing-guidelines] when drafting this test plan.

[testing-guidelines]: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md

#### Removing a deprecated flag

- Announce deprecation and support policy of the existing flag
- Two versions passed since introducing the functionality which deprecates the flag (to address version skew)
- Address feedback on usage/changed behavior, provided on GitHub issues
- Deprecate the flag

## Implementation History

Major milestones in the life cycle of a KEP should be tracked in `Implementation History`.
Major milestones might include

- the `Summary` and `Motivation` sections being merged signaling acceptance
- the `Proposal` section being merged signaling agreement on a proposed design
- the date implementation started
- the first Kubernetes release where an initial version of the KEP was available
- the version of Kubernetes where the KEP graduated to general availability
- when the KEP was retired or superseded

## Drawbacks [optional]

Why should this KEP _not_ be implemented.

## Alternatives [optional]

Similar to the `Drawbacks` section the `Alternatives` section is used to highlight and record other possible approaches to delivering the value proposed by a KEP.

title  KEP Template authors        janedoe  reviewers      TBD       alicedoe  approvers      TBD       oscardoe  editor  TBD creation date  yyyy mm dd last updated  yyyy mm dd status  provisional implementable implemented deferred rejected withdrawn replaced see also        docs enhancements 20190101 we heard you like keps md        docs enhancements 20190102 everyone gets a kep md  replaces        docs enhancements 20181231 replaced kep md  superseded by        docs enhancements 20190104 superseding kep md         Title  This is the title of the KEP  Keep it simple and descriptive  A good title can help communicate what the KEP is and should be considered as part of any review   The title should be lowercased and spaces punctuation should be replaced with       To get started with this template   1    Make a copy of this template      Create a copy of this template and name it  YYYYMMDD my title md   where  YYYYMMDD  is the date the KEP was first drafted  1    Fill out the  overview  sections      This includes the Summary and Motivation sections    These should be easy if you ve preflighted the idea of the KEP in an issue  1    Create a PR      Assign it to folks that are sponsoring this process  1    Create an issue     When filing an enhancement tracking issue  please ensure to complete all fields in the template  1    Merge early      Avoid getting hung up on specific details and instead aim to get the goal of the KEP merged quickly    The best way to do this is to just start with the  Overview  sections and fill out details incrementally in follow on PRs    View anything marked as a  provisional  as a working document and subject to change    Aim for single topic PRs to keep discussions focused    If you disagree with what is already in a document  open a new PR with suggested changes   The canonical place for the latest set of instructions  and the likely source of this file  is  here  YYYYMMDD kep template md    The  Metadata  section above is intended to support the creation of tooling around the KEP process  This will be a YAML section that is fenced as a code block  See the KEP process for details on each of these items      Table of Contents  A table of contents is helpful for quickly jumping to sections of a KEP and for highlighting any additional information provided beyond the standard KEP template   Ensure the TOC is wrapped with  code  lt     toc    rt  lt      toc    rt   code  tags  and then generate with  hack update toc sh         toc        Summary   summary     Motivation   motivation       Goals   goals       Non Goals   non goals     Proposal   proposal       User Stories  optional    user stories optional         Story 1   story 1         Story 2   story 2       Implementation Details Notes Constraints  optional    implementation detailsnotesconstraints optional       Risks and Mitigations   risks and mitigations     Design Details   design details       Test Plan   test plan         Removing a deprecated flag   removing a deprecated flag     Implementation History   implementation history     Drawbacks  optional    drawbacks optional     Alternatives  optional    alternatives optional        toc         Summary  The  Summary  section is incredibly important for producing high quality user focused documentation such as release notes or a development roadmap  It should be possible to collect this information before implementation begins in order to avoid requiring implementers to split their attention between writing release notes and implementing the feature itself   A good summary is probably at least a paragraph in length      Motivation  This section is for explicitly listing the motivation  goals and non goals of this KEP  Describe why the change is important and the benefits to users  The motivation section can optionally provide links to  experience reports    to demonstrate the interest in a KEP within the wider Kubernetes community    experience reports   https   github com golang go wiki ExperienceReports      Goals  List the specific goals of the KEP  How will we know that this has succeeded       Non Goals  What is out of scope for this KEP  Listing non goals helps to focus discussion and make progress      Proposal  This is where we get down to the nitty gritty of what the proposal actually is       User Stories  optional   Detail the things that people will be able to do if this KEP is implemented  Include as much detail as possible so that people can understand the  how  of the system  The goal here is to make this feel real for users without getting bogged down        Story 1       Story 2      Implementation Details Notes Constraints  optional   What are the caveats to the implementation  What are some important details that didn t come across above  Go in to as much detail as necessary here  This might be a good place to talk about core concepts and how they relate       Risks and Mitigations  What are the risks of this proposal and how do we mitigate  Think broadly  For example  consider both security and how this will impact the larger kubernetes ecosystem   How will security be reviewed and by whom  How will UX be reviewed and by whom   Consider including folks that also work outside project      Design Details      Test Plan    Note     Section not required until targeted at a release    Consider the following in developing a test plan for this enhancement     Will there be e2e and integration tests  in addition to unit tests    How will it be tested in isolation vs with other components   No need to outline all of the test cases  just the general strategy  Anything that would count as tricky in the implementation and anything particularly challenging to test should be called out   All code is expected to have adequate tests  eventually with coverage expectations   Please adhere to the  Kubernetes testing guidelines  testing guidelines  when drafting this test plan    testing guidelines   https   git k8s io community contributors devel sig testing testing md       Removing a deprecated flag    Announce deprecation and support policy of the existing flag   Two versions passed since introducing the functionality which deprecates the flag  to address version skew    Address feedback on usage changed behavior  provided on GitHub issues   Deprecate the flag     Implementation History  Major milestones in the life cycle of a KEP should be tracked in  Implementation History   Major milestones might include    the  Summary  and  Motivation  sections being merged signaling acceptance   the  Proposal  section being merged signaling agreement on a proposed design   the date implementation started   the first Kubernetes release where an initial version of the KEP was available   the version of Kubernetes where the KEP graduated to general availability   when the KEP was retired or superseded     Drawbacks  optional   Why should this KEP  not  be implemented      Alternatives  optional   Similar to the  Drawbacks  section the  Alternatives  section is used to highlight and record other possible approaches to delivering the value proposed by a KEP


ingress nginx Ingress NGINX Code Overview Core Golang code This document provides an overview of Ingress NGINX code This part of the code is responsible for the main logic of Ingress NGINX It contains all the logics that parses watches Endpoints and turn them into usable nginx conf configuration

Answer:
# Ingress NGINX - Code Overview

This document provides an overview of Ingress NGINX code.

## Core Golang code

This part of the code is responsible for the main logic of Ingress NGINX. It contains all the logics that parses [Ingress Objects](https://kubernetes.io/docs/concepts/services-networking/ingress/),
[annotations](https://kubernetes.io/docs/reference/glossary/?fundamental=true#term-annotation), watches Endpoints and turn them into usable nginx.conf configuration.

### Core Sync Logics:

Ingress-nginx has an internal model of the ingresses, secrets and endpoints in a given cluster. It maintains two copies of that:

1. One copy is the currently running configuration model
2. Second copy is the one generated in response to some changes in the cluster

The sync logic diffs the two models and if there's a change it tries to converge the running configuration to the new one.

There are static and dynamic configuration changes.

All endpoints and certificate changes are handled dynamically by posting the payload to an internal NGINX endpoint that is handled by Lua.

---

The following parts of the code can be found:

### Entrypoint

The `main` package is responsible for starting ingress-nginx program, which can be found in [cmd/nginx](https://github.com/kubernetes/ingress-nginx/tree/main/cmd/nginx) directory.

### Version

Is the package of the code responsible for adding `version` subcommand, and can be found in [version](https://github.com/kubernetes/ingress-nginx/tree/main/version) directory.

### Internal code

This part of the code contains the internal logics that compose Ingress NGINX Controller, and it's split into:

#### Admission Controller

Contains the code of [Kubernetes Admission Controller](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/) which validates the syntax of ingress objects before accepting it.

This code can be found in [internal/admission/controller](https://github.com/kubernetes/ingress-nginx/tree/main/internal/admission/controller) directory.

#### File functions

Contains auxiliary codes that deal with files, such as generating the SHA1 checksum of a file, or creating required directories.

This code can be found in [internal/file](https://github.com/kubernetes/ingress-nginx/blob/main/internal/file) directory.

#### Ingress functions

Contains all the logics from Ingress-Nginx Controller, with some examples being:

* Expected Golang structures that will be used in templates and other parts of the code - [internal/ingress/types.go](https://github.com/kubernetes/ingress-nginx/blob/main/internal/ingress/types.go).
* supported annotations and its parsing logics - [internal/ingress/annotations](https://github.com/kubernetes/ingress-nginx/tree/main/internal/ingress/annotations).
* reconciliation loops and logics - [internal/ingress/controller](https://github.com/kubernetes/ingress-nginx/tree/main/internal/ingress/controller)
* defaults - define the default struct - [internal/ingress/defaults](https://github.com/kubernetes/ingress-nginx/tree/main/internal/ingress/defaults).
* Error interface and types implementation - [internal/ingress/errors](https://github.com/kubernetes/ingress-nginx/tree/main/internal/ingress/errors)
* Metrics collectors for Prometheus exporting - [internal/ingress/metric](https://github.com/kubernetes/ingress-nginx/tree/main/internal/ingress/metric).
* Resolver - Extracts information from a controller - [internal/ingress/resolver](https://github.com/kubernetes/ingress-nginx/tree/main/internal/ingress/resolver).
* Ingress Object status publisher - [internal/ingress/status](https://github.com/kubernetes/ingress-nginx/tree/main/internal/ingress/status).

And other parts of the code that will be written in this document in a future.

#### K8s functions

Contains helper functions for parsing Kubernetes objects.

This part of the code can be found in [internal/k8s](https://github.com/kubernetes/ingress-nginx/tree/main/internal/k8s) directory.

#### Networking functions

Contains helper functions for networking, such as IPv4 and IPv6 parsing, SSL certificate parsing, etc.

This part of the code can be found in [internal/net](https://github.com/kubernetes/ingress-nginx/tree/main/internal/net) directory.

#### NGINX functions

Contains helper function to deal with NGINX, such as verify if it's running and reading it's configuration file parts.

This part of the code can be found in [internal/nginx](https://github.com/kubernetes/ingress-nginx/tree/main/internal/nginx) directory.

#### Tasks / Queue

Contains the functions responsible for the sync queue part of the controller.

This part of the code can be found in [internal/task](https://github.com/kubernetes/ingress-nginx/tree/main/internal/task) directory.

#### Other parts of internal

Other parts of internal code might not be covered here, like runtime and watch but they can be added in a future.

## E2E Test

The e2e tests code is in [test](https://github.com/kubernetes/ingress-nginx/tree/main/test) directory.

## Other programs

Describe here `kubectl plugin`, `dbg`, `waitshutdown` and cover the hack scripts.

### kubectl plugin

It contains kubectl plugin for inspecting your ingress-nginx deployments.
This part of code can be found in [cmd/plugin](https://github.com/kubernetes/ingress-nginx/tree/main/cmd/plugin) directory
Detail functions flow and available flow can be found in [kubectl-plugin](https://github.com/kubernetes/ingress-nginx/blob/main/docs/kubectl-plugin.md)

## Deploy files

This directory contains the `yaml` deploy files used as examples or references in the docs to deploy Ingress NGINX and other components.

Those files are in [deploy](https://github.com/kubernetes/ingress-nginx/tree/main/deploy) directory.

## Helm Chart

Used to generate the Helm chart published.

Code is in [charts/ingress-nginx](https://github.com/kubernetes/ingress-nginx/tree/main/charts/ingress-nginx).

## Documentation/Website

The documentation used to generate the website https://kubernetes.github.io/ingress-nginx/

This code is available in [docs](https://github.com/kubernetes/ingress-nginx/tree/main/docs) and it's main "language" is `Markdown`, used by [mkdocs](https://github.com/kubernetes/ingress-nginx/blob/main/mkdocs.yml) file to generate static pages.

## Container Images

Container images used to run ingress-nginx, or to build the final image.

### Base Images

Contains the `Dockerfiles` and scripts used to build base images that are used in other parts of the repo. They are present in [images](https://github.com/kubernetes/ingress-nginx/tree/main/images) repo. Some examples:
* [nginx](https://github.com/kubernetes/ingress-nginx/tree/main/images/nginx) - The base NGINX image ingress-nginx uses is not a vanilla NGINX. It bundles many libraries together and it is a job in itself to maintain that and keep things up-to-date.
* [custom-error-pages](https://github.com/kubernetes/ingress-nginx/tree/main/images/custom-error-pages) - Used on the custom error page examples.

There are other images inside this directory.

### Ingress Controller Image

The image used to build the final ingress controller, used in deploy scripts and Helm charts.

This is NGINX with some Lua enhancement. We do dynamic certificate, endpoints handling, canary traffic split, custom load balancing etc at this component. One can also add new functionalities using Lua plugin system.

The files are in [rootfs](https://github.com/kubernetes/ingress-nginx/tree/main/rootfs) directory and contains:

* The Dockerfile
* [nginx config](https://github.com/kubernetes/ingress-nginx/tree/main/rootfs/etc/nginx)

#### Ingress NGINX Lua Scripts

Ingress NGINX uses Lua Scripts to enable features like hot reloading, rate limiting and monitoring. Some are written using the [OpenResty](https://openresty.org/en/) helper.

The directory containing Lua scripts is [rootfs/etc/nginx/lua](https://github.com/kubernetes/ingress-nginx/tree/main/rootfs/etc/nginx/lua).

#### Nginx Go template file

One of the functions of Ingress NGINX is to turn [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) objects into nginx.conf file.

To do so, the final step is to apply those configurations in [nginx.tmpl](https://github.com/kubernetes/ingress-nginx/tree/main/rootfs/etc/nginx/template) turning it into a final nginx.conf file.

Ingress NGINX   Code Overview  This document provides an overview of Ingress NGINX code       Core Golang code  This part of the code is responsible for the main logic of Ingress NGINX  It contains all the logics that parses  Ingress Objects  https   kubernetes io docs concepts services networking ingress      annotations  https   kubernetes io docs reference glossary  fundamental true term annotation   watches Endpoints and turn them into usable nginx conf configuration        Core Sync Logics   Ingress nginx has an internal model of the ingresses  secrets and endpoints in a given cluster  It maintains two copies of that   1  One copy is the currently running configuration model 2  Second copy is the one generated in response to some changes in the cluster  The sync logic diffs the two models and if there s a change it tries to converge the running configuration to the new one    There are static and dynamic configuration changes    All endpoints and certificate changes are handled dynamically by posting the payload to an internal NGINX endpoint that is handled by Lua        The following parts of the code can be found       Entrypoint  The  main  package is responsible for starting ingress nginx program  which can be found in  cmd nginx  https   github com kubernetes ingress nginx tree main cmd nginx  directory       Version  Is the package of the code responsible for adding  version  subcommand  and can be found in  version  https   github com kubernetes ingress nginx tree main version  directory       Internal code  This part of the code contains the internal logics that compose Ingress NGINX Controller  and it s split into        Admission Controller  Contains the code of  Kubernetes Admission Controller  https   kubernetes io docs reference access authn authz admission controllers   which validates the syntax of ingress objects before accepting it   This code can be found in  internal admission controller  https   github com kubernetes ingress nginx tree main internal admission controller  directory         File functions  Contains auxiliary codes that deal with files  such as generating the SHA1 checksum of a file  or creating required directories   This code can be found in  internal file  https   github com kubernetes ingress nginx blob main internal file  directory        Ingress functions  Contains all the logics from Ingress Nginx Controller  with some examples being     Expected Golang structures that will be used in templates and other parts of the code    internal ingress types go  https   github com kubernetes ingress nginx blob main internal ingress types go     supported annotations and its parsing logics    internal ingress annotations  https   github com kubernetes ingress nginx tree main internal ingress annotations     reconciliation loops and logics    internal ingress controller  https   github com kubernetes ingress nginx tree main internal ingress controller    defaults   define the default struct    internal ingress defaults  https   github com kubernetes ingress nginx tree main internal ingress defaults     Error interface and types implementation    internal ingress errors  https   github com kubernetes ingress nginx tree main internal ingress errors    Metrics collectors for Prometheus exporting    internal ingress metric  https   github com kubernetes ingress nginx tree main internal ingress metric     Resolver   Extracts information from a controller    internal ingress resolver  https   github com kubernetes ingress nginx tree main internal ingress resolver     Ingress Object status publisher    internal ingress status  https   github com kubernetes ingress nginx tree main internal ingress status    And other parts of the code that will be written in this document in a future        K8s functions  Contains helper functions for parsing Kubernetes objects   This part of the code can be found in  internal k8s  https   github com kubernetes ingress nginx tree main internal k8s  directory        Networking functions  Contains helper functions for networking  such as IPv4 and IPv6 parsing  SSL certificate parsing  etc   This part of the code can be found in  internal net  https   github com kubernetes ingress nginx tree main internal net  directory        NGINX functions  Contains helper function to deal with NGINX  such as verify if it s running and reading it s configuration file parts   This part of the code can be found in  internal nginx  https   github com kubernetes ingress nginx tree main internal nginx  directory        Tasks   Queue  Contains the functions responsible for the sync queue part of the controller   This part of the code can be found in  internal task  https   github com kubernetes ingress nginx tree main internal task  directory        Other parts of internal  Other parts of internal code might not be covered here  like runtime and watch but they can be added in a future      E2E Test  The e2e tests code is in  test  https   github com kubernetes ingress nginx tree main test  directory      Other programs  Describe here  kubectl plugin    dbg    waitshutdown  and cover the hack scripts       kubectl plugin  It contains kubectl plugin for inspecting your ingress nginx deployments  This part of code can be found in  cmd plugin  https   github com kubernetes ingress nginx tree main cmd plugin  directory Detail functions flow and available flow can be found in  kubectl plugin  https   github com kubernetes ingress nginx blob main docs kubectl plugin md      Deploy files  This directory contains the  yaml  deploy files used as examples or references in the docs to deploy Ingress NGINX and other components   Those files are in  deploy  https   github com kubernetes ingress nginx tree main deploy  directory      Helm Chart  Used to generate the Helm chart published   Code is in  charts ingress nginx  https   github com kubernetes ingress nginx tree main charts ingress nginx       Documentation Website  The documentation used to generate the website https   kubernetes github io ingress nginx   This code is available in  docs  https   github com kubernetes ingress nginx tree main docs  and it s main  language  is  Markdown   used by  mkdocs  https   github com kubernetes ingress nginx blob main mkdocs yml  file to generate static pages      Container Images  Container images used to run ingress nginx  or to build the final image       Base Images  Contains the  Dockerfiles  and scripts used to build base images that are used in other parts of the repo  They are present in  images  https   github com kubernetes ingress nginx tree main images  repo  Some examples     nginx  https   github com kubernetes ingress nginx tree main images nginx    The base NGINX image ingress nginx uses is not a vanilla NGINX  It bundles many libraries together and it is a job in itself to maintain that and keep things up to date     custom error pages  https   github com kubernetes ingress nginx tree main images custom error pages    Used on the custom error page examples   There are other images inside this directory       Ingress Controller Image  The image used to build the final ingress controller  used in deploy scripts and Helm charts    This is NGINX with some Lua enhancement  We do dynamic certificate  endpoints handling  canary traffic split  custom load balancing etc at this component  One can also add new functionalities using Lua plugin system   The files are in  rootfs  https   github com kubernetes ingress nginx tree main rootfs  directory and contains     The Dockerfile    nginx config  https   github com kubernetes ingress nginx tree main rootfs etc nginx        Ingress NGINX Lua Scripts  Ingress NGINX uses Lua Scripts to enable features like hot reloading  rate limiting and monitoring  Some are written using the  OpenResty  https   openresty org en   helper   The directory containing Lua scripts is  rootfs etc nginx lua  https   github com kubernetes ingress nginx tree main rootfs etc nginx lua         Nginx Go template file  One of the functions of Ingress NGINX is to turn  Ingress  https   kubernetes io docs concepts services networking ingress   objects into nginx conf file    To do so  the final step is to apply those configurations in  nginx tmpl  https   github com kubernetes ingress nginx tree main rootfs etc nginx template  turning it into a final nginx conf file


ingress nginx that are needed to work with the Kubernetes ingress resource here is a link to the Developing for Ingress Nginx Controller http request termination of connection reverseproxy etc etc you can skip this and move on to the sections below For the really new contributors who want to contribute to the INGRESS NGINX project but need help with understanding some basic concepts This guide contains tips on how a http https request travels from a browser or a curl command to the webserver process running inside a container in a pod in a Kubernetes cluster but enters the cluster via a ingress resource For those who are familiar with those basic networking concepts like routing of a packet with regards to a This document explains how to get started with developing for Ingress Nginx Controller

Answer:
 Developing for Ingress-Nginx Controller

This document explains how to get started with developing for Ingress-Nginx Controller.

For the really new contributors, who want to contribute to the INGRESS-NGINX project, but need help with understanding some basic concepts,
that are needed to work with the Kubernetes ingress resource, here is a link to the [New Contributors Guide](https://github.com/kubernetes/ingress-nginx/blob/main/NEW_CONTRIBUTOR.md).
This guide contains tips on how a http/https request travels, from a browser or a curl command,
to the webserver process running inside a container, in a pod, in a Kubernetes cluster, but enters the cluster via a ingress resource.
For those who are familiar with those basic networking concepts like routing of a packet with regards to a
http request, termination of connection, reverseproxy etc. etc., you can skip this and move on to the sections below.
(or read it anyways just for context and also provide feedbacks if any)

## Prerequisites

Install [Go 1.14](https://golang.org/dl/) or later.

!!! note
    The project uses [Go Modules](https://github.com/golang/go/wiki/Modules)

Install [Docker](https://docs.docker.com/engine/install/) (v19.03.0 or later with experimental feature on)

Install [kubectl](https://kubernetes.io/docs/tasks/tools/) (1.24.0 or higher)

Install [Kind](https://kind.sigs.k8s.io/)

!!! important
    The majority of make tasks run as docker containers

## Quick Start

1. Fork the repository
2. Clone the repository to any location in your work station
3. Add a `GO111MODULE` environment variable with `export GO111MODULE=on`
4. Run `go mod download` to install dependencies

### Local build

Start a local Kubernetes cluster using [kind](https://kind.sigs.k8s.io/), build and deploy the ingress controller

```console
make dev-env
```
- If you are working on the v1.x.x version of this controller, and you want to create a cluster with kubernetes version 1.22, then please visit the [documentation for kind](https://kind.sigs.k8s.io/docs/user/configuration/#a-note-on-cli-parameters-and-configuration-files), and look for how to set a custom image for the kind node (image: kindest/node...), in the kind config file.

### Testing

**Run go unit tests**

```console
make test
```

**Run unit-tests for lua code**

```console
make lua-test
```

Lua tests are located in the directory `rootfs/etc/nginx/lua/test`

!!! important
    Test files must follow the naming convention `<mytest>_test.lua` or it will be ignored

**Run e2e test suite**

```console
make kind-e2e-test
```

To limit the scope of the tests to execute, we can use the environment variable `FOCUS`

```console
FOCUS="no-auth-locations" make kind-e2e-test
```

!!! note
    The variable `FOCUS` defines Ginkgo [Focused Specs](https://onsi.github.io/ginkgo/#focused-specs)

Valid values are defined in the describe definition of the e2e tests like [Default Backend](https://github.com/kubernetes/ingress-nginx/blob/main/test/e2e/defaultbackend/default_backend.go#L29)

The complete list of tests can be found [here](../e2e-tests.md)

### Custom docker image

In some cases, it can be useful to build a docker image and publish such an image to a private or custom registry location.

This can be done setting two environment variables, `REGISTRY` and `TAG`

```console
export TAG="dev"
export REGISTRY="$USER"

make build image
```

and then publish such version with

```console
docker push $REGISTRY/controller:$TAG
```

Developing for Ingress Nginx Controller  This document explains how to get started with developing for Ingress Nginx Controller   For the really new contributors  who want to contribute to the INGRESS NGINX project  but need help with understanding some basic concepts  that are needed to work with the Kubernetes ingress resource  here is a link to the  New Contributors Guide  https   github com kubernetes ingress nginx blob main NEW CONTRIBUTOR md   This guide contains tips on how a http https request travels  from a browser or a curl command  to the webserver process running inside a container  in a pod  in a Kubernetes cluster  but enters the cluster via a ingress resource  For those who are familiar with those basic networking concepts like routing of a packet with regards to a http request  termination of connection  reverseproxy etc  etc   you can skip this and move on to the sections below   or read it anyways just for context and also provide feedbacks if any      Prerequisites  Install  Go 1 14  https   golang org dl   or later       note     The project uses  Go Modules  https   github com golang go wiki Modules   Install  Docker  https   docs docker com engine install    v19 03 0 or later with experimental feature on   Install  kubectl  https   kubernetes io docs tasks tools    1 24 0 or higher   Install  Kind  https   kind sigs k8s io        important     The majority of make tasks run as docker containers     Quick Start   1  Fork the repository 2  Clone the repository to any location in your work station 3  Add a  GO111MODULE  environment variable with  export GO111MODULE on  4  Run  go mod download  to install dependencies      Local build  Start a local Kubernetes cluster using  kind  https   kind sigs k8s io    build and deploy the ingress controller     console make dev env       If you are working on the v1 x x version of this controller  and you want to create a cluster with kubernetes version 1 22  then please visit the  documentation for kind  https   kind sigs k8s io docs user configuration  a note on cli parameters and configuration files   and look for how to set a custom image for the kind node  image  kindest node      in the kind config file       Testing    Run go unit tests       console make test        Run unit tests for lua code       console make lua test      Lua tests are located in the directory  rootfs etc nginx lua test       important     Test files must follow the naming convention   mytest  test lua  or it will be ignored     Run e2e test suite       console make kind e2e test      To limit the scope of the tests to execute  we can use the environment variable  FOCUS      console FOCUS  no auth locations  make kind e2e test          note     The variable  FOCUS  defines Ginkgo  Focused Specs  https   onsi github io ginkgo  focused specs   Valid values are defined in the describe definition of the e2e tests like  Default Backend  https   github com kubernetes ingress nginx blob main test e2e defaultbackend default backend go L29   The complete list of tests can be found  here     e2e tests md       Custom docker image  In some cases  it can be useful to build a docker image and publish such an image to a private or custom registry location   This can be done setting two environment variables   REGISTRY  and  TAG      console export TAG  dev  export REGISTRY   USER   make build image      and then publish such version with     console docker push  REGISTRY controller  TAG


istio Shows how to do health checking for Istio services help ops app health check docs ops app health check help ops setup app health check aliases title Health Checking of Istio Services docs tasks traffic management app health check docs ops security health checks and mtls weight 50

Answer:
---
title: Health Checking of Istio Services
description: Shows how to do health checking for Istio services.
weight: 50
aliases:
  - /docs/tasks/traffic-management/app-health-check/
  - /docs/ops/security/health-checks-and-mtls/
  - /help/ops/setup/app-health-check
  - /help/ops/app-health-check
  - /docs/ops/app-health-check
  - /docs/ops/setup/app-health-check
keywords: [security,health-check]
owner: istio/wg-user-experience-maintainers
test: yes
---

[Kubernetes liveness and readiness probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/)
describes several ways to configure liveness and readiness probes:

1. [Command](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-a-liveness-command)
1. [HTTP request](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-a-liveness-http-request)
1. [TCP probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-a-tcp-liveness-probe)
1. [gRPC probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-a-grpc-liveness-probe)

The command approach works with no changes required, but HTTP requests, TCP probes, and gRPC probes require Istio to make changes to the pod configuration.

The health check requests to the `liveness-http` service are sent by Kubelet.
This becomes a problem when mutual TLS is enabled, because the Kubelet does not have an Istio issued certificate.
Therefore the health check requests will fail.

TCP probe checks need special handling, because Istio redirects all incoming traffic into the sidecar, and so all TCP ports appear open. The Kubelet simply checks if some process is listening on the specified port, and so the probe will always succeed as long as the sidecar is running.

Istio solves both these problems by rewriting the application `PodSpec` readiness/liveness probe,
so that the probe request is sent to the [sidecar agent](/docs/reference/commands/pilot-agent/).

## Liveness probe rewrite example

To demonstrate how the readiness/liveness probe is rewritten at the application `PodSpec` level, let us use the [liveness-http-same-port sample](/samples/health-check/liveness-http-same-port.yaml).

First create and label a namespace for the example:

$ kubectl create namespace istio-io-health-rewrite
$ kubectl label namespace istio-io-health-rewrite istio-injection=enabled

And deploy the sample application:

$ kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: liveness-http
  namespace: istio-io-health-rewrite
spec:
  selector:
    matchLabels:
      app: liveness-http
      version: v1
  template:
    metadata:
      labels:
        app: liveness-http
        version: v1
    spec:
      containers:
      - name: liveness-http
        image: docker.io/istio/health:example
        ports:
        - containerPort: 8001
        livenessProbe:
          httpGet:
            path: /foo
            port: 8001
          initialDelaySeconds: 5
          periodSeconds: 5
EOF

Once deployed, you can inspect the pod's application container to see the changed path:

$ kubectl get pod "$LIVENESS_POD" -n istio-io-health-rewrite -o json | jq '.spec.containers[0].livenessProbe.httpGet'
{
  "path": "/app-health/liveness-http/livez",
  "port": 15020,
  "scheme": "HTTP"
}

The original `livenessProbe` path is now mapped against the new path in the sidecar container environment variable `ISTIO_KUBE_APP_PROBERS`:

$ kubectl get pod "$LIVENESS_POD" -n istio-io-health-rewrite -o=jsonpath="{.spec.containers[1].env[?(@.name=='ISTIO_KUBE_APP_PROBERS')]}"
{
  "name":"ISTIO_KUBE_APP_PROBERS",
  "value":"{\"/app-health/liveness-http/livez\":{\"httpGet\":{\"path\":\"/foo\",\"port\":8001,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1}}"
}

For HTTP and gRPC requests, the sidecar agent redirects the request to the application and strips the response body, only returning the response code. For TCP probes, the sidecar agent will then do the port check while avoiding the traffic redirection.

The rewriting of problematic probes is enabled by default in all built-in Istio
[configuration profiles](/docs/setup/additional-setup/config-profiles/) but can be disabled as described below.

## Liveness and readiness probes using the command approach

Istio provides a [liveness sample](/samples/health-check/liveness-command.yaml) that
implements this approach. To demonstrate it working with mutual TLS enabled,
first create a namespace for the example:

$ kubectl create ns istio-io-health

To configure strict mutual TLS, run:

$ kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: "default"
  namespace: "istio-io-health"
spec:
  mtls:
    mode: STRICT
EOF

Next, change directory to the root of the Istio installation and run the following command to deploy the sample service:

$ kubectl -n istio-io-health apply -f <(istioctl kube-inject -f @samples/health-check/liveness-command.yaml@)

To confirm that the liveness probes are working, check the status of the sample pod to verify that it is running.

$ kubectl -n istio-io-health get pod
NAME                             READY     STATUS    RESTARTS   AGE
liveness-6857c8775f-zdv9r        2/2       Running   0           4m

## Liveness and readiness probes using the HTTP, TCP, and gRPC approach {#liveness-and-readiness-probes-using-the-http-request-approach}

As stated previously, Istio uses probe rewrite to implement HTTP, TCP, and gRPC probes by default. You can disable this
feature either for specific pods, or globally.

### Disable the probe rewrite for a pod {#disable-the-http-probe-rewrite-for-a-pod}

You can [annotate the pod](/docs/reference/config/annotations/) with `sidecar.istio.io/rewriteAppHTTPProbers: "false"`
to disable the probe rewrite option. Make sure you add the annotation to the
[pod resource](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/) because it will be ignored
anywhere else (for example, on an enclosing deployment resource).

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: liveness-http
spec:
  selector:
    matchLabels:
      app: liveness-http
      version: v1
  template:
    metadata:
      labels:
        app: liveness-http
        version: v1
      annotations:
        sidecar.istio.io/rewriteAppHTTPProbers: "false"
    spec:
      containers:
      - name: liveness-http
        image: docker.io/istio/health:example
        ports:
        - containerPort: 8001
        livenessProbe:
          httpGet:
            path: /foo
            port: 8001
          initialDelaySeconds: 5
          periodSeconds: 5
EOF

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: liveness-grpc
spec:
  selector:
    matchLabels:
      app: liveness-grpc
      version: v1
  template:
    metadata:
      labels:
        app: liveness-grpc
        version: v1
      annotations:
        sidecar.istio.io/rewriteAppHTTPProbers: "false"
    spec:
      containers:
      - name: etcd
        image: registry.k8s.io/etcd:3.5.1-0
        command: ["--listen-client-urls", "http://0.0.0.0:2379", "--advertise-client-urls", "http://127.0.0.1:2379", "--log-level", "debug"]
        ports:
        - containerPort: 2379
        livenessProbe:
          grpc:
            port: 2379
          initialDelaySeconds: 10
          periodSeconds: 5
EOF

This approach allows you to disable the health check probe rewrite gradually on individual deployments,
without reinstalling Istio.

### Disable the probe rewrite globally

[Install Istio](/docs/setup/install/istioctl/) using `--set values.sidecarInjectorWebhook.rewriteAppHTTPProbe=false`
to disable the probe rewrite globally. **Alternatively**, update the configuration map for the Istio sidecar injector:

$ kubectl get cm istio-sidecar-injector -n istio-system -o yaml | sed -e 's/"rewriteAppHTTPProbe": true/"rewriteAppHTTPProbe": false/' | kubectl apply -f -

## Cleanup

Remove the namespaces used for the examples:

$ kubectl delete ns istio-io-health istio-io-health-rewrite

title  Health Checking of Istio Services description  Shows how to do health checking for Istio services  weight  50 aliases       docs tasks traffic management app health check       docs ops security health checks and mtls       help ops setup app health check      help ops app health check      docs ops app health check      docs ops setup app health check keywords   security health check  owner  istio wg user experience maintainers test  yes       Kubernetes liveness and readiness probes  https   kubernetes io docs tasks configure pod container configure liveness readiness probes   describes several ways to configure liveness and readiness probes   1   Command  https   kubernetes io docs tasks configure pod container configure liveness readiness startup probes  define a liveness command  1   HTTP request  https   kubernetes io docs tasks configure pod container configure liveness readiness startup probes  define a liveness http request  1   TCP probe  https   kubernetes io docs tasks configure pod container configure liveness readiness startup probes  define a tcp liveness probe  1   gRPC probe  https   kubernetes io docs tasks configure pod container configure liveness readiness startup probes  define a grpc liveness probe   The command approach works with no changes required  but HTTP requests  TCP probes  and gRPC probes require Istio to make changes to the pod configuration   The health check requests to the  liveness http  service are sent by Kubelet  This becomes a problem when mutual TLS is enabled  because the Kubelet does not have an Istio issued certificate  Therefore the health check requests will fail   TCP probe checks need special handling  because Istio redirects all incoming traffic into the sidecar  and so all TCP ports appear open  The Kubelet simply checks if some process is listening on the specified port  and so the probe will always succeed as long as the sidecar is running   Istio solves both these problems by rewriting the application  PodSpec  readiness liveness probe  so that the probe request is sent to the  sidecar agent   docs reference commands pilot agent        Liveness probe rewrite example  To demonstrate how the readiness liveness probe is rewritten at the application  PodSpec  level  let us use the  liveness http same port sample   samples health check liveness http same port yaml    First create and label a namespace for the example      kubectl create namespace istio io health rewrite   kubectl label namespace istio io health rewrite istio injection enabled   And deploy the sample application      kubectl apply  f     EOF apiVersion  apps v1 kind  Deployment metadata    name  liveness http   namespace  istio io health rewrite spec    selector      matchLabels        app  liveness http       version  v1   template      metadata        labels          app  liveness http         version  v1     spec        containers          name  liveness http         image  docker io istio health example         ports            containerPort  8001         livenessProbe            httpGet              path   foo             port  8001           initialDelaySeconds  5           periodSeconds  5 EOF   Once deployed  you can inspect the pod s application container to see the changed path      kubectl get pod   LIVENESS POD   n istio io health rewrite  o json   jq   spec containers 0  livenessProbe httpGet       path     app health liveness http livez      port   15020     scheme    HTTP      The original  livenessProbe  path is now mapped against the new path in the sidecar container environment variable  ISTIO KUBE APP PROBERS       kubectl get pod   LIVENESS POD   n istio io health rewrite  o jsonpath    spec containers 1  env     name   ISTIO KUBE APP PROBERS           name   ISTIO KUBE APP PROBERS      value       app health liveness http livez      httpGet      path      foo     port   8001   scheme     HTTP      timeoutSeconds   1        For HTTP and gRPC requests  the sidecar agent redirects the request to the application and strips the response body  only returning the response code  For TCP probes  the sidecar agent will then do the port check while avoiding the traffic redirection   The rewriting of problematic probes is enabled by default in all built in Istio  configuration profiles   docs setup additional setup config profiles   but can be disabled as described below      Liveness and readiness probes using the command approach  Istio provides a  liveness sample   samples health check liveness command yaml  that implements this approach  To demonstrate it working with mutual TLS enabled  first create a namespace for the example      kubectl create ns istio io health   To configure strict mutual TLS  run      kubectl apply  f     EOF apiVersion  security istio io v1 kind  PeerAuthentication metadata    name   default    namespace   istio io health  spec    mtls      mode  STRICT EOF   Next  change directory to the root of the Istio installation and run the following command to deploy the sample service      kubectl  n istio io health apply  f   istioctl kube inject  f  samples health check liveness command yaml     To confirm that the liveness probes are working  check the status of the sample pod to verify that it is running      kubectl  n istio io health get pod NAME                             READY     STATUS    RESTARTS   AGE liveness 6857c8775f zdv9r        2 2       Running   0           4m      Liveness and readiness probes using the HTTP  TCP  and gRPC approach   liveness and readiness probes using the http request approach   As stated previously  Istio uses probe rewrite to implement HTTP  TCP  and gRPC probes by default  You can disable this feature either for specific pods  or globally       Disable the probe rewrite for a pod   disable the http probe rewrite for a pod   You can  annotate the pod   docs reference config annotations   with  sidecar istio io rewriteAppHTTPProbers   false   to disable the probe rewrite option  Make sure you add the annotation to the  pod resource  https   kubernetes io docs concepts workloads pods pod overview   because it will be ignored anywhere else  for example  on an enclosing deployment resource         kubectl apply  f     EOF apiVersion  apps v1 kind  Deployment metadata    name  liveness http spec    selector      matchLabels        app  liveness http       version  v1   template      metadata        labels          app  liveness http         version  v1       annotations          sidecar istio io rewriteAppHTTPProbers   false      spec        containers          name  liveness http         image  docker io istio health example         ports            containerPort  8001         livenessProbe            httpGet              path   foo             port  8001           initialDelaySeconds  5           periodSeconds  5 EOF        kubectl apply  f     EOF apiVersion  apps v1 kind  Deployment metadata    name  liveness grpc spec    selector      matchLabels        app  liveness grpc       version  v1   template      metadata        labels          app  liveness grpc         version  v1       annotations          sidecar istio io rewriteAppHTTPProbers   false      spec        containers          name  etcd         image  registry k8s io etcd 3 5 1 0         command      listen client urls    http   0 0 0 0 2379      advertise client urls    http   127 0 0 1 2379      log level    debug           ports            containerPort  2379         livenessProbe            grpc              port  2379           initialDelaySeconds  10           periodSeconds  5 EOF       This approach allows you to disable the health check probe rewrite gradually on individual deployments  without reinstalling Istio       Disable the probe rewrite globally   Install Istio   docs setup install istioctl   using    set values sidecarInjectorWebhook rewriteAppHTTPProbe false  to disable the probe rewrite globally    Alternatively    update the configuration map for the Istio sidecar injector      kubectl get cm istio sidecar injector  n istio system  o yaml   sed  e  s  rewriteAppHTTPProbe   true  rewriteAppHTTPProbe   false     kubectl apply  f        Cleanup  Remove the namespaces used for the examples      kubectl delete ns istio io health istio io health rewrite


istio test no owner istio wg networking maintainers keywords scalability title Configuration Scoping In order to program the service mesh the Istio control plane Istiod reads a variety of configurations including core Kubernetes types like and Shows how to scope configuration in Istio for operational and performance benefits weight 60

Answer:
---
title: Configuration Scoping
description: Shows how to scope configuration in Istio, for operational and performance benefits.
weight: 60
keywords: [scalability]
owner: istio/wg-networking-maintainers
test: no
---

In order to program the service mesh, the Istio control plane (Istiod) reads a variety of configurations, including core Kubernetes types like `Service` and `Node`,
and Istio's own types like `Gateway`.
These are then sent to the data plane (see [Architecture](/docs/ops/deployment/architecture/) for more information).

By default, the control plane will read all configuration in all namespaces.
Each proxy instance will receive configuration for all namespaces as well.
This includes information about workloads that are not enrolled in the mesh.

This default ensures correct behavior out of the box, but comes with a scalability cost.
Each configuration has a cost (in CPU and memory, primarily) to maintain and keep up to date.
At large scales, it is critical to limit the configuration scope to avoid excessive resource consumption.

## Scoping mechanisms

Istio offers a few tools to help control the scope of a configuration to meet different use cases.
Depending on your requirements, these can be used alone or together.

* `Sidecar` provides a mechanism for specific workloads to _import_ a set of configurations
* `exportTo` provides a mechanism to _export_ a configuration to a set of workloads
* `discoverySelectors` provides a mechanism to let Istio completely ignore a set of configurations

### `Sidecar` import

The [`egress.hosts`](/docs/reference/config/networking/sidecar/#IstioEgressListener) field in `Sidecar`
allows specifying a list of configurations to import.
Only configurations matching the specified criteria will be seen by sidecars impacted by the `Sidecar` resource.

For example:

apiVersion: networking.istio.io/v1
kind: Sidecar
metadata:
  name: default
spec:
  egress:
  - hosts:
    - "./*" # Import all configuration from our own namespace
    - "bookinfo/*" # Import all configuration from the bookinfo namespace
    - "external-services/example.com" # Import only 'example.com' from the external-services namespace

### `exportTo`

Istio's `VirtualService`, `DestinationRule`, and `ServiceEntry` provide a `spec.exportTo` field.
Similarly, `Service` can be configured with the `networking.istio.io/exportTo` annotation.

Unlike `Sidecar` which allows a workload owner to control what dependencies it has, `exportTo` works in the opposite way, and allows the service owners to control
their own service's visibility.

For example, this configuration makes the `details` `Service` only visible to its own namespace, and the `client` namespace:

apiVersion: v1
kind: Service
metadata:
  name: details
  annotations:
    networking.istio.io/exportTo: ".,client"
spec: ...

### `DiscoverySelectors`

While the previous controls operate on a workload or service owner level, [`DiscoverySelectors`](/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig) provides mesh wide control over configuration visibility.
Discovery selectors allows specifying criteria for which namespaces should be visible to the control plane.
Any namespaces not matching are ignored by the control plane entirely.

This can be configured as part of `meshConfig` during installation. For example:

meshConfig:
  discoverySelectors:
    - matchLabels:
        # Allow any namespaces with `istio-discovery=enabled`
        istio-discovery: enabled
    - matchLabels:
        # Allow "kube-system"; Kubernetes automatically adds this label to each namespace
        kubernetes.io/metadata.name: kube-system

Istiod will always open a watch to Kubernetes for all namespaces.
However, discovery selectors will ignore objects that are not selected very early in its processing, minimizing costs.

## Frequently asked questions

### How can I understand the cost of a certain configuration?

In order to get the best return-on-investment for scoping down configuration, it can be helpful to understand the cost of each object.
Unfortunately, there is not a straightforward answer; scalability depends on a large number of factors.
However, there are a few general guidelines:

Configuration *changes* are expensive in Istio, as they require recomputation.
While `Endpoints` changes (generally from a Pod scaling up or down) are heavily optimized, most other configurations are fairly expensive.
This can be especially harmful when controllers are constantly making changes to an object (sometimes this happens accidentally!).
Some tools to detect which configurations are changing:
* Istiod will log each change like: `Push debounce stable 1 for config Gateway/default/gateway: ..., full=true`.
  This shows a `Gateway` object in the `default` namespace changed. `full=false` would represent and optimized update such as `Endpoint`.
  Note: changes to `Service` and `Endpoints` will all show as `ServiceEntry`.
* Istiod exposes metrics `pilot_k8s_cfg_events` and `pilot_k8s_reg_events` for each change.
* `kubectl get <resource> --watch -oyaml --show-managed-fields` can show changes to an object (or objects) to help understand what is changing, and by whom.

[Headless services](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services) (besides ones declared as [HTTP](/docs/ops/configuration/traffic-management/protocol-selection/#explicit-protocol-selection))
scale with the number of instances. This makes large headless services expensive, and a good candidate for exclusion with `exportTo` or equivalent.

### What happens if I connect to a service outside of my scope?

When connecting to a service that has been excluded through one of the scoping mechanisms, the data plane will not know anything about the destination,
so it will be treated as [Unmatched traffic](/docs/ops/configuration/traffic-management/traffic-routing/#unmatched-traffic).

### What about Gateways?

While [Gateways](/docs/setup/additional-setup/gateway/) will respect `exportTo` and `DiscoverySelectors`, `Sidecar` objects do not impact Gateways.
However, unlike sidecars, gateways do not have configuration for the entire cluster by default.
Instead, each configuration is explicitly attached to the gateway, which mostly avoids this problem.

However, [currently](https://github.com/istio/istio/issues/29131) part of the data plane configuration (a "cluster", in Envoy terms), is always sent for
the entire cluster, even if it is not referenced explicitly.

title  Configuration Scoping description  Shows how to scope configuration in Istio  for operational and performance benefits  weight  60 keywords   scalability  owner  istio wg networking maintainers test  no      In order to program the service mesh  the Istio control plane  Istiod  reads a variety of configurations  including core Kubernetes types like  Service  and  Node   and Istio s own types like  Gateway   These are then sent to the data plane  see  Architecture   docs ops deployment architecture   for more information    By default  the control plane will read all configuration in all namespaces  Each proxy instance will receive configuration for all namespaces as well  This includes information about workloads that are not enrolled in the mesh   This default ensures correct behavior out of the box  but comes with a scalability cost  Each configuration has a cost  in CPU and memory  primarily  to maintain and keep up to date  At large scales  it is critical to limit the configuration scope to avoid excessive resource consumption      Scoping mechanisms  Istio offers a few tools to help control the scope of a configuration to meet different use cases  Depending on your requirements  these can be used alone or together      Sidecar  provides a mechanism for specific workloads to  import  a set of configurations    exportTo  provides a mechanism to  export  a configuration to a set of workloads    discoverySelectors  provides a mechanism to let Istio completely ignore a set of configurations       Sidecar  import  The   egress hosts    docs reference config networking sidecar  IstioEgressListener  field in  Sidecar  allows specifying a list of configurations to import  Only configurations matching the specified criteria will be seen by sidecars impacted by the  Sidecar  resource   For example    apiVersion  networking istio io v1 kind  Sidecar metadata    name  default spec    egress      hosts                Import all configuration from our own namespace        bookinfo      Import all configuration from the bookinfo namespace        external services example com    Import only  example com  from the external services namespace        exportTo   Istio s  VirtualService    DestinationRule   and  ServiceEntry  provide a  spec exportTo  field  Similarly   Service  can be configured with the  networking istio io exportTo  annotation   Unlike  Sidecar  which allows a workload owner to control what dependencies it has   exportTo  works in the opposite way  and allows the service owners to control their own service s visibility   For example  this configuration makes the  details   Service  only visible to its own namespace  and the  client  namespace    apiVersion  v1 kind  Service metadata    name  details   annotations      networking istio io exportTo     client  spec             DiscoverySelectors   While the previous controls operate on a workload or service owner level    DiscoverySelectors    docs reference config istio mesh v1alpha1  MeshConfig  provides mesh wide control over configuration visibility  Discovery selectors allows specifying criteria for which namespaces should be visible to the control plane  Any namespaces not matching are ignored by the control plane entirely   This can be configured as part of  meshConfig  during installation  For example    meshConfig    discoverySelectors        matchLabels            Allow any namespaces with  istio discovery enabled          istio discovery  enabled       matchLabels            Allow  kube system   Kubernetes automatically adds this label to each namespace         kubernetes io metadata name  kube system    Istiod will always open a watch to Kubernetes for all namespaces  However  discovery selectors will ignore objects that are not selected very early in its processing  minimizing costs       Frequently asked questions      How can I understand the cost of a certain configuration   In order to get the best return on investment for scoping down configuration  it can be helpful to understand the cost of each object  Unfortunately  there is not a straightforward answer  scalability depends on a large number of factors  However  there are a few general guidelines   Configuration  changes  are expensive in Istio  as they require recomputation  While  Endpoints  changes  generally from a Pod scaling up or down  are heavily optimized  most other configurations are fairly expensive  This can be especially harmful when controllers are constantly making changes to an object  sometimes this happens accidentally    Some tools to detect which configurations are changing    Istiod will log each change like   Push debounce stable 1 for config Gateway default gateway       full true     This shows a  Gateway  object in the  default  namespace changed   full false  would represent and optimized update such as  Endpoint     Note  changes to  Service  and  Endpoints  will all show as  ServiceEntry     Istiod exposes metrics  pilot k8s cfg events  and  pilot k8s reg events  for each change     kubectl get  resource    watch  oyaml   show managed fields  can show changes to an object  or objects  to help understand what is changing  and by whom    Headless services  https   kubernetes io docs concepts services networking service  headless services   besides ones declared as  HTTP   docs ops configuration traffic management protocol selection  explicit protocol selection   scale with the number of instances  This makes large headless services expensive  and a good candidate for exclusion with  exportTo  or equivalent       What happens if I connect to a service outside of my scope   When connecting to a service that has been excluded through one of the scoping mechanisms  the data plane will not know anything about the destination  so it will be treated as  Unmatched traffic   docs ops configuration traffic management traffic routing  unmatched traffic        What about Gateways   While  Gateways   docs setup additional setup gateway   will respect  exportTo  and  DiscoverySelectors    Sidecar  objects do not impact Gateways  However  unlike sidecars  gateways do not have configuration for the entire cluster by default  Instead  each configuration is explicitly attached to the gateway  which mostly avoids this problem   However   currently  https   github com istio istio issues 29131  part of the data plane configuration  a  cluster   in Envoy terms   is always sent for the entire cluster  even if it is not referenced explicitly


istio Shows common examples of using Istio security policy title Security policy examples owner istio wg security maintainers test yes weight 60 Background

Answer:
---
title: Security policy examples
description: Shows common examples of using Istio security policy.
weight: 60
owner: istio/wg-security-maintainers
test: yes
---

## Background

This page shows common patterns of using Istio security policies. You may find them useful in your deployment or use this
as a quick reference to example policies.

The policies demonstrated here are just examples and require changes to adapt to your actual environment
before applying.

Also read the [authentication](/docs/tasks/security/authentication/authn-policy) and
[authorization](/docs/tasks/security/authorization) tasks for a hands-on tutorial of using the security policy in
more detail.

## Require different JWT issuer per host

JWT validation is common on the ingress gateway and you may want to require different JWT issuers for different
hosts. You can use the authorization policy for fine grained JWT validation in addition to the
[request authentication](/docs/tasks/security/authentication/authn-policy/#end-user-authentication) policy.

Use the following policy if you want to allow access to the given hosts if JWT principal matches. Access to other hosts
will always be denied.

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: jwt-per-host
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  action: ALLOW
  rules:
  - from:
    - source:
        # the JWT token must have issuer with suffix "@example.com"
        requestPrincipals: ["*@example.com"]
    to:
    - operation:
        hosts: ["example.com", "*.example.com"]
  - from:
    - source:
        # the JWT token must have issuer with suffix "@another.org"
        requestPrincipals: ["*@another.org"]
    to:
    - operation:
        hosts: [".another.org", "*.another.org"]

## Namespace isolation

The following two policies enable strict mTLS on namespace `foo`, and allow traffic from the same namespace.

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: foo
spec:
  mtls:
    mode: STRICT
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: foo-isolation
  namespace: foo
spec:
  action: ALLOW
  rules:
  - from:
    - source:
        namespaces: ["foo"]

## Namespace isolation with ingress exception

The following two policies enable strict mTLS on namespace `foo`, and allow traffic from the same namespace and also
from the ingress gateway.

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: foo
spec:
  mtls:
    mode: STRICT
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: ns-isolation-except-ingress
  namespace: foo
spec:
  action: ALLOW
  rules:
  - from:
    - source:
        namespaces: ["foo"]
    - source:
        principals: ["cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"]

## Require mTLS in authorization layer (defense in depth)

You have configured `PeerAuthentication` to `STRICT` but want to make sure the traffic is indeed protected by mTLS with
an extra check in the authorization layer, i.e., defense in depth.

The following policy denies the request if the principal is empty. The principal will be empty if plain text is used.
In other words, the policy allows requests if the principal is non-empty.
`"*"` means non-empty match and using with `notPrincipals` means matching on empty principal.

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: require-mtls
  namespace: foo
spec:
  action: DENY
  rules:
  - from:
    - source:
        notPrincipals: ["*"]

## Require mandatory authorization check with `DENY` policy

You can use the `DENY` policy if you want to require mandatory authorization check that must be satisfied and cannot be
bypassed by another more permissive `ALLOW` policy. This works because the `DENY` policy takes precedence over the
`ALLOW` policy and could deny a request early before `ALLOW` policies.

Use the following policy to enforce mandatory JWT validation in addition to the [request authentication](/docs/tasks/security/authentication/authn-policy/#end-user-authentication) policy.
The policy denies the request if the request principal is empty. The request principal will be empty if JWT validation failed.
In other words, the policy allows requests if the request principal is non-empty.
`"*"` means non-empty match and using with `notRequestPrincipals` means matching on empty request principal.

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: require-jwt
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  action: DENY
  rules:
  - from:
    - source:
        notRequestPrincipals: ["*"]

Similarly, Use the following policy to require mandatory namespace isolation and also allow requests from ingress gateway.
The policy denies the request if the namespace is not `foo` and the principal is not `cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account`.
In other words, the policy allows the request only if the namespace is `foo` or the principal is `cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account`.

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: ns-isolation-except-ingress
  namespace: foo
spec:
  action: DENY
  rules:
  - from:
    - source:
        notNamespaces: ["foo"]
        notPrincipals: ["cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"]

title  Security policy examples description  Shows common examples of using Istio security policy  weight  60 owner  istio wg security maintainers test  yes         Background  This page shows common patterns of using Istio security policies  You may find them useful in your deployment or use this as a quick reference to example policies   The policies demonstrated here are just examples and require changes to adapt to your actual environment before applying   Also read the  authentication   docs tasks security authentication authn policy  and  authorization   docs tasks security authorization  tasks for a hands on tutorial of using the security policy in more detail      Require different JWT issuer per host  JWT validation is common on the ingress gateway and you may want to require different JWT issuers for different hosts  You can use the authorization policy for fine grained JWT validation in addition to the  request authentication   docs tasks security authentication authn policy  end user authentication  policy   Use the following policy if you want to allow access to the given hosts if JWT principal matches  Access to other hosts will always be denied    apiVersion  security istio io v1 kind  AuthorizationPolicy metadata    name  jwt per host   namespace  istio system spec    selector      matchLabels        istio  ingressgateway   action  ALLOW   rules      from        source            the JWT token must have issuer with suffix   example com          requestPrincipals      example com       to        operation          hosts    example com      example com       from        source            the JWT token must have issuer with suffix   another org          requestPrincipals      another org       to        operation          hosts     another org      another org        Namespace isolation  The following two policies enable strict mTLS on namespace  foo   and allow traffic from the same namespace    apiVersion  security istio io v1 kind  PeerAuthentication metadata    name  default   namespace  foo spec    mtls      mode  STRICT     apiVersion  security istio io v1 kind  AuthorizationPolicy metadata    name  foo isolation   namespace  foo spec    action  ALLOW   rules      from        source          namespaces    foo        Namespace isolation with ingress exception  The following two policies enable strict mTLS on namespace  foo   and allow traffic from the same namespace and also from the ingress gateway    apiVersion  security istio io v1 kind  PeerAuthentication metadata    name  default   namespace  foo spec    mtls      mode  STRICT     apiVersion  security istio io v1 kind  AuthorizationPolicy metadata    name  ns isolation except ingress   namespace  foo spec    action  ALLOW   rules      from        source          namespaces    foo         source          principals    cluster local ns istio system sa istio ingressgateway service account        Require mTLS in authorization layer  defense in depth   You have configured  PeerAuthentication  to  STRICT  but want to make sure the traffic is indeed protected by mTLS with an extra check in the authorization layer  i e   defense in depth   The following policy denies the request if the principal is empty  The principal will be empty if plain text is used  In other words  the policy allows requests if the principal is non empty        means non empty match and using with  notPrincipals  means matching on empty principal    apiVersion  security istio io v1 kind  AuthorizationPolicy metadata    name  require mtls   namespace  foo spec    action  DENY   rules      from        source          notPrincipals             Require mandatory authorization check with  DENY  policy  You can use the  DENY  policy if you want to require mandatory authorization check that must be satisfied and cannot be bypassed by another more permissive  ALLOW  policy  This works because the  DENY  policy takes precedence over the  ALLOW  policy and could deny a request early before  ALLOW  policies   Use the following policy to enforce mandatory JWT validation in addition to the  request authentication   docs tasks security authentication authn policy  end user authentication  policy  The policy denies the request if the request principal is empty  The request principal will be empty if JWT validation failed  In other words  the policy allows requests if the request principal is non empty        means non empty match and using with  notRequestPrincipals  means matching on empty request principal    apiVersion  security istio io v1 kind  AuthorizationPolicy metadata    name  require jwt   namespace  istio system spec    selector      matchLabels        istio  ingressgateway   action  DENY   rules      from        source          notRequestPrincipals          Similarly  Use the following policy to require mandatory namespace isolation and also allow requests from ingress gateway  The policy denies the request if the namespace is not  foo  and the principal is not  cluster local ns istio system sa istio ingressgateway service account   In other words  the policy allows the request only if the namespace is  foo  or the principal is  cluster local ns istio system sa istio ingressgateway service account     apiVersion  security istio io v1 kind  AuthorizationPolicy metadata    name  ns isolation except ingress   namespace  foo spec    action  DENY   rules      from        source          notNamespaces    foo           notPrincipals    cluster local ns istio system sa istio ingressgateway service account