Controlling if/when the ApplicationSet controller modifies Application resources

The ApplicationSet controller supports a number of settings that limit the ability of the controller to make changes to generated Applications, for example, preventing the controller from deleting child Applications.

These settings allow you to exert control over when, and how, changes are made to your Applications, and to their corresponding cluster resources (Deployments, Services, etc).

Here are some of the controller settings that may be modified to alter the ApplicationSet controller’s resource-handling behaviour.

Dry run: prevent ApplicationSet from creating, modifying, or deleting all Applications

To prevent the ApplicationSet controller from creating, modifying, or deleting any Application resources, you may enable dry-run mode. This essentially switches the controller into a “read only” mode, where the controller Reconcile loop will run, but no resources will be modified.

To enable dry-run, add --dryrun true to the ApplicationSet Deployment’s container launch parameters.

See ‘How to modify ApplicationSet container parameters’ below for detailed steps on how to add this parameter to the controller.

Managed Applications modification Policies

The ApplicationSet controller supports a parameter --policy, which is specified on launch (within the controller Deployment container), and which restricts what types of modifications will be made to managed Argo CD Application resources.

The --policy parameter takes four values: sync, create-only, create-delete, and create-update. (sync is the default, which is used if the --policy parameter is not specified; the other policies are described below).

It is also possible to set this policy per ApplicationSet.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
spec:
  # (...)
  syncPolicy:
    applicationsSync: create-only # create-update, create-delete sync

• Policy create-only: Prevents ApplicationSet controller from modifying or deleting Applications.
• Policy create-update: Prevents ApplicationSet controller from deleting Applications. Update is allowed.
• Policy create-delete: Prevents ApplicationSet controller from modifying Applications. Delete is allowed.
• Policy sync: Update and Delete are allowed.

If the controller parameter --policy is set, it takes precedence on the field applicationsSync. It is possible to allow per ApplicationSet sync policy by setting variable ARGOCD_APPLICATIONSET_CONTROLLER_ENABLE_POLICY_OVERRIDE to argocd-cmd-params-cm applicationsetcontroller.enable.policy.override or directly with controller parameter --enable-policy-override (default to false).

This does not prevent deletion of Applications if the ApplicationSet is deleted

Controller parameter

To allow the ApplicationSet controller to create Application resources, but prevent any further modification, such as deletion, or modification of Application fields, add this parameter in the ApplicationSet controller:

--policy create-only

At ApplicationSet level

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
spec:
  # (...)
  syncPolicy:
    applicationsSync: create-only

Policy - create-update: Prevent ApplicationSet controller from deleting Applications

To allow the ApplicationSet controller to create or modify Application resources, but prevent Applications from being deleted, add the following parameter to the ApplicationSet controller Deployment:

--policy create-update

This may be useful to users looking for additional protection against deletion of the Applications generated by the controller.

At ApplicationSet level

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
spec:
  # (...)
  syncPolicy:
    applicationsSync: create-update

Prevent an Application‘s child resources from being deleted, when the parent Application is deleted

By default, when an Application resource is deleted by the ApplicationSet controller, all of the child resources of the Application will be deleted as well (such as, all of the Application’s Deployments, Services, etc).

To prevent an Application’s child resources from being deleted when the parent Application is deleted, add the preserveResourcesOnDeletion: true field to the syncPolicy of the ApplicationSet:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
spec:
  # (...)
  syncPolicy:
    preserveResourcesOnDeletion: true

More information on the specific behaviour of preserveResourcesOnDeletion, and deletion in ApplicationSet controller and Argo CD in general, can be found on the Application Deletion page.

Prevent an Application’s child resources from being modified

Changes made to the ApplicationSet will propagate to the Applications managed by the ApplicationSet, and then Argo CD will propagate the Application changes to the underlying cluster resources (as per Argo CD Integration).

The propagation of Application changes to the cluster is managed by the automated sync settings, which are referenced in the ApplicationSet template field:

• spec.template.syncPolicy.automated: If enabled, changes to Applications will automatically propagate to the cluster resources of the cluster.
  • Unset this within the ApplicationSet template to ‘pause’ updates to cluster resources managed by the Application resource.
• spec.template.syncPolicy.automated.prune: By default, Automated sync will not delete resources when Argo CD detects the resource is no longer defined in Git.
  • For extra safety, set this to false to prevent unexpected changes to the backing Git repository from affecting cluster resources.

How to modify ApplicationSet container launch parameters

There are a couple of ways to modify the ApplicationSet container parameters, so as to enable the above settings.

A) Use kubectl edit to modify the deployment on the cluster

Edit the applicationset-controller Deployment resource on the cluster:

kubectl edit deployment/argocd-applicationset-controller -n argocd

Locate the .spec.template.spec.containers[0].command field, and add the required parameter(s):

spec:
    # (...)
  template:
    # (...)
    spec:
      containers:
      - command:
        - entrypoint.sh
        - argocd-applicationset-controller
        # Insert new parameters here, for example:
        # --policy create-only
    # (...)

Save and exit the editor. Wait for a new Pod to start containing the updated parameters.

Or, B) Edit the install.yaml manifest for the ApplicationSet installation

Rather than directly editing the cluster resource, you may instead choose to modify the installation YAML that is used to install the ApplicationSet controller:

Applicable for applicationset versions less than 0.4.0.

# Clone the repository
git clone https://github.com/argoproj/applicationset
# Checkout the version that corresponds to the one you have installed.
git checkout "(version of applicationset)"
# example: git checkout "0.1.0"
cd applicationset/manifests
# open 'install.yaml' in a text editor, make the same modifications to Deployment 
# as described in the previous section.
# Apply the change to the cluster
kubectl apply -n argocd -f install.yaml

Preserving changes made to an Applications annotations

It is common practice in Kubernetes to store state in annotations, operators will often make use of this. To allow for this, it is possible to configure a list of annotations that the ApplicationSet should preserve when reconciling.

For example, imagine that we have an Application created from an ApplicationSet, but a custom annotation has since been added (to the Application) that does not exist in the ApplicationSet resource:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations: 
    # This annotation exists only on this Application, and not in 
    # the parent ApplicationSet template:
    my-custom-annotation: some-value
spec:
  # (...)

To preserve this annotation we can use the preservedFields property of the ApplicationSet like so:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
spec:
  # (...)
  preservedFields:
    annotations: ["my-custom-annotation"]

The ApplicationSet controller will leave this annotation as-is when reconciling, even though it is not an annotation defined in the metadata of the ApplicationSet itself.

By default, the Argo CD notifications and the Argo CD refresh type annotations are also preserved.

Limitations: what isn’t supported as of the current release

Here is a list of commonly requested resource modification features which are not supported as of the current release. This lack of support is not necessarily by design; rather these behaviours are documented here to provide clear, concise descriptions of the current state of the feature.

Limitation: No support for manual edits to individual Applications

There is currently no way to allow modification of a single child Application of an ApplicationSet, for example, if you wanted to make manual edits to a single Application for debugging/testing purposes.

For example:

• Imagine that you have an ApplicationSet that created Applications app1, app2, and app3.
• You now want to edit app3 with kubectl edit application/app3, to update one of the app3‘s fields.
• However, as soon as you make edits to app3 (or any of the individual Applications), they will be immediately reverted by the ApplicationSet reconciler back to the template-ized version (by design).

As of this writing, there is an issue open for discussion of this behaviour.

Limitation: ApplicationSet controller will not selectively ignore changes to individual fields

Currently, you can only instruct the ApplicationSet controller to ignore changes to Application annotations.

For example, imagine that we have an Application created from an ApplicationSet, but a user has attempted to add a custom annotation (to the Application) that does not exist in the ApplicationSet resource:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  labels: 
    # This label exists only on this Application, and not in 
    # the parent ApplicationSet template:
    my-custom-label: some-value
spec:
  # (...)

As above, the ApplicationSet resource does not have a my-custom-label: some-value label in the .spec.template.labels for the Application.

Since this field is not in the ApplicationSet template, as soon as a user adds this custom annotation, it will be immediately reverted (removed) by the ApplicationSet controller.

There is currently no support for disabling or customizing this behaviour.

To some extent this is by design: the main principle of ApplicationSets is that we maintain a 1-to-many relationship between the ApplicationSet and the Applications that it owns, such that all the Applications necessarily conform to a strict template.

This provides the advantages of the ‘cattle not pets’ philosophy of microservice/cloud native application resource management, wherein you don’t need to worry about individual Applications differing from each other in subtle ways: they will all necessarily be reconciled to be consistent with the parent template.

BUT, support exists for preserving changes to Application annotations as documented above.