Automate deprovisioning of environments

[gnarea]

I didn't do it as part of the provisioning because deprovisioning is a risky endeavour and has to be considered properly.

I'm inclined to solve this in two steps:

• Prepare the Terraform-managed cloud resources for removal. I.e.:
  • Disable prevent_destroy and resource-specific equivalents.
  • Disable the GCB trigger for k8s deployments.
  • Create (or enable) a new GCB trigger to be run before terraform destroy (see next step).
• Run a GCB trigger to:
  • Deprovision k8s resources, including cloud resources managed by k8s (e.g., the LB).
  • Empty GCS buckets by creating a lifecycle rule that sets their objects' age to 0 days. This is much quicker and cheaper than deleting objects, especially in versioned buckets.

That way, deprovisioning an environment will involve the following steps:

1. Turn on the flag to allow destruction of the environment in Terraform.
2. Run the deprovisioning GCB trigger.
3. Delete the Terraform-managed resources.