Bug: Seclist resource does not get deleted with VCN resource count is changed

[sanketdjain]

Terraform Version

Terraform v0.11.1

provider.null v1.0.0
provider.oci v2.0.5
provider.template v1.0.0

OCI Provider Version

oci_v2.0.5 
2018/01/05 09:35:28 [INFO] terraform-provider-oci 2.0.5

Description:

I have a virtual network resource with a conditional count (see test case below).

1. Create the virtual network resource and the subnets and seclists
2. Set count of virtual network resource to zero, this will automatically delete the VCN resource and subnet resource, but it does not delete the seclist resource.

Test case

networking.tf

locals {
  create_vcn   = "${var.vcn_id == "" ? 1 : 0}"
  subnet_count = "${local.create_vcn == 1 ? length(data.oci_identity_availability_domains.ADs.availability_domains) : 0}"
}

resource "oci_core_virtual_network" "vcn1" {
    count = "${local.create_vcn}"
    cidr_block = "${var.cidr_block}"
    compartment_id = "${oci_identity_compartment.comp_network.id}"
    display_name = "VCN_${replace(var.comp_name, "COMP_", "")}"
    dns_label    = "vcn${format("%0.5s", replace(var.comp_name, "COMP_", ""))}"
}

....
....

resource "oci_core_subnet" "sub_app" {
  count = "${local.subnet_count}"
  availability_domain = "${lookup(data.oci_identity_availability_domains.ADs.availability_domains[count.index], "name")}"
  cidr_block = "${cidrsubnet(var.cidr_block, 6, count.index + var.app_block)}"
  display_name = "${format("%s%s","sub_app", "${count.index}")}"
  dns_label = "${format("%s%s","subapp", "${count.index}")}"
  compartment_id = "${oci_core_virtual_network.vcn1.compartment_id}"
  vcn_id = "${oci_core_virtual_network.vcn1.id}"
  security_list_ids = ["${oci_core_security_list.sec_app.id}"]
  route_table_id = "${oci_core_default_route_table.rt_service.id}"
  dhcp_options_id = "${oci_core_virtual_network.vcn1.default_dhcp_options_id}"
  #prohibit_public_ip_on_vnic = true

  provisioner "local-exec" {
       command = "sleep 5"
  }
}

seclist.tf

/* app seclist */
resource "oci_core_security_list" "sec_app" {
  compartment_id = "${oci_core_virtual_network.vcn1.compartment_id}"
  vcn_id = "${oci_core_virtual_network.vcn1.id}"
  display_name = "sec_app"

  /* allow outbound tcp traffic on all ports */
  egress_security_rules {
    destination = "0.0.0.0/0"
    protocol = "6" // tcp
  }

  /* allow inbound ssh traffic */
  ingress_security_rules {
    protocol = "6" // tcp
    source = "0.0.0.0/0"
    stateless = false

    tcp_options {
      "min" = 22
      "max" = 22
    }
  }
}

variables.tf

variable "vcn_id" {
  default = ""
  description = "ID of existing VCN"
}

variable "cidr_block" {
  default = "10.0.0.0/16"
  description = "CIDR Block for VCN"
}

Terraform Plan

Terraform Plan is showing some sensitive information in shell scripts of templates, so I have included the test case above.

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy
 <= read (data resources)

Terraform will perform the following actions:

 <= data.oci_core_images.img
      id:             <computed>
      compartment_id: "ocid1.compartment.oc1..aaaaaaaanzpexwmcbowil3ghu4p6rljx7kcxpklkktka5bdy4o5fvt2i5rva"
      display_name:   "serv-image"
      images.#:       <computed>

  - oci_core_subnet.sub_app

  - oci_core_subnet.sub_app[1]

  - oci_core_subnet.sub_app[2]

  - oci_core_virtual_network.vcn1

[alexng-canuck]

@sanketdjain, this appears to be by design because the security list and virtual network exist independently of each other and your count was only applied to the virtual network resource.

It's also interesting that you didn't get an error after setting the virtual network resource count to zero. This should have resulted in an error because the sec list has a dependency on the virtual network.

I tried an internal repro with the similar setup as yours. After setting the virtual network count to zero, I see an error like this:
* oci_core_virtual_network.vcn1: Status: 409; Code: IncorrectState; OPC Request ID: ...; Message: ocid1.vcn.oc1.phx..... is associated with security list that is in use

[sanketdjain]

@alexng-canuck , I did not understand how the sec list can exist independently without the virtual network. Sec list requires the VCN ID. On the OCI website, sec list is only visible in the Virtual Network section. So it should be treated as implicit dependent of virtual network resource. I think sec list should be treated similar to subnets.

[alexng-canuck]

@sanketdjain When I said that the security list exists independently, I meant that it's currently designed so that the security list can change to be associated to a different VCN if need be.

Similarly a subnet exists independently from the VCN and doesn't get deleted if the VCN is removed. A possible reason why you see it being destroyed, is because your config has the count parameter applied to the subnet as well.

That being said, when you destroy a VCN in the console UI, it will also destroy the associated subnets, sec lists, etc. So we should consider mimicking this behavior in Terraform as well. I will log an internal issue to track this.

In the meantime, there's a concept of sec list, route table, and dhcp options that are intrinsically tied to the VCN and are automatically removed when the VCN is destroyed. These are known as default resources and the way to configure them is described here: https://github.com/oracle/terraform-provider-oci/blob/master/docs/Managing%20Default%20Resources.md

[alexng-canuck]

One additional comment: This may be difficult to support in Terraform because of the semantics in your config files. When you set VCN's count to 0, Terraform interprets this as "destroy the VCN but to keep one sec list alive" (since sec list has no count, Terraform will assume count is 1).

If you want Terraform to delete the sec list, you would have to set the count zero there as well to avoid any confusion.

[alexng-canuck]

Closing the enhancement as won't fix.

Supporting destruction semantics for resources that are associated with the VCN puts Terraform in a position of destroying resources that it has no knowledge about from the statefile or config.

This violates the principle of explicitly defining desired state in your configs.

@sanketdjain let us know if you are still having issues with your configs.

[sanketdjain]

@alexng-canuck I don't think this issue should be closed.

Terraform has implicit and explicit dependencies. In OCI Console, the seclists appear under a VCN. When we delete a VCN manually, it automatically destroys all the subnets and security lists. This gives an impression that the VCN does include implicit dependencies on these sub-components.

However, in Terraform, when destroying a VCN, it does not automatically detect implicit dependency on the security list. Note, that the subnets are being destroyed, so the argument of explicit dependency is not applicable. The "depends_on" option in Terraform only controls the order of creating/updating/destroying resources.

For test/dev purpose, we create a VCN and destroy it frequently. There are 10s of security lists in the VCN. If we have to explicitly code up "-destroy -target" for each security list, then there it gets very cumbersome to use Terraform.

The explanation given earlier was that the current design allows a seclist can be associated with different VCN (if required). But this is not currently the case in OCI Console. The user experience should be same across different SDKs of OCI.

[alexng-canuck]

Hi @sanketdjain

We should make a distinction between the console's expected behavior and Terraform's expected behavior; as they are different use cases with different guiding principles.

Terraform is a declarative language, which means that you need to be explicit about all the resources that you are managing in your configuration. Your configuration should always reflect how you want your infrastructure to look like.

That being said, it looks like terraform destroy -target already supports the destroy semantics that you're looking for. It deletes resources that depend on the VCN when you target the VCN (provided those resources are part of your config)

For example, if I have a plan that looks like this:

resource "oci_core_virtual_network" "ExampleVCN" {
  cidr_block     = "10.1.0.0/16"
  compartment_id = "${var.compartment_ocid}"
  display_name   = "TFExampleVCN"
  dns_label      = "tfexamplevcn"
}

resource "oci_core_subnet" "ExampleSubnet" {
  compartment_id      = "${var.compartment_ocid}"
  vcn_id              = "${oci_core_virtual_network.ExampleVCN.id}"
  ...
}

resource "oci_core_security_list" "ExampleSecurityList" {
  compartment_id = "${var.compartment_ocid}"
  vcn_id         = "${oci_core_virtual_network.ExampleVCN.id}"
  ...
}

Running terraform destroy -target oci_core_virtual_network.ExampleVCN will result in a plan like this:

Terraform will perform the following actions:

  - oci_core_security_list.ExampleSecurityList

  - oci_core_subnet.ExampleSubnet

  - oci_core_virtual_network.ExampleVCN


Plan: 0 to add, 0 to change, 3 to destroy.

Is this what you were looking for?

[alexng-canuck]

Closing this issue, since we've provided a way for VCN and dependent resources to be destroyed as part of target destroy.