Provider crashes when trying to configure or read VRF-lite peers on VPC-type Border Gateways

[richwats]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v1.4.5
on darwin_arm64

DCNM version

• 12.1(3b)

Affected Resource(s)

dcnm_vrf

Terraform Configuration Files

See here..
https://github.com/cisco-apjc-cloud-se/ndfc-automation-lab-v2

Debug Output

Strangely this bug only affects VRF-lite peers created on VPC-type border gateways (as opposed to Anycast-type border gateways).

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Plugin did not respond
│ 
│   with dcnm_vrf.vrf-1,
│   on 2-vrfs.tf line 2, in resource "dcnm_vrf" "vrf-1":
│    2: resource "dcnm_vrf" "vrf-1" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with dcnm_vrf.vrf-2,
│   on 2-vrfs.tf line 102, in resource "dcnm_vrf" "vrf-2":
│  102: resource "dcnm_vrf" "vrf-2" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more details.
╵

Stack trace from the terraform-provider-dcnm plugin:

panic: set item just set doesn't exist

goroutine 186 [running]:
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setSet(0x140005058a8, {0x14000410900, 0x3, 0x3}, {0x104ceb100, 0x1400071aeb8}, 0x140002cfa40)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:328 +0x7c8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0x140005058a8, {0x14000410900, 0x3, 0x3}, {0x104ceb100, 0x1400071aeb8})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:107 +0x120
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setObject(0x14000696da0?, {0x14000696da0?, 0x2, 0x2}, {0x104d2ae80, 0x140003cc9c0}, 0x20?)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:225 +0x238
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0x140005058a8, {0x14000696da0, 0x2, 0x2}, {0x104d2ae80, 0x140003cc9c0})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:109 +0xf8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList.func1(...)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:123
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList(0x140005058a8, {0x1400049b0c0?, 0x1, 0x1}, {0x104ceb100, 0x14000505890}, 0x1400060f2c0?)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:143 +0x258
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0x140005058a8, {0x1400049b0c0, 0x1, 0x1}, {0x104ceb100, 0x14000505890})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:103 +0x90
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).WriteField(0x140005058a8, {0x1400049b0c0, 0x1, 0x1}, {0x104ceb100, 0x14000505890})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:89 +0x38c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setSet(0x140005049d8, {0x1400049b0c0, 0x1, 0x1}, {0x104ceb100, 0x14000505890}, 0x140002cfb80)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:307 +0x2c8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0x140005049d8, {0x1400049b0c0, 0x1, 0x1}, {0x104ceb100, 0x14000505890})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:107 +0x120
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).WriteField(0x140005049d8, {0x1400049b0c0, 0x1, 0x1}, {0x104ceb100, 0x14000505890})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/field_writer_map.go:89 +0x38c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*ResourceData).Set(0x1400015be80, {0x104b91e3a, 0xb}, {0x104ceb100, 0x14000505890})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/resource_data.go:227 +0x200
github.com/CiscoDevNet/terraform-provider-dcnm/dcnm.resourceDCNMVRFRead(0x1400015be80, {0x104dcff80?, 0x14000292800})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/dcnm/resource_dcnm_vrf.go:1317 +0xaa4
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x104b8d400?, {0x104df61a0?, 0x140005064b0?}, 0x24?, {0x104dcff80?, 0x14000292800?})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/resource.go:290 +0x58
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0x140000da840, {0x104df61a0, 0x140005064b0}, 0x140001462a0, {0x104dcff80, 0x14000292800})
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/resource.go:564 +0x30c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0x1400000d1a0, {0x104df61a0, 0x140005064b0}, 0x140004c47c0)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/grpc_provider.go:575 +0x380
github.com/hashicorp/terraform-plugin-go/tfprotov5/server.(*server).ReadResource(0x14000081820, {0x104df6248?, 0x140008041e0?}, 0x140001645a0)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/server/server.go:298 +0x240
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x104dc0f40?, 0x14000081820}, {0x104df6248, 0x140008041e0}, 0x14000164540, 0x0)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:344 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0x14000280fc0, {0x104df9a78, 0x14000580000}, 0x14000126100, 0x14000390690, 0x105234550, 0x0)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/google.golang.org/grpc/server.go:1194 +0xb00
google.golang.org/grpc.(*Server).handleStream(0x14000280fc0, {0x104df9a78, 0x14000580000}, 0x14000126100, 0x0)
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/google.golang.org/grpc/server.go:1517 +0x82c
google.golang.org/grpc.(*Server).serveStreams.func1.2()
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/google.golang.org/grpc/server.go:859 +0x84
created by google.golang.org/grpc.(*Server).serveStreams.func1
        /Users/richwats/go/src/github.com/ciscoecosystem/terraform-provider-dcnm/vendor/google.golang.org/grpc/server.go:857 +0x28c

Error: The terraform-provider-dcnm plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Expected Behavior

That VRF-lite peering configuration is applied to the VRF attachments for both Anycast and VPC based Border / Border Gateway role switches.

Actual Behavior

If VRF-lite peers are attached to vPC role Border / BGW switches, the provider crashes above.

Steps to Reproduce

Use the following resource, uncomment the last two VRF-lite sections, then apply

resource "dcnm_vrf" "vrf-2" {
  fabric_name             = var.dcnm_fabric
  name                    = "TERRAFORM-VRF2"
  vlan_id                 = 112
  segment_id              = 50112
  vlan_name               = "TERRAFORM-VRF2"
  description             = "VRF #2 built by Terraform"
  intf_description        = "TERRAFORM-VRF2"
  # max_bgp_path           = 2
  # max_ibgp_path          = 2
  advertise_host_route    = true
  advertise_default_route = true
  static_default_route    = false
  deploy                  = true

  ### Attach VRF to DC1 Compute Leaves ###
  attachments {
    serial_number = lookup(local.serial_numbers, "DC1-LEAF3")
    vlan_id       = 112
    attach        = true
  }

  attachments {
    serial_number = lookup(local.serial_numbers, "DC1-LEAF4")
    vlan_id       = 112
    attach        = true
  }

  ### Attach VRF to DC1 Anycast Border Gateways ###
  attachments {
    serial_number = lookup(local.serial_numbers, "DC1-ABGW1")
    vlan_id       = 112
    attach        = true
    vrf_lite {
      peer_vrf_name = "global"
      interface_name = "Ethernet1/6"
      dot1q_id = 12
      neighbor_ip = "192.168.253.254"
      neighbor_asn = 65100
      ip_mask = "192.168.253.253/30"
    }
  }

  attachments {
    serial_number = lookup(local.serial_numbers, "DC1-ABGW2")
    vlan_id       = 112
    attach        = true
    vrf_lite {
      peer_vrf_name = "global"
      interface_name = "Ethernet1/6"
      dot1q_id = 12
      neighbor_ip = "192.168.253.250"
      neighbor_asn = 65100
      ip_mask = "192.168.253.249/30"
    }
  }

  ### Attach VRF to DC2 vPC Border Gateways ###
  attachments {
    serial_number = lookup(local.serial_numbers, "DC2-VBGW1")
    vlan_id = 112
    attach = true
    # vrf_lite {
    #   peer_vrf_name = "global"
    #   interface_name = "Ethernet1/6"
    #   dot1q_id = 12
    #   neighbor_ip = "192.168.253.246"
    #   neighbor_asn = 65100
    #   ip_mask = "192.168.253.245/30"
    # }
  }

  attachments {
    serial_number = lookup(local.serial_numbers, "DC2-VBGW2")
    vlan_id = 112
    attach = true
    # vrf_lite {
    #   peer_vrf_name = "global"
    #   interface_name = "Ethernet1/6"
    #   dot1q_id = 12
    #   neighbor_ip = "192.168.253.242"
    #   neighbor_asn = 65100
    #   ip_mask = "192.168.253.241/30"
    # }
  }

}