[RFC0038] IPv6 support for Silk CNI and VXLAN policy agent

[plamen-bardarov]

• Read the Contributing document.

Summary

RFC0038
Tracking Issue

Key things to note:

• There's a dependency on BOSH for exporting the ipv6 prefix for the cells so for now its hardcoded.
• Cloud Controller has to be extended as it currently doesn't support IPv6 ASGs
• There's a dependency on garden, bbs and executor, where a branch "icmpv6-support-asg" exists
• C2C support for IPv6 is not in the scope of the PR
• Some IPv6 specific tests cant be ran in the concourse pipeline so setting "GINKGO_TEST_IPV6=true" will trigger the execution of all tests. It requires IPv6 enabled docker.

Testing

In order to run the IPv6 tests locally "GINKGO_TEST_IPV6=true" has to be set inside the container. Also docker engine has to have IPv6 enabled with the following config:

{
  "fixed-cidr-v6": "2001:db8:2::/64",
  "ipv6": true
}

Code Changes

CNI Wrapper Plugin

Spec Changes

• Added host_tcp_services_ipv6, similar to host_tcp_services for ipv4, adds those networks as ACCEPT iptables rules
• Added deny_networks_ipv6, similar to deny_networks for ipv4
• Added ipv6.enable

Code Changes

• If IPv6 is enabled on the host and the host-local plugin assigns an IPv6 address to the container, NetOut and NetOutChain are created specifically for IPv6 and used to initialize default and egress rules.
• netrules.NetOut:
  • Changed default overlay rules for IPv6 to REJECT all instead of allowing marked packets.
  • Removed packet marking.
  • Implemented support for host TCP services and DNS validation.
• netrules.RuleConverter:
  • Added ICMPv6 protocol support.

---

Silk CNI (Plugin)

Code Changes

• If silk-daemon returns an IPv6 prefix, pass it to the IPAM host-local plugin.
• If an IPv6 address is allocated by the IPAM plugin, assign it to both host and container devices, configuring them similarly to IPv4.

---

Silk Daemon

Spec Changes

• Added ipv6_prefix
• Added ipv6.enable
• Added ipv6.prefix_network, the network name from spec.networks that holds the ipv6 prefix for containers

Note:
ipv6_prefix is not part of the spec, it is dynamically assigned value, taken from a network named ipv6.prefix_network

Code Changes

• The healthcheck endpoint returns the network lease (from the silk-controller) for the cell. If IPv6 is enabled(ipv6.enable=true) and the host supports it and ipv6_prefix is set in the config, return the configured prefix as well.

---

VXLAN Policy Agent

Spec Changes

• Added ipv6.enable. Disclaimer: if ipv6.enable=true but the cni-wrapper plugin is not configured to assign IPv6 addresses, the vxlan-policy-agent fails to setup ASG rules, because of the missing netout chains for IPv6

Code Changes

• If IPv6 is enabled in the config and the host supports it:
  1. netrules.NetOutChain, Enforcer, and planner.VxlanPolicyPlanner are created with IPv6 configurations.
  2. A new converger.SinglePollCycle is instantiated for IPv6 and added to PollCycleGroup, wrapping both IPv6 and IPv4 implementations.
• planner.VxlanPolicyPlanner:
  • Extended with an IPv6 flag.
  • Security group rules received from Policy Server are filtered based on IP version.

---

Backward Compatibility

Breaking Change? TBD

[linux-foundation-easycla]

• ✅ login: plamen-bardarov / name: Plamen Bardarov (0982b1f, cccc910, 0c620b4, 1628e5f, 83a7dd1, 87d4686, 5cb33ba, 13cdc45, 08292fc, bb7b485, 1a18487, 8f4540a, b699fea, 7ac993f, 2a1322b, 06c1e5b, ec7a013, 0ceeb54, b92af56, c9d1e05, d162a5b, c25e494, c31f317, 181719b, 0dfbc3b, 30d02d7, 9ae283d, 4e9f269, 37ea508, 24d61f0, 93f0cbb)
• ❌ - login: @dimitardimitrov13 / name: Dimitar Dimitrov . The commit (2ebbcde, 2bc122a, 65dd0b9, e222b5b, ddf7545) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

[plamen-bardarov]

Ginkgo test results(tl;dr all green): https://gist.github.com/plamen-bardarov/b4d40c27383be0269b39f191864b6b13

[winkingturtle-vmw]

@plamen-bardarov Can you please resolve the conflict so that we can start the process for merging ?

[winkingturtle-vmw]

Changes looks good to me as far keeping the backward compatibility and keeping ipv6 behind a flag. Since there is a lot of code changes here, I've asked for a second pair of eyes to make sure I didn't miss anything.

[winkingturtle-vmw]

@ameowlia This looks to good to me now. Is there any other changes you are requesting ? Ideally I would to squash all of these commits into one for a better visibility of the changes.

[plamen-bardarov]

cf-acceptance-tests results: https://gist.github.com/plamen-bardarov/49852860f721b99dd6f928754ee32e02
Ran with the following config:

{
    "admin_user": "admin",
    "api": "api.api",
    "apps_domain": "apps.apps",
    "artifacts_directory": "logs",
    "include_apps": true,
    "include_container_networking": true,
    "include_detect": true,
    "include_deployments": true,
    "include_docker": true,
    "include_http2_routing": false,
    "include_internet_dependent": true,
    "include_routing_isolation_segments": false,
    "include_isolation_segments": false,
    "include_route_services": true,
    "include_routing": true,
    "include_security_groups": true,
    "include_services": true,
    "include_service_discovery": true,
    "include_service_instance_sharing": true,
    "include_ssh": true,
    "include_sso": true,
    "include_tasks": true,
    "include_tcp_isolation_segments": false,
    "include_tcp_routing": false,
    "include_user_provided_services": true,
    "include_v3": true,
    "include_volume_services": false,
    "include_zipkin": true,
    "skip_ssl_validation": true,
    "stacks": ["cflinuxfs4"],
    "timeout_scale": 10,
    "use_http": false,
    "include_app_syslog_tcp": false
}

[peanball]

@winkingturtle-vmw, @ameowlia just as quick reminder: I think we've now provided everything that is needed to consider merging this. It is still considered experimental, but having a release will significantly help us in testing this with real world scenarios.

[n-sandalski]

Hi @ameowlia, the CATs have been executed successfully, and the changes don't affect the IPv4 functionality.

Is there anything further we can do on our end, or may we proceed with the merge?