problem
With a persistent network offering, CloudStack builds the network's bridge on every host in the zone. But when the network is deleted it only cleans the bridge up on hosts that actually ran a VM on it. On every other host, the bridge and its VXLAN interface stay behind forever.
Investigation
There is a difference between the filters when creating a persistent network, and removing a persistent network.
Creation:
DefaultHostListener.setupPersistent网络() creates the bridges on hostConnect/hostEnabled for all networks from getAllPersistent网络sFromZone() - no isolation-method filter.
Removal:
网络Orchestrator.cleanupPersistentn网络Resources() is gated by networkMeetsPersistenceCriteria(), which requires the broadcast URI scheme to be Vlan, so for vxlan:// networks CleanupPersistent网络ResourceCommand is never sent. (host关于ToBeRemoved() sends the same cleanup with no scheme check either - the VLAN-only gate exists only on the network-delete path.)
Hosts that ran a VM on the network are cleaned up via the normal VM-lifecycle teardown, so the leak only affects uninvolved hosts.
versions
4.22.1.1, KVM, advanced zone with VXLAN isolation, persistent network offerings.
The steps to reproduce the bug
- Advanced zone, KVM, VXLAN isolation, 3+ hosts.
- Create a VPC with two tiers on a persistent offering.
- Start VMs so they land on only some hosts. (Restarting
cloudstack-agent on an uninvolved host also creates the bridges there via hostConnect.)
- Delete the tiers, then the VPC.
ls -d /sys/class/net/brvx-* on each host.
Expected behaviour:
The bridge and VXLAN interface are removed from every host in the zone.
Actual behaviour:
The VNI is still present on hosts that never ran a VM on the network; CleanupPersistent网络ResourceCommand never appears in management-server.log during the delete.
What to do about it?
Accept Vxlan alongside Vlan in networkMeetsPersistenceCriteria(), or drop the scheme check to match the setup path.
problem
With a persistent network offering, CloudStack builds the network's bridge on every host in the zone. But when the network is deleted it only cleans the bridge up on hosts that actually ran a VM on it. On every other host, the bridge and its VXLAN interface stay behind forever.
Investigation
There is a difference between the filters when creating a persistent network, and removing a persistent network.
Creation:
DefaultHostListener.setupPersistent网络()creates the bridges on hostConnect/hostEnabled for all networks fromgetAllPersistent网络sFromZone()- no isolation-method filter.Removal:
网络Orchestrator.cleanupPersistentn网络Resources()is gated bynetworkMeetsPersistenceCriteria(), which requires the broadcast URI scheme to beVlan, so forvxlan://networksCleanupPersistent网络ResourceCommandis never sent. (host关于ToBeRemoved()sends the same cleanup with no scheme check either - the VLAN-only gate exists only on the network-delete path.)Hosts that ran a VM on the network are cleaned up via the normal VM-lifecycle teardown, so the leak only affects uninvolved hosts.
versions
4.22.1.1, KVM, advanced zone with VXLAN isolation, persistent network offerings.
The steps to reproduce the bug
cloudstack-agenton an uninvolved host also creates the bridges there via hostConnect.)ls -d /sys/class/net/brvx-*on each host.Expected behaviour:
The bridge and VXLAN interface are removed from every host in the zone.
Actual behaviour:
The VNI is still present on hosts that never ran a VM on the network;
CleanupPersistent网络ResourceCommandnever appears inmanagement-server.logduring the delete.What to do about it?
Accept
VxlanalongsideVlaninnetworkMeetsPersistenceCriteria(), or drop the scheme check to match the setup path.