Skip to content

Failed to recover VLAN when deleting L2VPN after controller restarts #495

Closed

Description

@italovalcy

Related to #485 now deleting a L2VPN works fine after controller restarts, however the VLANs previous allocated to the L2VPN are not released. If the controller does not restarts, it works fine. However if you restart the controller, then Delete the L2VPN, vlans are not recovered.

How to reproduce:

  1. Run the Mininet deployment as described here: https://sdx-docs.readthedocs.io/en/latest/network-use-cases.html#pre-requirements
  2. Check the VLAN range available on all ports (all ports should have the available vlans range from 1 to 4094):
$ ./scripts/show-sdx-controller.sh ports
ID                                   STATUS  STATE    L2VPN-VLAN-RANGE
--                                   ------  -----    ----------------
urn:sdx:port:ampath.net:Ampath1:1    up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath1:40   up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath1:2    up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath1:50   up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath2:1    up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath2:50   up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath2:40   up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath2:3    up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath3:2    up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath3:50   up      enabled  [[1,4094]]
urn:sdx:port:ampath.net:Ampath3:3    up      enabled  [[1,4094]]
urn:sdx:port:sax.net:Sax01:41        up      enabled  [[1,4094]]
urn:sdx:port:sax.net:Sax01:1         up      enabled  [[1,4094]]
urn:sdx:port:sax.net:Sax01:40        up      enabled  [[1,4094]]
urn:sdx:port:sax.net:Sax01:50        up      enabled  [[1,4094]]
urn:sdx:port:sax.net:Sax02:41        up      enabled  [[1,4094]]
urn:sdx:port:sax.net:Sax02:1         up      enabled  [[1,4094]]
urn:sdx:port:sax.net:Sax02:40        up      enabled  [[1,4094]]
urn:sdx:port:sax.net:Sax02:50        up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet01:41  up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet01:1   up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet01:2   up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet01:50  up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet02:41  up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet02:1   up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet02:50  up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet03:50  up      enabled  [[1,4094]]
urn:sdx:port:tenet.ac.za:Tenet03:2   up      enabled  [[1,4094]]
  1. Create two L2VPNs between Ampath3-eth50 and Tenet03-eth50:
$ docker compose exec -it mininet curl -s -X POST -H 'Content-type: application/json' http://sdx-controller:8080/SDX-Controller/l2vpn/1.0 -d '{"name": "L2VPN-Ampath3-Tenet03-vlan-4001", "endpoints": [{"port_id": "urn:sdx:port:ampath.net:Ampath3:50", "vlan": "4001"}, {"port_id": "urn:sdx:port:tenet.ac.za:Tenet01:50", "vlan": "4001"}]}'
{
  "reason": "Connection published",
  "service_id": "c4d87617-b118-4bb4-99f4-6ba9e13b5d56",
  "status": "under provisioning"
}
$ docker compose exec -it mininet curl -s -X POST -H 'Content-type: application/json' http://sdx-controller:8080/SDX-Controller/l2vpn/1.0 -d '{"name": "L2VPN-Ampath3-Tenet03-vlan-3001", "endpoints": [{"port_id": "urn:sdx:port:ampath.net:Ampath3:50", "vlan": "3001"}, {"port_id": "urn:sdx:port:tenet.ac.za:Tenet01:50", "vlan": "3001"}]}'
{
  "reason": "Connection published",
  "service_id": "d5e5c940-46e5-48cd-baef-db1cb341e85d",
  "status": "under provisioning"
}
  1. Check if L2VPNs are UP and ports:
$ ./scripts/show-sdx-controller.sh l2vpn
ID                                    STATUS  ENDPOINT-1                          VLAN-1  ENDPOINT-2                           VLAN-2
--                                    ------  ----------                          ------  ----------                           ------
c4d87617-b118-4bb4-99f4-6ba9e13b5d56  up      urn:sdx:port:ampath.net:Ampath3:50  4001    urn:sdx:port:tenet.ac.za:Tenet01:50  4001
d5e5c940-46e5-48cd-baef-db1cb341e85d  up      urn:sdx:port:ampath.net:Ampath3:50  3001    urn:sdx:port:tenet.ac.za:Tenet01:50  3001
$ ./scripts/show-sdx-controller.sh ports
ID                                   STATUS  STATE    L2VPN-VLAN-RANGE
--                                   ------  -----    ----------------
urn:sdx:port:ampath.net:Ampath1:40   up      enabled  ["3-4094"]
urn:sdx:port:ampath.net:Ampath3:50   up      enabled  ["1-3000","3002-4000","4002-4094"]
urn:sdx:port:sax.net:Sax01:41        up      enabled  ["3-4094"]
urn:sdx:port:sax.net:Sax01:40        up      enabled  ["3-4094"]
urn:sdx:port:tenet.ac.za:Tenet01:41  up      enabled  ["3-4094"]
urn:sdx:port:tenet.ac.za:Tenet01:50  up      enabled  ["1-3000","3002-4000","4002-4094"]
  1. Now restart the controller:
docker compose up -d sdx-controller --force-recreate
  1. Delete the L2VPNs:
$ for ID in $(docker compose exec -it mininet curl -s http://sdx-controller:8080/SDX-Controller/l2vpn/1.0 | jq -r '.[].service_id'); do docker compose exec -it mininet curl -s -X DELETE http://sdx-controller:8080/SDX-Controller/l2vpn/1.0/$ID; done
"OK"
"OK"
$ ./scripts/show-sdx-controller.sh l2vpn
ID  STATUS  ENDPOINT-1  VLAN-1  ENDPOINT-2  VLAN-2
--  ------  ----------  ------  ----------  ------
$

All good so far.. however..

  1. Check the VLANs on the ports:
$ ./scripts/show-sdx-controller.sh ports
ID                                   STATUS  STATE    L2VPN-VLAN-RANGE
--                                   ------  -----    ----------------
urn:sdx:port:ampath.net:Ampath1:1    up      enabled  ["1-4094"]
urn:sdx:port:ampath.net:Ampath1:40   up      enabled  ["3-4094"]
urn:sdx:port:ampath.net:Ampath1:2    up      enabled  ["1-4094"]
urn:sdx:port:ampath.net:Ampath1:50   up      enabled  ["1-4094"]
urn:sdx:port:ampath.net:Ampath2:1    up      enabled  ["1-4094"]
urn:sdx:port:ampath.net:Ampath2:50   up      enabled  ["1-4094"]
urn:sdx:port:ampath.net:Ampath2:40   up      enabled  ["1-4094"]
urn:sdx:port:ampath.net:Ampath2:3    up      enabled  ["1-4094"]
urn:sdx:port:ampath.net:Ampath3:2    up      enabled  ["1-4094"]
urn:sdx:port:ampath.net:Ampath3:50   up      enabled  ["1-3000","3002-4000","4002-4094"]
urn:sdx:port:ampath.net:Ampath3:3    up      enabled  ["1-4094"]
urn:sdx:port:sax.net:Sax01:41        up      enabled  ["3-4094"]
urn:sdx:port:sax.net:Sax01:1         up      enabled  ["1-4094"]
urn:sdx:port:sax.net:Sax01:40        up      enabled  ["3-4094"]
urn:sdx:port:sax.net:Sax01:50        up      enabled  ["1-4094"]
urn:sdx:port:sax.net:Sax02:41        up      enabled  ["1-4094"]
urn:sdx:port:sax.net:Sax02:1         up      enabled  ["1-4094"]
urn:sdx:port:sax.net:Sax02:40        up      enabled  ["1-4094"]
urn:sdx:port:sax.net:Sax02:50        up      enabled  ["1-4094"]
urn:sdx:port:tenet.ac.za:Tenet01:41  up      enabled  ["3-4094"]
urn:sdx:port:tenet.ac.za:Tenet01:1   up      enabled  ["1-4094"]
urn:sdx:port:tenet.ac.za:Tenet01:2   up      enabled  ["1-4094"]
urn:sdx:port:tenet.ac.za:Tenet01:50  up      enabled  ["1-3000","3002-4000","4002-4094"]
urn:sdx:port:tenet.ac.za:Tenet02:41  up      enabled  ["1-4094"]
urn:sdx:port:tenet.ac.za:Tenet02:1   up      enabled  ["1-4094"]
urn:sdx:port:tenet.ac.za:Tenet02:50  up      enabled  ["1-4094"]
urn:sdx:port:tenet.ac.za:Tenet03:50  up      enabled  ["1-4094"]
urn:sdx:port:tenet.ac.za:Tenet03:2   up      enabled  ["1-4094"]

Checking the logs we see this error:

sdx-controller-1  | 2025-09-09T20:28:02.944827051Z INFO:sdx_controller.controllers.l2vpn_controller:Removing connection: c4d87617-b118-4bb4-99f4-6ba9e13b5d56 DELETED
sdx-controller-1  | 2025-09-09T20:28:02.952107682Z DEBUG:sdx_controller.handlers.connection_handler:Error when removing breakdown: Unknown connection request (ID: c4d87617-b118-4bb4-99f4-6ba9e13b5d56)
sdx-controller-1  | 2025-09-09T20:28:02.954330138Z DEBUG:connexion.apis.abstract:Getting data and status code
...
sdx-controller-1  | 2025-09-09T20:28:03.150606062Z INFO:sdx_controller.controllers.l2vpn_controller:Removing connection: d5e5c940-46e5-48cd-baef-db1cb341e85d DELETED
sdx-controller-1  | 2025-09-09T20:28:03.156161529Z DEBUG:sdx_controller.handlers.connection_handler:Error when removing breakdown: Unknown connection request (ID: d5e5c940-46e5-48cd-baef-db1cb341e85d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions