Implementation principle of MPLS virtual private network cross-domain solution

Overview:

As MPLS technology matures, its applications are becoming more and more popular, especially in VPN. Through the VPN service provided by the operator, the sites distributed in various places are connected through the operator's network, avoiding leased lines and saving a lot of costs. In recent years, due to the rapid development of the MPLS VPN business, the number of corporate sites has also been increasing. Mergers and acquisitions and integration often occur among companies, making their different branch sites may belong to different operators. Therefore, for large telecom operators, the current problem that needs to be solved is how to deploy a cross-domain MPLS L3 VPN that is easy to expand and easy to maintain.

Cross-domain VPN has the following two most common forms. One situation is that for a large telecom operator’s network, an AS is generally allocated to a province.

Seeking to provide customers with MPLS VPN services across provinces; or within the scope of a provincial network, a reserved AS is often assigned to the metropolitan area network of each city, requiring the provision of MPLS VPN services across cities. Another situation is that the VPN customer network traverses multiple different operator networks, and the operators cooperate with each other (especially the cooperation with foreign operators in terms of international business).

The normal MPLS VPN architecture runs in an AS. Any VPN routing information can only be diffused in one AS on demand, and it does not provide the function of diffusing VPN information in the AS to other ASs. As shown in the figure below, in order to support the exchange of VPN routing information between different ASs of operators, it is necessary to extend the existing protocol and modify the MPLS VPN system framework to provide an interconnection model that is different from the basic MPLS VPN system structure. Domain (Inter-AS) MPLS VPN, so that it can advertise routing prefix and label information through the link between operators.

Method to realize:

At present, there are three interworking methods for the mainstream cross-domain MPLS VPN. This solution is defined by RFC4364. They are as follows:

1. Cross-domain VPN-OptionA (Inter-Provider Backbones Option A) method: The cross-domain VPN needs to manage its own VPN routing between ASBR (AS Boundary Router) through a dedicated interface, also known as VRF-to-VRF.
2. Cross-domain VPN-Option B (Inter-Provider Backbones Option B) method: The ASBR advertises labeled VPN-IPv4 routes through MP-eBGP, also known as eBGP redistribution of labeled VPN-IPv4 routes.
3. Cross-domain VPN-OptionC (Inter-Provider Backbones Option C) method: PEs publish labeled VPN-IPv4 routes through Multi-hopMP-eBGP, also known as Multihop eBGP redistribution of labeledVPN-IPv4 routeso

In different situations, these solutions have different advantages and disadvantages. In the actual network environment, Option B and Option C also have different variants. Only their conventional implementation solutions are introduced here.

OptionA：

Overview: It is

necessary to establish an MP-INGP peer relationship between PE and ASBR, create a VPN instance for each VPN on the ASBR in each AS domain (to achieve isolation between different VPN routes), and connect to a physical Interfaces or sub-interfaces are bound, and then common IGP or BGP routing methods (eBGP is preferred) are used to advertise common unicast IPv4 routes to the opposite end to achieve Layer 3 interconnection between ASBRs.

Principles of route publishing:

We only explain the working process of the control plane in a single direction. At the same time, assume that there is a VPN route Client1 connection at Site1. As shown in the figure above, we now need to pass the Client1 route from CE1 through AS100 and AS200 to CE2:

1. In AS100, by running the LDP protocol, PE1 assigns a tunnel label (outer label) T1 associated with the route to PE1 to P1.
2. In AS100, by running the LDP protocol, P1 assigns a tunnel label (outer label) T2 associated with the route to PE1 to ASBR-PE1.
3. In AS200, also by running the LDP protocol, ASBR-PE2 assigns a tunnel label (outer label) T3 associated with the route to ASBR-PE2 to P2.
4. In AS200, by running the LDP protocol, P2 assigns a tunnel label (outer label) T4 associated with the route to ASBR-PE2 to PE2.
5. CE1 advertises the route Client1 to PE1, and the next hop of the route is the interface address of CE1.
6. PE1 re-advertises the IPv4 route Client1 as a VPNv4 route through MP-BGP, and changes the next hop to PE1, assigns a VPN label V1, and then advertises it to ASBR-PE1.
7. ASBR-PE1 turns the VPNv4 route into an IPv4 route, advertises the IPv4 route Client1 to ASBR-PE2, and the next hop points to ASBR-PE1.
8. ASBR-PE2 re-advertises the IPv4 route Client1 through MP-BGP as a VPNv4 route, and the next hop is ASBR-PE2, assigns a VPN label V2 to the route, and advertises it to PE2.
9. PE2 converts the VPNv4 route into an IPv4 route Client1, advertises the route Client1 to CE2, and the next hop points to PE2.

Packet forwarding principle:

Through the analysis of the previous article, we analyze the working process of the forwarding plane from the reverse direction, that is, CE2 needs to send an IP packet with a destination of Client1 to CE1, as shown in the figure above:

1. CE2 sends an IP packet with a destination of Client1 to PE2.
2. PE2 encapsulates the MPLS label after receiving the IP packet, first encapsulates the VPN label V2, then the outer label T4, and then sends the packet to P2.
3. P2 performs label switching, changes the outer label T4 to T3, and then sends this message to ASBR-PE2.
4. ASBR-PE2 removes all labels, and forwards the message (ordinary IP message) to ASBR-PE1.
5. After receiving the IP packet, ASBR-PE1 encapsulates the MPLS label, first encapsulates the VPN label V1, then encapsulates the outer label T2, and then sends the packet to P1.
6. P1 performs label switching, changes the outer label T2 to T1, and then sends this message to PE1.
7. After receiving it, PE1 removes all labels, and forwards the message (normal IP message) to CE1.

Features of cross-domain VPN-
OptionA : Advantages: Simple configuration, because there is no need to run MPLS between ASBRs, and no special configuration for cross-domain.
Disadvantages: poor scalability, because ASBR needs to manage all VPN routes and create VPN instances for each VPN. This will cause the number of VPN-IPv4 routes on the ASBR to be too large. In addition, because common IP forwarding is used between ASBRs, different interfaces are required for each cross-domain VPN, which increases the requirements for PE equipment. If it spans multiple autonomous domains, the intermediate domain must support VPN services, which not only has a large amount of configuration, but also has a great impact on the intermediate domain. When the number of cross-domain VPNs is relatively small, it can be used first.

OptionB：

The ASBR advertises labeled VPN-IPv4 routes through MP-EBGP, which is also called EBGP redistribution of labeled VPN-IPv4 routes.

In the cross-domain VPN-OptionB solution, the ASBR receives all cross-domain VPN-IPv4 routes from within and outside the domain, and then advertises the VPN-IPv4 routes. However, in the basic implementation of MPLS VPN, only VPN routes that match the VPN Target of the local VPN instance are stored on the PE. Therefore, you can configure the ASBR not to do RT filtering to transmit routes, so there is no need to create a VPN instance on the ASBR and no need to bind any interfaces.

RR equipment can be superimposed on the network to be responsible for the delivery of VPN routes on the client side.

Principles of route release:
1. Scenario without RR:

We only explain the working process of the control plane in a single direction, and assume that there is a VPN route Client1 connection at Site1:

1. In AS100, by running the LDP protocol, PE1 assigns a tunnel label (outer label) T1 associated with the route to PE1 to P1.
2. In AS100, by running the LDP protocol, P1 assigns a tunnel label (outer label) T2 associated with the route to PE1 to ASBR-PE1.
3. In AS200, also by running the LDP protocol, ASBR-PE2 assigns a tunnel label (outer label) T3 associated with the route to ASBR-PE2 to P2.
4. In AS200, by running the LDP protocol, P2 assigns a tunnel label (outer label) T4 associated with the route to ASBR-PE2 to PE2.
5. CE1 advertises the route Client1 to PE1, and the next hop of the route is the interface address of CE1.
6. PE1 re-advertises the IPv4 route Client1 as a VPNv4 route through MP-IBGP, and changes the next hop to PE1, assigns a VPN label V1, and then advertises it to ASBR-PE1.
7. ASBR-PE1 advertises the VPNv4 route of Client1 to ASBR-PE2 through MP-EBGP, changes the next hop to ASBR-PE1, and reassigns a VPN label V2.
8. ASBR-PE2 advertises the received VPNv4 route of Client1 to PE2 through MP-IBGP, points the next hop to itself, and redistributes a VPN label V3.
9. PE2 changes the VPNv4 route of Client1 to an IPv4 route, advertises the route Client1 to CE2, and changes the next hop to PE2.

2. Scenarios with RR:

When the number of VPN instances is large, special RR equipment can be deployed. As shown in the figure, the PE and ASBR devices in the AS only establish the MP-BGP neighbor relationship with the RR device. The RR is responsible for route reflection and transmission, and there is no need to establish BGP neighbor relationships between PE and ASBR.

The RR is only responsible for the VPNv4 routing transmission of the control plane. When data is forwarded, the traffic does not pass through the RR.

Data forwarding principle:

Through the above process analysis, we analyze the working process of the forwarding plane from the reverse direction:

1. CE2 sends an IP packet with a destination of Client1 to PE2.
2. PE2 encapsulates the MPLS label after receiving the IP packet, first encapsulates the VPN label V3, and then encapsulates the outer label T4, and then sends the packet to P2.
3. P2 performs label switching, changes the outer label T4 to T3, and then sends this message to ASBR-PE2.
4. ASBR-PE2 removes the outer label, exchanges the VPN label V3 to V2, and then forwards it to ASBR-PE1 (the packet only carries a private network label at this time).
5. ASBR-PE exchanges VPN label V2 into V1, adds an outer label T2, and forwards the message to P1.
6. P1 performs label switching, changes the outer label T2 to T1, and then sends this message to PE1.
7. After receiving it, PE1 removes all labels, and forwards the message (normal IP message) to CE1.

Features of optionB:
Advantages: Unlike OptionA, OptionB is not limited by the number of interconnection links between ASBRs.
Disadvantages: VPN routing information is stored and diffused through the ASBR between ASs. When there are many VPN routes, the ASBR is heavily burdened and easily becomes a point of failure. Therefore, in the MP-EBGP scheme, the ASBR that needs to maintain VPN routing information is generally no longer responsible for public network IP forwarding.

OptionC: Option 1

The PE or RR advertises labeled VPN-IPv4 routes through Multi-hop MP-EBGP, which is also called Multihop EBGP redistribution of labeled VPN-IPv4 routes.

In cross-domain VPN-OptionC, the ASBR advertises labeled IPv4 routes to PE devices in the respective AS through MP-IBGP, and advertises the labeled IPv4 routes to the PEs in the AS to its ASBR peers in the peer AS for transition autonomy The ASBR in the system also advertises labeled IPv4 routes. In this way, a BGP LSP is established between the ingress PE and the egress PE.

Multihop EBGP connections are established between PEs in different ASs to exchange VPNv4 routes. The ASBR does not store VPN-IPv4 routes, nor does it advertise VPNv4 routes to each other.

When the scale of the network is large, RR equipment can be deployed in the solution, which is specifically responsible for the transmission of user-side routes. That is, PE and RR establish MP-IBGP neighbors, RR1 and RR2 establish MP-EBGP neighbors, and route transmission is PE1-RR1-RR2-PE2. There is no need to directly establish BGP neighbor relationships between PEs. When the number of VPNs is large, the RR’s The method can reduce the workload of PE.

In this scenario, ASBR no longer maintains or advertises VPNv4 routes (so as shown in the figure above, here the ASBR-PE router is renamed ASBR). The ASBR only needs to maintain all labeled routes to the PE and advertise them to the peer AS through EBGP. The ASBR in the transit AS also needs to use EBGP to advertise these labeled IPv4 routes. In this way, an LSP is established between PEs in different ASs, so that a multi-hop MP-EBGP connection between PEs can be established and VPNv4 route announcements can be made.

If the P router of each AS can know the route to the PE router of other AS, the situation will be relatively simple. But if P does not know, then when PE receives VPN data from CE, it will add a three-layer label. The bottom layer label is the VPN label associated with the VPN route assigned by the peer PE, and the middle label is ASBR. The assigned label is associated with the route to the peer PE, and the outer label is the label associated with the route to the next-hop ASBR.

In order to further expand performance, multi-hop MP-EBGP sessions can be established between VPN RRs in different ASs. And when these VPN RRs advertise VPNv4 routes, they do not change the next hop information. The PE only establishes an MP-iBGP session with the VPN RR.

Note: For convenience, as shown in the figure above, a symmetric LSP is used for illustration, but in fact, in the working process of the control plane and the data plane, the LSP structure of the AS at both ends is asymmetric. This will be explained in detail below.

Understanding of the establishment of a BGP tunnel:
Since the P device does not have a routing entry to the peer PE, when it receives the entry transmitted by MP-BGP, it does not know how to send it to the device in the peer AS domain (the next hop is the peer PE address), which causes the routing entries to be discarded. In order to prevent this from happening, a tunnel from the PE source device to the ASBR of the destination AS needs to be established. This tunnel helps forward MP-BGP routing packets. The destination address of the tunnel is also The ASBR of this AS. In this way, there is a new destination address inside to replace the original destination address to establish an MPLS tunnel for the P device (that is, the next hop of the BGP tunnel, the address of the local ASBR), which ensures that the packets can arrive normally. In the process of BGP tunnel establishment, the ASBR sends the label corresponding to the IP address of the local MP-BGP to the peer ASBR and then to the PE to complete the tunnel establishment.

Message sending process:
1. Scenario without RR:

We only explain the working process of the control plane in a single direction. At the same time, assume that there is a VPN route Client1 connection at Site1, and neither P1 nor P2 routers go to the PE of another AS. Routing, the above figure is an example:

1. In AS100, by running the LDP protocol, PE1 assigns a tunnel label (outer label) T1 associated with the route to PE1 to P1.
2. In AS100, by running the LDP protocol, P1 assigns a tunnel label (outer label) T2 associated with the route to PE1 to ASBR-PE1.
3. In AS200, also by running the LDP protocol, ASBR-PE2 assigns a tunnel label (outer label) T3 associated with the route to ASBR-PE2 to P2.
4. In AS200, by running the LDP protocol, P2 assigns a tunnel label (outer label) T4 associated with the route to ASBR-PE2 to PE2.
5. ASBR1 advertises a labeled IPv4 route to PE1 to ASBR2 through an EBGP session, where the next hop is ASBR1, the label is the BGP label, and the value is B1.
6. ASBR2 advertises a labeled IPv4 route to PE1 to PE2 through a BGP session. The next hop is ASBR2, the label is BGP label, and the value is B2. Note: It is assumed here that the AS where PE2 and ASBR1 are located has assigned tunnel labels (public network labels) for routes to them, and the labeled routes to PE2 have also been advertised.
7. PE1 and PE2 establish an MP-EBGP session
8. CE1 advertises the route Client1 to PE1, and the next hop of the route is the interface address of CE1
9. PE1 redistributes the IPv4 route Client1 as a VPNv4 route through MP-EBGP, and changes the next hop to PE1, assigns a VPN label V1, and advertises it to PE2
10. PE2 changes the VPNv4 route to IPv4 route, advertises the IPv4 route Client1 to CE2, and changes the next hop to PE2

2. Scenarios with RR:

VPNv4 neighbors: The local PE only establishes VPNv4 neighbors with the local RR, and the local RR establishes VPNv4 neighbors with the opposite RR, realizing the transmission of cross-domain VPN routes.

ASBR, P, and PE establish BGP unicast IPv4 neighbors with RR:

1. The ASBR uses IPv4 neighbor learning to pass the loopback of the RR learned from the peer ASBR to the local RR for establishing vpnv4 neighbors between the local RR and the peer RR.
2. The ASBR learns the loopback of the RR and PE learned from the peer ASBR through ipv4 neighbor learning, and transmits it to the local RR, and the local RR reflects it to the P for recursive query of cross-domain bgp routing.
3. The ASBR learns the RR and PE loopback from the peer ASBR through ipv4 neighbor learning, and transmits it to the local RR, and the local RR reflects it to the PE to establish BGP LSPs between PEs across domains.

In the scenario with RR, the RR is responsible for the IPv4 route reflection of the control plane and the transmission of VPNv4 routes. The traffic on the forwarding plane does not pass through the RR.

Traffic forwarding process:

Through the above process analysis, we analyze the working process of the forwarding plane from the reverse direction:

1. CE2 sends an IP packet with a destination of Client1 to PE2.
2. PE2 encapsulates the MPLS label after receiving the IP packet, and first encapsulates the VPN label V1. Since PE1, the next hop to Client1, is not a directly connected neighbor, it is found through the table lookup that the BGP route to PE1 is a labeled route. Add the assigned BGP label B2 as an intermediate label. Finally, because the next hop ASBR2 of the route to PE1 is not a directly connected neighbor, it is found through a table lookup that there is also an associated label T4 to ASBR2. Therefore, the outer layer is encapsulated. Label T4.
3. P2 performs label switching, changes the outer label T4 to T3, and then sends this message to ASBR-PE2.
4. ASBR2 removes the outer label, exchanges BGP label B2 to B1, and then forwards it to ASBR1.
5. When ASBR1 receives the packet, it finds that B1 is allocated, so it removes B1 and further checks the table for forwarding. It is found that the route to PE1 has an associated label T2. Therefore, ASBR1 adds it to the top of the stack and forwards it. Give to P1.
6. P1 performs label switching, changes the outer label T2 to T1, and then sends this message to PE1.
7. After receiving it, PE1 removes all labels, and forwards the message (normal IP message) to CE1.

OptionC: Option 2

The second scheme of cross-domain VPN-OptionC is similar to the scheme. The difference is that in solution 1, three layers of labels, namely VPN label, BGP LSP, and Tunnel LSP are needed to carry traffic, while solution 2 only requires two layers.

Solution 1: After the ASBR receives the BGP label route from the peer ASBR, it needs to configure a policy to generate a new label and advertise it to the PE or RR device in the AS to establish a complete BGP LSP. In the second solution, the ASBR needs to configure MPLS to trigger the distribution of labels for BGP label routes. Therefore, the PEs in the AS can see the LDP LSP to the peer PE instead of the BGP LSP. Similarly, the second solution supports the deployment of RR equipment.

In this scenario, ASBR no longer maintains or advertises VPNv4 routes. The ASBR only needs to maintain all labeled routes to the PE and advertise them to the peer ASBR through EBGP.

After the peer ASBR receives the BGP-labeled route, MPLS LDP will trigger the generation of a label for the BGP-labeled route and transfer it among LDP neighbors in the AS. Therefore, you can see the LDP LSP to the peer PE on the PE.

In order to further expand performance, multi-hop MP-EBGP sessions can be established between VPN RRs in different ASs. PEs in this AS only need to establish MP-IBGP with the RR. These VPN RRs do not change the next hop information when advertising VPNv4 routes, so that when the peer PE forwards traffic, it can iterate to the correct tunnel.

Message sending process:
1. Scenario without RR:

We only explain the working process of the control plane in a single direction. At the same time, assume that there is a VPN route Client1 connection at Site1, and neither P1 nor P2 routers go to the PE of another AS. Routing, the above figure is an example:

1. In AS100, by running the LDP protocol, PE1 assigns a tunnel label (outer label) T1 associated with the route to PE1 to P1.
2. In AS100, by running the LDP protocol, P1 assigns a tunnel label (outer label) T2 associated with the route to PE1 to ASBR1.
3. In AS200, also by running the LDP protocol, ASBR2 assigns a tunnel label (outer label) T3 associated with the route to ASBR2 to P2.
4. In AS200, by running the LDP protocol, P2 assigns a tunnel label (outer label) T4 associated with the route to ASBR2 to PE2.
5. ASBR1 advertises a labeled IPv4 route to PE1 to ASBR2 through an EBGP session, where the next hop is ASBR1, the label is the BGP label, and the value is B1.
6. ASBR2 triggers the establishment of LSP for this BGP label route, and distributes LDP labels T5 to P2, and P2 then distributes T6 to PE2.
7. PE1 and PE2 establish an MP-EBGP session.
8. CE1 advertises the route Client1 to PE1, and the next hop of the route is the interface address of CE1.
9. PE1 re-advertises the IPv4 route Client1 as a VPNv4 route through MP-EBGP, and changes the next hop to PE1, assigns a VPN label V1, and advertises it to PE2.
10. PE2 changes the VPNv4 route to an IPv4 route, advertises the IPv4 route Client1 to CE2, and changes the next hop to PE2.

2. Scenarios with RR:

VPNv4 neighbors:
The local PE only establishes VPNv4 neighbors with the local RR, and the local RR establishes VPNv4 neighbors with the opposite RR, realizing the transmission of cross-domain VPN routes. In the RR scenario, the RR is only responsible for the transmission of VPNv4 routes on the control plane, and the traffic on the forwarding plane does not pass through the RR.

Message forwarding process:

Through the above process analysis, we analyze the working process of the forwarding plane from the reverse direction:

1. CE2 sends an IP packet with a destination of Client1 to PE2.
2. PE2 encapsulates the MPLS label after receiving the IP packet, and first encapsulates the VPN label V1. Since PE1, the next hop to Client1, is not a directly connected neighbor, it is found through a table lookup that the label to PE1 is T6, which is marked with T6.
3. P2 performs label switching, changes the outer label T6 to T5, and then sends this message to ASBR2.
4. ASBR2 removes the outer label, exchanges T5 to B1, and then forwards it to ASBR1.
5. When ASBR1 receives the packet, it finds that B1 is allocated, so it removes B1 and further checks the table for forwarding. It is found that the route to PE1 has an associated label T2. Therefore, ASBR1 adds it to the top of the stack and forwards it. Give to P1.
6. P1 performs label switching, changes the outer label T2 to T1, and then sends this message to PE1.
7. After receiving it, PE1 removes all labels, and forwards the message (normal IP message) to CE1.

Why optionC method 2 only has two layers of labels:
Because the P device already has a route to the opposite AS in method 2, it can directly establish an MPLS VPN tunnel. This tunnel is equivalent to a long tunnel across AS, consisting of two short ASs. Short tunnel + BGP label tunnel between ASBRs. Since it is a long tunnel, each device needs a protocol exchange connection, and the MPLS LDP protocol is not directly enabled between the two ASs to distribute labels. This is obviously not possible, so it needs to be done through the BGP label between the ASBRs. Link, so in the upstream of the entire tunnel (divided according to the flow direction), in order to link the tunnel with the downstream, LDP must be connected to the BGP protocol, and labels that can be connected normally must be assigned.

Features of OptionC method:
Advantages:

1. VPN routes are directly exchanged between the ingress PE and the egress PE, without the need for storage and forwarding by intermediate devices.
2. VPN routing information only appears on PE equipment, while P and ASBR are only responsible for packet forwarding, so that the equipment in the intermediate domain may not support MPLS VPN services and only need to support MPLS forwarding, and ASBR equipment no longer becomes a performance bottleneck. Therefore, the cross-domain VPN-OptionC is more suitable for use when crossing multiple ASs.
3. It is more suitable for supporting MPLS VPN load sharing.

Disadvantage: It is
expensive to maintain an end-to-end PE connection management.

Comparison of three solutions:

Source of information: Huawei HCIE training documents, "HCIE Routing and Exchange Learning Guide"