ACI 故障排除：深入了解 PBR.pdf

《ACI 故障排除：深入了解 PBR.pdf》由會員分享，可在線閱讀，更多相關《ACI 故障排除：深入了解 PBR.pdf（74頁珍藏版）》請在三個皮匠報告上搜索。

1、#CiscoLive#CiscoLiveMichael Garcia,ACI Technical Leader,Customer ExperienceBRKDCN-3615ACI Troubleshooting:A Deep Dive into PBR 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveEnter your personal notes hereCisco Webex App Questions?Use Cisco Webex App to chat with the speak

2、er after the sessionFind this session in the Cisco Live Mobile AppClick“Join the Discussion”Install the Webex App or go directly to the Webex spaceEnter messages/questions in the Webex spaceHowWebex spaces will be moderated by the speaker until June 9,2023.12343https:/ 2023 Cisco and/or its affiliat

3、es.All rights reserved.Cisco PublicBRKDCN-36153Agenda 2023 Cisco and/or its affiliates.All rights reserved.Cisco PublicWhy do we need PBR?How PBR WorksPBR Validation ToolsPBR Packet Walk w/CLI VerificationPBR Troubleshooting WorkflowCommon PBR IssuesBRKDCN-36154 2023 Cisco and/or its affiliates.All

4、rights reserved.Cisco Public#CiscoLiveGlossary of Acronyms5BRKDCN-3615AcronymsAcronymsDefinitionsDefinitionsACIApplication Centric InfrastructureAPICApplication Policy Infrastructure ControllerEPEndpointEPGEndpoint GroupBDBridge DomainVRFVirtual Routing and ForwardingCOOPCouncil of Oracle ProtocolPB

5、RPolicy-Based RedirectVxLANVirtual eXtensible LANpcTagPolicy Class TagAcronymsAcronymsDefinitionsDefinitionsdXXXoOuter Destination XXX(dIPo=Outer Destination IP)sXXXoOuter Source XXX(sIPo=Outer Source IP)dXXXiInner Destination XXX(dIPi=Inner Destination IP)sXXXiInner Source XXX(sIPi=Inner Source IP)

6、VNIDVirtual Network IdentifierVxLAN packet acronymsWhy do we need PBR?2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveWEB EPGAPP EPGService Insertion without PBRRequires extended ACLs and route-mapsMore complex route configuration to redirect traffic to desired next-hopLea

7、fLeaf-1 1LeafLeaf-2 2LeafLeaf-3 3Needs Extended ACLs and possible VRF leakingBRKDCN-36157Customer has to manually configure contractsContract to allow APP EPG-FWContract to allow WEB EPG-FW 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveWhy do we service graphs with PBR i

8、n ACI?Service graphs automate contract configurationPBR gives us the ability to attach forwarding constructs to contractsWEB EPGAPP EPGBRKDCN-36158Service graph tied to subject of contractHow PBR Works 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveHow PBR worksService Gr

9、aph Templates Graph Templates definesdefines HOW traffic should flowThe Device Selection Policy Device Selection Policy defines how the Device will communicate with the fabricThe DeviceDevice tells us how many interfaces and logical connectors on the Service DevicesContractContract selects traffic t

10、o redirectRedirect policy Redirect policy defines parameters about service device we are redirecting trafficBRKDCN-3615101EP2EP2EP1EP2APPEPGWEBEPGShadow EPGLeafLeaf-1 1LeafLeaf-2 2LeafLeaf-3 3 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLivePBR Components(Review)DevicesDev

11、icesPhysical Device&interfaces it connects to in fabric.Converted to Consumer Connector and Provider Connector The Device tells us how many interfaces and logical connectors on the Service DevicesService Graph TemplateService Graph TemplateDefine the flow of traffic ContractContractPlaces Contract b

12、etween Consumer&Provider and the shadow EPGSelects traffic to redirectDevice Selection PolicyDevice Selection PolicyTies the physical device to a Graph template and contractDefines how the Device will communicate with the fabricRedirect PolicyRedirect PolicyDefines information about the service node

13、 that is used for redirectionBRKDCN-361511 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveWhere is policy enforced?ScenarioScenarioVRF enforcement VRF enforcement modemodeConsumerConsumerProviderProviderPolicy enforced onPolicy enforced onIntraIntra-VRFVRFIngress/egressIn

14、gress/egressEPGEPGIf destination endpoint is learned:ingress leafIf destination endpoint is learned:ingress leaf*If destination endpoint is not learned:egress leafIf destination endpoint is not learned:egress leafIngressIngressEPGL3Out EPGConsumer leaf(non-border leaf)IngressIngressL3Out EPGEPGProvi

15、der leaf(non-border leaf)EgressEgressEPGL3Out EPGBorder leaf-non-border leaf trafficIf destination endpoint is learned:border leafIf destination endpoint is learned:border leafIf destination endpoint is not learned:nonIf destination endpoint is not learned:non-border leafborder leafNon-border leaf-b

16、order leaf traffic:Border leafEgressEgressL3Out EPGEPGIngress/egressIngress/egressL3Out EPGL3Out EPGIngress leaf*InterInter-VRFVRFIngress/egressIngress/egressEPGEPGConsumer leafIngress/egressIngress/egressEPGL3Out EPGConsumer leaf(non-border leaf)Ingress/egressIngress/egressL3Out EPGEPGIngress leaf*

17、Ingress/egressIngress/egressL3Out EPGL3Out EPGIngress leaf*Policy enforcement is applied on the first leaf hit by the packet.BRKDCN-361512 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveExternal and Internal InterfacesWhat are shadow EPGs?A two armed example Shadow EPGs c

18、onnect to the service Device External Interface is called the“Consumer Connector”Internal interface is the“Provider Connector”Each is represented by a VLAN and has its own PcTagKind of a way of stitchingEPG VLAN and servicenode VLAN to”steer”traffic to service nodeConsProvProvider ConnectorConsumer

19、ConnectorLeafLeaf-1 1EP1EP2EPG APPEPG WebShadow EPGLeafLeaf-2 2BRKDCN-361513 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveShadow EPGs&contracts EP1EP2EPG App to EPG Web(Redirect)Consumer Connection to App(uni-dir filter)EPG Web to EPG App(Redirect)Provider Connection to

20、 Web(uni-dir default)EPG APPEPG Weba1-leaf1#show zoning-rule scope 2293762|Rule ID|SrcEPG|DstEPG|FilterID|Dir|operSt|Scope|Action|+-+-+-+-+-+-+-+-+|4212|32771|16386|1|bi-dir|enabled|2293762|redir(destgrp-3)|4098|16389|32771|1|uni-dir|enabled|2293762|permit|4199|16386|32771|1|uni-dir-ignore|enabled|2

21、293762|redir(destgrp-3)|+4152|16389|16386|default|uni-dir|enabled|2293762|permit +3277116389BRKDCN-36151416386LeafLeaf-1 1LeafLeaf-2 2LeafLeaf-3 3ShadowEPGAll in same VRFHow PBR Forwarding Works 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveVRF1(ALL BDs in Same VRF)VRF1(

22、ALL BDs in Same VRF)IP:192.168.1.100IP:192.168.1.100GW:192.168.1.1GW:192.168.1.1MAC:APPMAC:APPEPGEPG-APPAPPLeafLeaf-1 1SpineSpineLeafLeaf-3 3SpineSpineIP:172.16.1.100IP:172.16.1.100FWFWLeafLeaf-2 2BD1BD1192.168.1.1192.168.1.1BD2BD2192.168.2.1192.168.2.1SVC_BDSVC_BD172.16.1.1172.16.1.1IP:192.168.2.10

23、0IP:192.168.2.100GW:192.168.2.1GW:192.168.2.1MAC:WEBMAC:WEBEPGEPG-WEBWEB1.EP1 Initiates Traffic to 1.EP1 Initiates Traffic to EP2EP2SRC IP:192.168.1.100SMAC:APP MACDST IP:192.168.2.100DMAC:LEAF MAC2.*PBR POLICY APPLIED 2.*PBR POLICY APPLIED HERE*HERE*SCLASS:APPDCLASS:WEBSRC IP:192.168.1.100SMAC:APPD

24、ST IP:192.168.2.100DMAC:FWDMAC:FW-MACMACSEGMENT ID(BDVNID):SEGMENT ID(BDVNID):SVC_BDSVC_BD3.Traffic is sent 3.Traffic is sent to MAC proxyto MAC proxy4.Spine does 4.Spine does COOP lookup for COOP lookup for Service MACService MAC6.Traffic from 6.Traffic from service nodeservice nodeSRC IP:192.168.1

25、.100SMAC:FW MACSMAC:FW MACDST IP:192.168.2.100DMAC:LEAF MACDMAC:LEAF MAC7.POLICY CHECK7.POLICY CHECKSCLASS:SCLASS:SVC_PROVSVC_PROVDCLASS:WEB8.Normal EP 8.Normal EP forwarding(Transit forwarding(Transit or Proxy)or Proxy)9.Traffic goes to 9.Traffic goes to provider(WEB EPG)provider(WEB EPG)SRC IP:192

26、.168.1.100SMAC:LEAF MACDST IP:192.168.2.100DMAC:WEBEndpointEndpointLocationLocation192.168.1.100Leaf-1192.168.2.200192.168.2.200LeafLeaf-3 3DST IP Is DST IP Is LEARNEDLEARNED on on LeafLeaf-1 15.Traffic arrives on 5.Traffic arrives on service leaf service leaf SCLASS:APPDCLASS:SVC_CONSDCLASS:SVC_CON

27、SBRKDCN-361516 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveVRF1(ALL BDs in Same VRF)VRF1(ALL BDs in Same VRF)IP:192.168.1.100IP:192.168.1.100GW:192.168.1.1GW:192.168.1.1MAC:APPMAC:APPEPGEPG-APPAPPLeafLeaf-1 1SpineSpineLeafLeaf-3 3SpineSpineIP:172.16.1.100IP:172.16.1.10

28、0LeafLeaf-2 2BD1BD1192.168.1.1192.168.1.1BD2BD2192.168.2.1192.168.2.1SVC_BDSVC_BD172.16.1.1172.16.1.1IP:192.168.2.100IP:192.168.2.100GW:192.168.2.1GW:192.168.2.1MAC:WEBMAC:WEBEPGEPG-WEBWEB*Return Traffic*Return Traffic*EPG WEB EPG WEB-EPG EPG APPAPP1.EP2 sends traffic back to EP11.EP2 sends traffic

29、back to EP1SRC IP:192.168.2.100SMAC:APP MACDST IP:192.168.1.100DMAC:LEAF MAC2.*PBR POLICY APPLIED 2.*PBR POLICY APPLIED HERE*HERE*SCLASS:WEBDCLASS:APPSRC IP:192.168.2.100SMAC:WEBDST IP:192.168.1.100DMAC:FWDMAC:FW-MACMACSEGMENT ID(BDVNID):SVC_BDSEGMENT ID(BDVNID):SVC_BD3.Traffic is sent to 3.Traffic

30、is sent to MAC proxyMAC proxy5.Traffic arrives on 5.Traffic arrives on service leaf service leaf SCLASS:WEBDCLASS:SVC_PROVDCLASS:SVC_PROV7.POLICY CHECK7.POLICY CHECKSCLASS:SCLASS:SVC_CONSSVC_CONSDCLASS:APP6.Traffic from service 6.Traffic from service nodenodeSRC IP:192.168.2.100SMAC:FW MACSMAC:FW MA

31、CDST IP:192.168.1.100DMAC:LEAF MACDMAC:LEAF MAC9.Traffic returns to 9.Traffic returns to consumer(APP EPG)consumer(APP EPG)SRC IP:192.168.2.100SMAC:LEAF MACDST IP:192.168.1.100DMAC:APP8.Spines already has 8.Spines already has APP EP location in COOP APP EP location in COOP and sends packet directly

32、and sends packet directly to Leafto Leaf-1 14.Spine does COOP 4.Spine does COOP lookup for Service lookup for Service MACMACBRKDCN-361517FWFW 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLivePlease refer to BRKDCN-3982:So what about PBR multi-location deployments?BRKDCN-361

33、518PBR Validation Tools 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveContract Parser ScriptLeaf#contract_parser.py-help-nz,-nonzero display only entries with non-zero hits-incremented display only entries that have incremented since last checked-node NODES NODES.display

34、 entries specific to one or more leaf nodes-contract CONTRACT CONTRACT.display only rules that match a specific contract.Thename of the contract is in the formuni/tn-/brc-vrf VRF VRF.display entries for a specific vrf.The integer vnidof the vrf can be provided or the vrf name in the form:-epg EPG EP

35、G.display entires for specific EPG.The integer pcTag orDN name can be provided.Note the dn is a partial dnin the formtn-/ap-/epg-Validates contract rules on the LeafBRKDCN-361520Available since 3.2(2)2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveContract Parser Script Ex

36、amplea1-leaf1#contract_parser.py-vrf mg-cisco-live:v1-sepg tn-mg-cisco-live/ap-a1/epg-e1-depg tn-mg-cisco-live/ap-a1/epg-e27:4249 vrf:mg-cisco-live:v1 redir iptn-mg-cisco-live/ap-a1/epg-e1(32771)tn-mg-cisco-live/ap-a1/epg-e2(16386)contract:uni/tn-mg-cisco-live/brc-iphit=130028 destgrp-3 vrf:mg-cisco

37、-live:v1 ip:172.16.1.100 mac:00:50:56:A8:48:97 bd:vxlan-16744311Rule IDVRFSRC EPG DST EPGContract Hit CountFor PBR gives redirect group infoCONTRACTAction+FilterService MAC and Service BD(in VXLAN header)BRKDCN-361521 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveELAM Em

38、bedded Logic Analyzer ModuleIt is a tripwire in hardwareThe first frame to match a specified condition trips itReport is created with vast amount of data regarding asic decisions22BRKDCN-3615Dst TCP 192.168.2.100:3000DstDst TCP TCP 192.168.2.100192.168.2.100:3001:3001Dst TCP 192.168.2.100:3002vsh_lc

39、debug platform internal tah elam asic 0trigger resettrigger init in-select 6 out-select 1set outer ipv4 dst_ip 192.168.2.100set outer l4 dst-port 3001startmodule-1(DBG-elam-insel6)#statELAM STATUS=Asic 0 Slice 0 Status ArmedAsic 0 Slice 1 Status Triggeredmodule-1(DBG-elam-insel6)#ereport|grep drop r

40、easonRW drop reason :no dropLU drop reason :no dropMatching frame was Matching frame was caught!caught!Frame was not dropped in lookups!2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLive23What ASIC should be set in the ELAM?BRKDCN-3615vsh_lcdebug platform internal elam asic

41、0ModelRoleAsic for ElamN9K-C*CFixed SpinerocN9K-C*GXFixed SpineappN9K-C*-EXLeaftahN9K-C*-FX/FXP/FX2LeafrocN9K-C*-GXLeafappN9K-C*-GX2LeafchoN9K-X97*-EXSpine LCtahN9K-X97*-FXSpine LCrocN9K-X97*-GXSpine LCappN9K-C95*-FM-ESpine FMtahN9K-C950*-FM-E2Spine FMrocN9K-C95*-FM-GSpine FMapp 2023 Cisco and/or it

42、s affiliates.All rights reserved.Cisco Public#CiscoLiveSteps to Using Elam on Gen2+Leaf or Fixed Spine24BRKDCN-3615vsh_lcdebug platform internal tah elam asic 0trigger resettrigger init in-select 6 out-select 0set outer ipv4 dst_ip 192.168.2.100set outer l4 dst-port 3001startElams are run from Elams

43、 are run from the line card shellthe line card shellRefer to“What ASIC should Refer to“What ASIC should be set in the ELAM”slide be set in the ELAM”slide Leafs and fixed spines are single Leafs and fixed spines are single asic switches.Always use asic 0asic switches.Always use asic 0Failing to reset

44、 the trigger Failing to reset the trigger can cause past elam can cause past elam configurations to take effect.configurations to take effect.Always reset the trigger!Always reset the trigger!module-1(DBG-elam)#trigger init in-select?!ommitted14 Outer(l2(vntag)|l3|l4)-inner(l2|l3|l4)-ieth6 Outerl2-o

45、uterl3-outerl47 Innerl2-innerl3-innerl4!ommittedDetermines which headers conditions can be matched in.Use 14 or 7 when matching vxlan encapsulated headers.Use 0 or 1Use 0 or 1 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSteps to Using Elam on Gen2+Leaf or Fixed Spine25

46、BRKDCN-3615vsh_lcdebug platform internal tah elam asic 0trigger resettrigger init in-select 6 out-select 0set outer ipv4 dst_ip 192.168.2.100set outer l4 dst-port 3001startUse“Use“set outer”or”or“set inner”depending”depending on inon in-select and if select and if matching outer or inner matching ou

47、ter or inner headers in vxlan packetheaders in vxlan packetWhich headers to match Which headers to match conditions for?conditions for?What to match in the What to match in the header?header?Finally enable the elam!Finally enable the elam!When running When running stat if if Triggered is seen,this i

48、s seen,this means a matching packet was receivedmeans a matching packet was received 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveELAM on Leaf CLI-PBR VerificationBRKDCN-361526module-1(DBG-elam-insel6)#statELAM STATUS=Asic 0 Slice 0 Status ArmedAsic 0 Slice 1 Status Tri

49、ggeredvsh_lcdebug platform internal tah elam asic 0trigger resettrigger init in-select 6 out-select 1set outer ipv4 src_ip 192.168.1.100 dst_ip 192.168.2.100 startmodule-1(DBG-elam-insel6)#ereport|egrep Source IP|Destination IP|Contract Applied|service_redirDestination IP :192.168.2.100 Source IP :1

50、92.168.1.100 Contract Applied :yessug_luc_latch_results_vec.luc3_0.service_redir:0 x1e ereportreport available since 4.2 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveELAM Assistant APP-PBR VerificationBRKDCN-361527Set ParametersTriggered!andReport is Ready.Click to see

51、report.2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveFTRIAGE-PBR Verificationa-apic1#ftriage help usage:ftriage-h-outdir OUTDIR-loglevel loglevel-user username-nsdropactionnsdropaction-xt xt-wait WAIT bridge,example,route,infraping.a-apic1#ftriage example*snippet*Example

52、s:#Bridge ftriage bridge-ii leaf1:Eth1/1-ie 100-ei leaf2:Eth1/2-ee 101-dmac 02:02:02:02:02:02#Route ftriage route-ii leaf1:Eth1/1-ie 100-ei leaf2:Eth1/2-ee 101-dip 192.168.1.101*snippet*End-to-End ELAMs run from the APIC:BRKDCN-361528 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public

53、#CiscoLive29BRKDCN-3615FTRIAGE Example for PBR FlowBRKDCN-391529a-apic1#ftriage-user admin route-ii a1-leaf1:Eth1/44-sip 192.168.1.100-dip 192.168.2.100 2023-04-23 19:28:20,133 INFO ftriage:main:1295 L3 packet Seen on a1-leaf1 Ingress:Eth1/44 Egress:Eth1/52 Vnid:16089032 2023-04-23 19:28:20,133 INFO

54、 ftriage:main:1337 a1-leaf1:Incoming Packet captured with SIP:192.168.1.100,DIP:192.168.2.100 2023-04-23 19:29:13,080 INFO ftriage:unicast:1543 a1-leaf1:traffic is redirected to vnid:16744311 mac:00:50:56:A8:48:97 via tenant:mg-cisco-live graph:pbr-one-arm-SG-template contract:ip2023-04-23 19:34:33,

55、920 INFO ftriage:acigraph:743 found matching devicenode:N1 ldev:pbr-one-arm-CL2023 dev:asavasavuni/tn-mg-cisco-live/lDevVip-pbr-one-arm-CL2023/cDev-asav/cIf-asav2023-04-23 19:34:33,921 INFO ftriage:unicast:2755 a1-leaf2:PBR first pass is done and traffic is sent to service device:node:N1 ldev:pbr-on

56、e-arm-CL2023 dev:asav2023-04-23 19:34:33,921 INFO ftriage:unicast:2757 a1-leaf2:expected traffic to return from:topology/pod-1/paths-102/pathep-eth1/43 encap:unknownBefore Service Device2023-04-23 19:34:53,165 INFO ftriage:main:1821 pbr return path,nxt_nifs a1-leaf2:Eth1/43,nxt_dbg_f_n ig,nxt_inst i

57、g,eg_ifs Eth1/43,Vnid:unknown2023-04-23 19:35:06,803 INFO ftriage:fcls:2379 a1-leaf2:Valid ELAM for asic:0 slice:0 srcid:114 pktid:1702023-04-23 19:35:07,823 INFO ftriage:main:1295 L3 packet Seen on a1-leaf2 Ingress:Eth1/43 Egress:Eth1/59 Vnid:2293762 2023-04-23 19:35:07,824 INFO ftriage:main:1337 a

58、1-leaf2:Incoming Packet captured with SIP:192.168.1.100,DIP:192.168.2.100 Coming back from Service DevicePBR Packet Walk with CLI Verification 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLivePBR Packet Walk Step 2(Packet Rewrite)EP212LeafLeaf-1 1LeafLeaf-2 2SpineSpine-1 1S

59、IPDIPProtoL4/Payload00FW MACSMAC802.1QSIPDIPVXLANL4/PayloadDMACSMAC802.1QBD VNIDFW MACBD VNIDService BDrewriteEP1 sends packet to EP2 via Leaf-1Leaf-1 does policy lookup and redirects packet to Service BD/Service MACAfter rewrite,sends to MAC Proxy on spineBRKDCN-361531LeafLeaf-3 3SpineSpine-2 2EP11

60、92.168.1.100192.168.2.100172.16.1.100 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommand Line VerificationConfirm Confirm pcTagspcTags of traffic flowof traffic flow2Verify zoningVerify zoning-rule has rule has redirredir action and matches desired traffic type(ex.act

61、ion and matches desired traffic type(ex.ipip traffic)traffic)a1-leaf1#show system internal epm endpoint ip192.168.1.100|egrep VRF vnid|sclassBD vnid:16089032:VRF vnid:2293762Flags:0 x80004c04:sclass:32771a1-leaf1#show system internal epm endpoint ip192.168.2.100|egrep VRF vnid|sclassBD vnid:15826920

62、:VRF vnid:2293762Flags:0 x80004c04:sclass:16386a1-leaf1#show zoning-rule scope 2293762 src-epg 32771 dst-epg 16386+-+-+-+-+-+-+-+|Rule ID|SrcEPG|DstEPG|FilterID|Dir|Scope|Action|-+-+-+-+-+-+|4308|32771|16386|1|bi-dir|2293762|redir(destgrp-3)|a1-leaf1#show zoning-filter filter 1+-+-+-+-+-+-+-+|Filter

63、Id|Name|EtherT|SFromPort|SToPort|DFromPort|DToPort|+-+-+-+-+-+-+-+|1|1_0|ip|unspecified|unspecified|unspecified|unspecified|dclassdclass for dest EPsclasssclass for source EPRedirect happening on ingress leaf since destination is knownknownIf destination EP is If destination EP is knownknown:redirec

64、t happens on ingress leafredirect happens on ingress leaf-If If destination EP is destination EP is unknownunknown:redirect will happen on egress leaf redirect will happen on egress leaf BRKDCN-361532EP1LeafLeaf-1 112DestDest EP is EP is known by leafknown by leaf-1 1IPIP Filter 2023 Cisco and/or it

65、s affiliates.All rights reserved.Cisco Public#CiscoLiveCheck redirect policy to see how packets will be redirected Check redirect policy to see how packets will be redirected a1-leaf1#show service redir info group 3=GrpIDName destination 3 destgrp-3 dest-172.16.1.100-vxlan-2293762a1-leaf1#show servi

66、ce redir info destination ip 172.16.1.100 vnid 2293762=Name bdVnidvMacvrf=dest-172.16.1.100-vxlan-2293762 vxlan-1674431100:50:56:A8:48:97mg-cisco-live:v1Service node redirect IPService node VRF VNID Parameters used to build redirected vxlan packetCommand Line Verification2BRKDCN-361533EP1LeafLeaf-1

67、112module-1(DBG-elam-insel6)#report detail|grep service_redirsug_luc_latch_results_vec.luc3_0.service_redir:0 x1Packet is being redirected0 x1 yes,redirected0 x0 not redirected 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLivePBR Packet Walk Step 3(Spine COOP MAC Lookup)BRK

68、DCN-361534EP2172.16.1.100213LeafLeaf-1 1LeafLeaf-2 2SpineSpine-1 1LeafLeaf-3 3SpineSpine-2 2EP1EP2192.168.1.100192.168.2.100Spine MAC COOP Lookup for Service EP 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveVerify Spine has installed MAC of service node in COOPVerify Spi

69、ne has installed MAC of service node in COOPSpineSpine-1 13LeafLeaf-1 1a1-spine1#show coop internal info repo ep key 16744311 00:50:56:A8:48:97|egrepTunnel|EP|head-n 3EP bd vnid:16744311 EP mac:00:50:56:A8:48:97Tunnel nh:10.0.216.68Map tunnel destination address to leafMap tunnel destination address

70、 to leafService BD VNIDService Device MACapic1#moquery-c ipv4Addr-f ipv4.Addr.addr=10.0.216.68|grep dndn:topology/pod-1/node-102/sys/ipv4/inst/dom-overlay-1/if-lo0/addr-10.0.216.68/32Command Line Verification3Tunnel points to Leaf 102 where service device is connectedBRKDCN-36153521EP1ELAM ingress o

71、n Spine sees Destination TEP as MAC ProxyRewrite info!Service BD VNID 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLivePBR Packet Walk Step 4(Service Leaf)BRKDCN-361536Traffic is sent to Service Leaf(Leaf-2)&Leaf-2 sends traffic to Service Device EP2LeafLeaf-1 1LeafLeaf-2 2

72、SpineSpine-1 1LeafLeaf-3 3SpineSpine-2 2EP1EP24192.168.1.100192.168.2.100172.16.1.100213 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveVerify Service Device programming on Service Node(LeafVerify Service Device programming on Service Node(Leaf-2)2)a1-leaf2#show system in

73、ternal epm endpoint mac 0050.56a8.4897MAC:0050.56a8.4897:Num IPs:1IP#0:172.16.1.100:IP#0 flags:host-trackedVlan id:57:Vlan vnid:10792:VRF name:mg-cisco-live:v1BD vnid:16744311:VRF vnid:2293762Phy If:0 x1a02a000:Tunnel If:0Interface:Ethernet1/43Flags:0 x80004c04:sclass:49155:Ref count:5EP Create Time

74、stamp:03/28/2023 15:23:44.027077EP Update Timestamp:04/14/2023 13:52:41.683129EP Flags:local|IP|MAC|sclass|timer|SpineSpine-1 1EP1EP2LeafLeaf-1 1LeafLeaf-2 24a1-leaf2#show vlan id 57 extended VLAN Name EncapPorts -57 mg-cisco-live:pbr-one-arm-vlan-1417 Eth1/43 CL2023ctxv1:provider:Command Line Verif

75、ication4BRKDCN-361537Shadow EPG pcTAGFD_VLANAccess Encap VLANEP1SpineSpine-2 2123Checking Service MAC Learning 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLivePBR Packet Walk Step 5(Return from FW)BRKDCN-361538EP25Service Device sends traffic back to router MAC(19:FF).Dest

76、ination IP is EP2 and policy lookup is madeLeafLeaf-1 1LeafLeaf-2 2SpineSpine-1 1LeafLeaf-3 3SpineSpine-2 2EP1EP2policy lookup is made(implicit permit)1234 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommand Line VerificationTraffic is sent to 1Traffic is sent to 1-ARM

77、 FW device.After inspection,traffic ARM FW device.After inspection,traffic comes back to Leafcomes back to Leaf-2 via Service EPG VLAN2 via Service EPG VLAN5a1-leaf2#show system internal epm endpoint mac 0050.56a8.4897|egrep VRF vnid|sclassBD vnid:16744311:VRF vnid:2293762Flags:0 x80004c04:sclass:16

78、389SpineSpine-1 1EP1EP2LeafLeaf-1 1LeafLeaf-2 2a1-leaf1#show system internal epm endpoint ip 192.168.2.100|egrep VRF vnid|sclass BD vnid:15826920:VRF vnid:2293762Flags:0 x80000c80:sclass:16386a1-leaf2#show zoning-rule scope 2293762 src-epg 16389 dst-epg 16386+-+-+-+-+-+-+-+-+-+-+|Rule ID|SrcEPG|DstE

79、PG|FilterID|Dir|operSt|Scope|Name|Action|Priority|+-+-+-+-+-+-+-+-+-+-+|4152|16389|16386|default|uni-dir|enabled|2293762|permit|src_dst_any(9)|+-+-+-+-+-+-+-+-+-+-+5pcTAG of service EPGpcTAG of dest EPG(provider of PBR flow)Shadow EPG(1638916389)to provider(1638616386)is implicitly allowed(default f

80、ilter)by service graph*NOTE FOR RETURN TRAFFIC:EP2*NOTE FOR RETURN TRAFFIC:EP2-EP1 PBR flow is the sameEP1 PBR flow is the sameBRKDCN-3615391234a1-leaf2#contract_parser.py-vrf mg-cisco-live:v1-depg tn-mg-cisco-live/ap-a1/epg-e29:4152 vrf:mg-cisco-live:v1 permit any tn-mg-cisco-live/G-pbr-one-arm-CL2

81、023ctxv1/C-provider(16389)tn-mg-cisco-live/ap-a1/epg-e2(16386)contract:uni/tn-mg-cisco-live/brc-ip hit=172Contract hitPBR Troubleshooting Workflow 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLivePBR Troubleshooting WorkflowIs the service graph Is the service graph deployed

82、?deployed?BRKDCN-3615Is the service device reachable from the service leaf?Is there a provider and consumer associated to PBR contract?Validate with contract_parserAre the service destination parameters(IP/MAC)for service device correct?ELAM to understand where packet is forwarded/droppedCorrect con

83、figuration in PBR redirect policyMake sure the correct service graph template is tied to subject of the PBR contract and EPGs respectivelyIs the service VLAN deployed on the service leaf?NOYESNONOYESNOYESNOMake sure cluster interface config matches to appropriate service VLANYESCheck faults regardin

84、g service graph deployment-Check zoning-rules on leaf-Look for “service_redirservice_redir”flag in ELAM report-Check service device for ACLsTIPSTIPSYESUse FTRIAGE and captures on service device to see if packet is seen41Common Issues#1:Service Graph is Not Deploying 2023 Cisco and/or its affiliates.

85、All rights reserved.Cisco Public#CiscoLive#1:Graph is not deploying#1:Graph is not deployingThings to check:Things to check:Is the service graph template is correctly tied to the contract?BRKDCN-361544Nothing actuallyNothing actuallydeployeddeployedCommon PBR ProblemsDoes contract have a filter asso

86、ciated?2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLive#1:Graph is not deploying#1:Graph is not deployingThings to check:Things to check:Is the contract filter valid?BRKDCN-361545Nothing actuallyNothing actuallydeployeddeployedCommon PBR ProblemsNo actual filter entry conf

87、igured!2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLive#1:Graph is not deploying#1:Graph is not deployingFIX:Associate proper filter or correct contract misconfigurationFIX:Associate proper filter or correct contract misconfigurationBRKDCN-361546Common PBR ProblemsService

88、Graph nowService Graph nowdeployeddeployedThe contract has the filter correctly configure but the scope is set to“VRF”even though its for inter-VRF or inter-tenant EPG communication.Additional Additional Config Config“Gotcha”for“Gotcha”for interinter-VRF PBRVRF PBR#2:Graph Shows Deployed but Traffic

89、 is Not Redirected 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveBRKDCN-361548Things to check:Things to check:Check the fault info from APIC(can give you hints to what could be wrong)Common PBR Problems#2:Graph shows deployed but traffic is not directed#2:Graph shows dep

90、loyed but traffic is not directed 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLive#2:Graph shows deployed but traffic is not directed#2:Graph shows deployed but traffic is not directedThings to check:Things to check:BRKDCN-361549a1-leaf2#show vlan encap-id 1417VLAN Name St

91、atus Ports -44 mg-cisco-live:FW:ASAVactive Eth1/43 VLAN Type Vlan-mode -44 enetCE Common PBR ProblemsCheck which EPG the service VLAN is mapped toThe service VLAN(1417)shows it is already tied to an EPG called “mg-cisco-live:FW:ASAV”GUI validation shows a static binding towards FW was deployed in an

92、 application EPG called “ASAV”2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLive#2:Graph shows deployed but traffic is not directed#2:Graph shows deployed but traffic is not directedThings to check:Things to check:BRKDCN-361550Common PBR ProblemsCheck PBR function connectors

93、 to see VLAN encap configuredService Graph configuration using the same VLAN as APP EPG(VLAN-1417)Since static binding was deployed first this PBR VLAN deployment fails due to overlapping VLAN encap ID:2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLive#2:Graph shows deployed

94、 but traffic is not directed#2:Graph shows deployed but traffic is not directedFIX:Delete EPG static binding that maps to service node VLANFIX:Delete EPG static binding that maps to service node VLANBRKDCN-361551Common PBR Problemsa1-leaf2#show vlan encap-id 1417VLAN Name Status Ports -62 mg-cisco-l

95、ive:pbr-one-arm-active Eth1/43 CL2023ctxv1:provider:Service VLAN is properly associated to service EPGa-apic1#moquery-c faultInst-f fault.Inst.code=F0497|grep cisco-livea-apic1#Fault is now cleared!#3:Traffic is Working but Does Not Hit Redirect Rule 2023 Cisco and/or its affiliates.All rights reser

96、ved.Cisco Public#CiscoLiveCommon PBR Problems#3:Traffic is working but#3:Traffic is working but does not hit redirect ruledoes not hit redirect ruleThings to check:Things to check:BRKDCN-361553a1-leaf1#contract_parser.py-vrf mg-cisco-live:v1-sepg tn-mg-cisco-live/ap-a1/epg-e1-depg tn-mg-cisco-live/a

97、p-a1/epg-e27:4351 vrf:mg-cisco-live:v1 redir ip tn-mg-cisco-live/ap-a1/epg-e1(32771)tn-mg-cisco-live/ap-a1/epg-e2(16386)contract:uni/tn-mg-cisco-live/brc-ip hit=348 destgrp-3 vrf:mg-cisco-live:v1 ip:172.16.1.100 mac:00:00:00:00:00:54 bd:vxlan-149749507:4330 vrf:mg-cisco-live:v1 permit ip icmp tn-mg-

98、cisco-live/ap-a1/epg-e1(32771)tn-mg-cisco-live/ap-a1/epg-e2(16386)contract:uni/tn-mg-cisco-live/brc-icmp hit=796,+10a1-leaf1#contract_parser.py-vrf mg-cisco-live:v1-sepg tn-mg-cisco-live/ap-a1/epg-e1-depg tn-mg-cisco-live/ap-a1/epg-e27:4330 vrf:mg-cisco-live:v1 permit ip icmp tn-mg-cisco-live/ap-a1/

99、epg-e1(32771)tn-mg-cisco-live/ap-a1/epg-e2(16386)contract:uni/tn-mg-cisco-live/brc-icmp hit=1294,+10asav-cisco-live#capture pbr_traffic interface inside real-time match icmp any anyUse ctrl-c to terminate real-time capture*PBR traffic not seen*Contract parser script on leafContract“ip”Contract“icmp”

100、Hit count is increasing for“icmp”contract!Rule ID 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommon PBR Problems#3:Traffic is working but#3:Traffic is working but does not hit redirect ruledoes not hit redirect ruleThings to check:Things to check:BRKDCN-361554module-1

101、(DBG-elam-insel6)#trigger reset module-1(DBG-elam)#debug platform internal tah elam asic 0module-1(DBG-elam)#trigger init in-select 6 out-select 1 module-1(DBG-elam-insel6)#set outer ipv4 src_ip 192.168.1.100 dst_ip 192.168.2.100module-1(DBG-elam-insel6)#startmodule-1(DBG-elam-insel6)#statELAM STATU

102、S=Asic 0 Slice 0 Status ArmedAsic 0 Slice 1 Status Triggeredasav-cisco-live#capture pbr_traffic interface inside real-time match icmp any anyUse ctrl-c to terminate real-time capture*PBR traffic not seen*Alternatively you can get an ELAM to see what rule is being hit(*Refer to slide 12 to see where

103、the PBR policy should be applied*)2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommon PBR Problems#3:Traffic is working but#3:Traffic is working but does not hit redirect ruledoes not hit redirect ruleThings to check:Things to check:BRKDCN-361555Contract Lookup Key-IP P

104、rotocol :ICMP(0 x1)L4 Src Port :2048(0 x800)L4 Dst Port :8647(0 x21C7)sclass(src pcTag):32771(0 x8003)EPG consumer 192.168.1.100dclass(dst pcTag):16386(0 x4002)EPG provider 192.168.2.100Contract ResultContract Drop :no Contract Logging :no Contract Applied :yes Contract Hit :yes Contract Aclqos Stat

105、s Index :81863 (show sys int aclqos zoning-rules|grep-B 9 Idx:81863)asav-cisco-live#capture pbr_traffic interface inside real-time match icmp any anyUse ctrl-c to terminate real-time capture*PBR traffic not seen*Check rule being hit in ereportUse this command to get rule ID!2023 Cisco and/or its aff

106、iliates.All rights reserved.Cisco Public#CiscoLiveCommon PBR Problems#3:Traffic is working but#3:Traffic is working but does not hit redirect ruledoes not hit redirect ruleThings to check:Things to check:BRKDCN-361556module-1#show sys int aclqos zoning-rules|grep-B 9 Idx:81863|grep Rule IDRule ID:43

107、28 Scope 44 Src EPG:32771 Dst EPG:16386 Filter 5asav-cisco-live#capture pbr_traffic interface inside real-time match icmp any anyUse ctrl-c to terminate real-time capture*PBR traffic not seen*Check rule being hit(command from previous slide)Check rule being hit(command from previous slide)a1-leaf1#s

108、how zoning-rule scope 2293762|egrep Rule ID|4328|Rule ID|SrcEPG|DstEPG|FilterID|Scope|Name|Action|Priority|4328|32771|16386|5|2293762|mg-cisco-live:icmp|permit|fully_qual(7)Check zoningCheck zoning-rule output for the rule ID above(4238)rule output for the rule ID above(4238)Map filter ID to type of

109、 trafficMap filter ID to type of traffica1-leaf1#show zoning-filter filter 5|grep-A 2 Protocol|FilterId|Name|EtherT|ArpOpc|Prot|ApplyToFrag|Stateful|SFromPort|SToPort|DFromPort|-+-+-+-+-+-+-+-+-+-|5|5_0|ip|unspecified|icmp|no|no|unspecified|unspecified|unspecified No“No“redirredir”action”actionVRF V

110、NIDFilter has specific protocol set for ICMP traffic 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommon PBR Problems#3:Traffic is working but#3:Traffic is working but does not hit redirect ruledoes not hit redirect ruleThings to check:Things to check:BRKDCN-361557a1-le

111、af1#show zoning-rule scope 2293762|grep redir|4325|16386|32771|1|uni-dir-ignore|enabled|2293762|redir(destgrp-3)|fully_qual(7)|4351|32771|16386|1|bi-dir|enabled|2293762|redir(destgrp-3)|fully_qual(7)asav-cisco-live#capture pbr_traffic interface inside real-time match icmp any anyUse ctrl-c to termin

112、ate real-time capture*PBR traffic not seen*Look for redirect rule and compare to previous slideLook for redirect rule and compare to previous slidea1-leaf1#show zoning-filter filter 1|grep-A 2 Protocol|FilterId|Name|EtherT|ArpOpc|Prot|ApplyToFrag|Stateful|SFromPort|SToPort|DFromPort|-+-+-+-+-+-+-+-+

113、-+-|1|1_0|ip|unspecified|ip|no|no|unspecified|unspecified|unspecified Filter has specific protocol set for IP trafficValidate contract filterValidate contract filter 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommon PBR Problems#3:Traffic is working but#3:Traffic is w

114、orking but does not hit redirect ruledoes not hit redirect ruleThings to check:Things to check:BRKDCN-361558a-apic1#moquery-c vzRtProv-f vz.RtProv.dn*tn-mg-cisco-live/brc-ip|grep dndn:uni/tn-mg-cisco-live/brc-ip/rtfvProv-uni/tn-mg-cisco-live/ap-a1/epg-e2a-apic1#moquery-c vzRtCons-f vz.RtCons.dn*tn-m

115、g-cisco-live/brc-ip|grep dndn:uni/tn-mg-cisco-live/brc-ip/rtfvCons-uni/tn-mg-cisco-live/ap-a1/epg-e1asav-cisco-live#capture pbr_traffic interface inside real-time match icmpUse ctrl-c to terminate real-time capture0 packets shown.0 packets shown.Check contract provider and consumer of both contracts

116、Check contract provider and consumer of both contractsa-apic1#moquery-c vzRtProv-f vz.RtProv.dn*tn-mg-cisco-live/brc-icmp|grep dndn:uni/tn-mg-cisco-live/brc-icmp/rtfvProv-uni/tn-mg-cisco-live/ap-a1/epg-e2a-apic1#moquery-c vzRtCons-f vz.RtCons.dn*tn-mg-cisco-live/brc-icmp|grep dndn:uni/tn-mg-cisco-li

117、ve/brc-icmp/rtfvCons-uni/tn-mg-cisco-live/ap-a1/epg-e1/brc-icmp/rtfvProv-uni/tn-mg-cisco-live/ap-a1/epg-e2Contract“icmp”(standard contract)Contract“ip”(PBR contract)Both EPGs have contracts”ip”and“icmp”associated.More specific filter is preferred!(ICMP in this case)2023 Cisco and/or its affiliates.A

118、ll rights reserved.Cisco Public#CiscoLiveCommon PBR Problems#3:Traffic is working but#3:Traffic is working but does not hit redirect ruledoes not hit redirect ruleBRKDCN-361559asav-cisco-live#capture pbr_traffic interface inside real-time match icmp any any1:17:40:29.668056 192.168.1.100 192.168.2.1

119、00 icmp:echo request 2:17:40:29.668224 192.168.1.100 192.168.2.100 icmp:echo request 3:17:40:29.668681 192.168.2.100 192.168.1.100 icmp:echo reply 4:17:40:29.668712 192.168.2.100 192.168.1.100 icmp:echo reply 1)Remove ICMP contract between EPGs so“IP”filter takes over and ping traffic is redirected1

120、)Remove ICMP contract between EPGs so“IP”filter takes over and ping traffic is redirected2)Reconfigure filters on contracts to match specific types of traffic to be redirected2)Reconfigure filters on contracts to match specific types of traffic to be redirectedFIXES:FIXES:#4:PBR Packet Never makes i

121、t to PBR Node with Redirect Rule in Place 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommon PBR Problems#4:Packet never makes it to PBR Node with redirect rule in place#4:Packet never makes it to PBR Node with redirect rule in placeBRKDCN-361561a1-leaf2#show system in

122、ternal epm endpoint ip 172.16.1.100 MAC:0050.56a8.4897:Num IPs:1IP#0:172.16.1.100:IP#0 flags:host-tracked|:l3-sw-hit:Yes:flags2:Vlan id:32:Vlan vnid:14214:VRF name:mg-cisco-live:v1BD vnid:16744311:VRF vnid:2293762asav-cisco-live#capture pbr_traffic interface inside real-time match icmp any anyUse ct

123、rl-c to terminate real-time capture0 packets shown.Things to check:Things to check:Check Service Node EP learningValidate with PBR capture you do not see traffic on service nodea1-leaf1#show zoning-rule scope 2293762|grep redir|4325|16386|32771|1|uni-dir-ignore|enabled|2293762|redir(destgrp-3)|fully

124、_qual(7)|4351|32771|16386|1|bi-dir|enabled|2293762|redir(destgrp-3)|fully_qual(7)Validate if redirect rule is there on ingress leaf 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommon PBR ProblemsThings to check:Things to check:BRKDCN-361562a1-leaf2#iping-V mg-cisco-liv

125、e:v1 172.16.1.100-S 172.16.1.1PING 172.16.1.100(172.16.1.100)from 172.16.1.1:56 data bytes64 bytes from 172.16.1.100:icmp_seq=0 ttl=255 time=1.044 ms64 bytes from 172.16.1.100:icmp_seq=1 ttl=255 time=1.027 ms64 bytes from 172.16.1.100:icmp_seq=2 ttl=255 time=0.979 ms64 bytes from 172.16.1.100:icmp_s

126、eq=3 ttl=255 time=0.888 ms64 bytes from 172.16.1.100:icmp_seq=4 ttl=255 time=0.955 msCan service node even reach the PBR NH?FTRIAGE to see where packet is lost2023-05-22 10:57:20,535 INFO ftriage:main:1295 L3 packet Seen on a1-spine1 Ingress:Eth2/24 Egress:LC-2/2 FC-22/0 Port-1 Vnid:16744311 2023-05

127、-22 10:58:37,770 ERROR ftriage:fib:727 a1-spine1:EP not found in COOP!for VRF VNID:167443112023-05-22 10:58:47,735 ERROR ftriage:unicast:2167 a1-spine1:EP is unknown in COOP.Ftriage will exit but continue with further fault isolation2023-05-22 10:58:47,735 INFO ftriage:unicast:2207 a1-spine1:Egress

128、node not provided.Cannot check local EP.Exiting!#4:Packet never makes it to PBR Node#4:Packet never makes it to PBR NodeWe have reachability from service node BD SVISpine gets the packet but EP lookup for redirect MAC in COOP fails!2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#Ci

129、scoLiveCommon PBR ProblemsThings to check:Things to check:BRKDCN-361563#4:Packet never makes it to PBR Node#4:Packet never makes it to PBR Nodeasav-cisco-live#show int gigabitEthernet MAC address 0050.56a8.4897,MTU 1500Is the redirect IP and MAC of service device set correctly?*FW MACRedirect MAC is

130、 Redirect MAC is configured configured incorrectly!incorrectly!*For releases later than APIC Release 5.2,the MAC address configuration is not mandatory for L3 PBR if IP-SLA tracking is enabled)*2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCommon PBR ProblemsBRKDCN-36156

131、4#4:Packet never makes it to PBR Node#4:Packet never makes it to PBR NodeFIX:Correct NH MAC address under Tenants FIX:Correct NH MAC address under Tenants-Policies Policies-Protocol Protocol-L4/L7 Policy L4/L7 Policy-Based RedirectBased Redirect 2023 Cisco and/or its affiliates.All rights reserved.C

132、isco Public#CiscoLiveCommon PBR ProblemsBRKDCN-361565#4:Packet never makes it to PBR Node#4:Packet never makes it to PBR NodeAfter correcting last octet of FW MAC tracking is seen on PBR node as expectedAfter correcting last octet of FW MAC tracking is seen on PBR node as expectedmg-asav-cisco-live#

133、capture pbr_traffic interface inside real-time match icmp any anyUse ctrl-c to terminate real-time capture1:15:06:07.671870 192.168.1.100 192.168.2.100 icmp:echo request 2:15:06:07.671916 192.168.1.100 192.168.2.100 icmp:echo request 3:15:06:07.672297 192.168.2.100 192.168.1.100 icmp:echo reply 4:15

134、:06:07.672313 192.168.2.100 192.168.1.100 icmp:echo reply Resources 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveLeafLeaf-2 267BRKDCN-36151FromFrom LeafLeaf-1 to fabric1 to fabric PACKET REWRITE PACKET REWRITE(EP(EP-2 is known to Leaf2 is known to Leaf-1)1)After packet

135、rewrite,Spine does MAC COOP Lookup in Service BDAfter packet rewrite,Spine does MAC COOP Lookup in Service BD2534After FW,between FW leaf and destination leaf 301After FW,between FW leaf and destination leaf 301From EP1 to LeafFrom EP1 to Leaf-1 1Packet FormatDIPDIPLEAFLEAF-3 PTEP3 PTEPSIPSIPLEAFLEA

136、F-2 PTEP2 PTEPSCLASSSCLASSSHADOW EPGSHADOW EPGVNIDVNIDVRFVRFOuter Outer DCLASSDCLASSEPG2EPG2Pod1Pod1DIPDIP192.168.2.100192.168.2.100SIPSIP192.168.1.100192.168.1.100DMACDMACEP2EP2SMACSMACLEAF MACLEAF MACInner Inner Pod1Pod1DIPDIP192.168.2.100192.168.2.100SIPSIP192.168.1.100192.168.1.100DMACDMACLEAF M

137、ACLEAF MACSMACSMACEP1EP1Inner Inner DIPDIPLEAFLEAF-2 PTEP2 PTEPSIPSIPSPINE PTEPSPINE PTEPSCLASSSCLASSEPG1EPG1VNIDVNIDService BDService BDOuter Outer DCLASSDCLASSShadow EPGShadow EPGPod1Pod1DIPDIP192.168.2.100192.168.2.100SIPSIP192.168.1.100192.168.1.100DMACDMACLEAF MACLEAF MACSPINE MACSPINE MACInner

138、 Inner DIPDIPSpine Anycast MACSpine Anycast MACSIPSIPLeaf101 PTEPLeaf101 PTEPSCLASSSCLASSEPG1EPG1VNIDVNIDService BDService BDOuter Outer DCLASSDCLASSEPG2EPG2Pod1Pod1DIPDIP192.168.2.100192.168.2.100SIPSIP192.168.1.100192.168.1.100DMACDMACFW MACFW MACSMACSMACEP1EP1Inner Inner Coming back from FW on Le

139、afComing back from FW on Leaf-2 2Pod1Pod1DIPDIP192.168.2.100192.168.2.100SIPSIP192.168.1.100192.168.1.100DMACDMACLEAF MACLEAF MACSMACSMACFW MACFW MACInner Inner EP1EP2LeafLeaf-1 1LeafLeaf-3 3FW172.16.1.100EP2 EPG2192.168.2.100SpineSpine-2 2SpineSpine-1 1EP1 EPG1192.168.1.100EP212345PBR Datapath ELAM

140、s(via Leaf CLI)EP1EP2LeafLeaf-1 1LeafLeaf-3 3FW172.16.1.100EP2 EPG2192.168.2.100SpineSpine-2 2SpineSpine-1 1EP1 EPG1192.168.1.100LeafLeaf-2 2*Note for these ELAMs were on*Note for these ELAMs were on EX leaf(“EX leaf(“tahtah”),”),asicasic family family is different for other is different for other s

141、witches*switches*For FX and FX2 For FX and FX2 Use“roc”Use“roc”For GX=Use“app”For GX=Use“app”For GX2=Use“For GX2=Use“chocho”On leaf from front panel Port(ingress from server,or back from firewall)debug platform internal tah elam asic 0 trigger resettrigger init in-select 6 out-select 1set outer ipv4

142、 src_ip 192.168.1.100 dst_ip 192.168.2.100set outer l2 src_mac On Spine or egress leaf Before Redirect(for example if EP is unknown in ingress leaf)debug platform internal roc elam asic 0 trigger resettrigger init in-select 14 out-select 1set inner ipv4 src_ip 192.168.1.100 dst_ip 192.168.2.100set o

143、uter l4 tn-seg-id 0 x2e001(VRF VNID in HEX)On Spine or egress leaf after redirect and before Firewalldebug platform internal tah elam asic 0 trigger resettrigger init in-select 14 out-select 1set inner ipv4 src_ip 192.168.1.100 dst_ip 192.168.2.100set outer l4 tn-seg-id 0 xe27fef(service BD VNID)On

144、leaf from front panel Port(ingress from server,or back from firewall)debug platform internal tah elam asic 0 trigger resettrigger init in-select 6 out-select 1set outer ipv4 src_ip 192.168.1.100 dst_ip 192.168.2.100set outer l2 src_mac On spine or egress leaf(ingress from server,or back from firewal

145、l)debug platform internal tah elam asic 0 trigger resettrigger init in-select 14 out-select 1set inner ipv4 src_ip 192.168.1.100 dst_ip 192.168.2.100set inner l2 src_mac BRKDCN-361568 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLivePBR White Paper:PBR White Paper:https:/ c

146、-hIdhId-293703813293703813ACI MultiACI Multi-Pod and Service Node Integration White PaperPod and Service Node Integration White Paperhttps:/ MultiACI Multi-Site and Service Node Integration White PaperSite and Service Node Integration White Paperhttps:/ ACI PolicyTroubleshoot ACI Policy-Based Redire

147、ctBased Redirecthttps:/ APIC Layer 4 to Layer 7 Deployment Guide,Release 5.0(x)Cisco APIC Layer 4 to Layer 7 Deployment Guide,Release 5.0(x)https:/ 5-x/l4x/l4-l7l7-services/ciscoservices/cisco-apicapic-layerlayer-4 4-toto-layerlayer-7 7-servicesservices-deploymentdeployment-guideguide-50 x/m_selecti

148、ng_a_layer_4_to_layer_7_device_to_render_a_graph.html50 x/m_selecting_a_layer_4_to_layer_7_device_to_render_a_graph.htmlCisco Live OnCisco Live On-Demand LibraryDemand Libraryhttps:/ 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveFill out your session surveys!Attendees wh

149、o fill out a minimum of four session surveys and the overall event survey will get Cisco Live-branded socks(while supplies last)!These points help you get on the leaderboard and increase your chances of winning daily and grand prizesAttendees will also earn 100 points in the Cisco Live Game for ever

150、y survey completed.BRKDCN-361570 2023 Cisco and/or its affiliates.All rights reserved.Cisco PublicContinue your educationVisit the Cisco Showcase for related demosBook your one-on-oneMeet the Engineer meetingAttend the interactive education with DevNet,Capture the Flag,and Walk-in LabsVisit the On-D

151、emand Library for more sessions at www.CiscoL you#CiscoLive 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLive73Gamify your Cisco Live experience!Get points Get points for attending this session!for attending this session!Open the Cisco Events App.Click on Cisco Live Challenge in the side menu.Click on View Your Badges at the top.Click the+at the bottom of the screen and scan the QR code:How:123473 2023 Cisco and/or its affiliates.All rights reserved.Cisco PublicBRKDCN-3615#CiscoLive