This is a summary of the notes I’ve written for CCIE-EI - CCIE - EI: 4.0 Security and Services Pt. 2. In other words, this only contains what I felt the need to write down and is not meant as a complete study resource. Please see the study resources I’ve used or related blogs for more coherent writeups.

4.0 - Infrastructure Security and Services Pt. 2

4.5 Network Services

4.5.a First-Hop Redundancy Protocols

HSRP, GLBP, VRRP

GLBP

Gateway Load Balancing Protocol. Achieving HA load-balancing through ARP trickery and multiple mac addresses.

Active Virtual Gateways(AVG) assign MAC addresses to Active Virtual Forwarders(AVF) and responds to ARP for the shared virtual IP address. The AVG allocates up to 4 MAC addresses per group to AVFs. GLBP routers can either be in Speak, Standby or Active state for a given group.

GLBP elects a standby AVG according to the highest glbp priority, or highest IP address. The standby AVG is responsible for replacing the AVG in case of failure. The GLBP AVG does not preempt by default.

When an AVF fails a listening secondary virtual forwarder will adopt the MAC address and replace the old AVF. The old MAC address will be used for ARP for the duration of the redirect-timer. Once the secondary holdtime has passed the old AVF will be removed from the group.

Weighting and tracking controls weighting for the load-balancing(number of devices) and eventually cut-off when the weight falls below a configured treshold. GLBP AVFs preempts once the configured treshold has been passed for the duration of the forwarder preempt delay.

GLBP supports SSO. ISSU-GLBP enables SSO while different IOS versions running.

Packets:

• Hello: Multicas, protocol information & keepalive
• Request & Reply: Unicast, MAC address assignement

Transport: 224.0.0.102, UDP/3222

Timers: Hello - 3s
Hold - 10s
Redirect - 10m, how long the AVG will direct clients to a given AVF MAC
Redirect timeout - 4h max age for an AVF
Forwarder preempt delay - 30s,

Preempt:

• AVG - No by default
• AVF - Yes by default

Authentication

• Plaintext
• MD5

int> glbp {group} ip {address}
int> glbp {group} name {group-name}
int> glbp {group} load-balancing [host-dependent | round-robin | weighted]

! AVG timers
int> glbp {group} timers [msec] {hello} [msec] {hold}
! AVF timers
int> glbp {group} timers redirect {redirect} {timeout}

! GLBP AVG election configuration
int> glbp {group} priority {priority}
int> glbp {group} preempt [delay minimum seconds]

! Auth, key-chain allows for roll-over
int> glbp {group} authentication md5 key-string {0|7} {pass}
int> glbp {group} authentication md5 key-chain {key-chain-name}

! AVF weight/treshold configuration
glbp group weighting maximum [lower {val}] [upper {val}]
glbp group weighting track object-number [decrement {value}]
glbp group forwarder preempt [delay minimum {seconds}]

! Enables the GLBP client cache
int> glbp {group} client-cache maximum {number} [timeout {minutes}]

VRRP

Standards based FHRP. Can manage multiple IP addresses(primary, secondary) per “virtual router”. Millisecond timers for VRRP is a Cisco-only feature. Supports SSO. The VRRP VIP can match the address of an interface. Like HSRP object tracking can be used to trigger change of master, if decrement isn’t specified the default value of 10 will be used.

Transport: IP/112 to 224.0.0.18

Timers:

Preempt: Preempts by default, priority can be set 1-254(100 default).

Authentication: MD5 only

VRRPv2

int> vrrp {group} ip {VIP}
int> vrrp {group} priority {1-254}
int> vrrp {group} track {track-num} [decrement {value}]
int> vrrp {group} authentication text {pass}
int> vrrp {group} timers advertise {}

! Learn timers from incoming packets when acting as backup
int> vrrp {n} timers learn

! Preempt delay, can reduce flapping 
int> vrrp 10 preempt delay minimum {seconds}

VRRPv3

Supports both IPv4 and IPv6 and introduces VRRP pathways. Does not support SSO. Does not support authentication. Allows up to 255 different groups on an interface Same transport as VRRPv2(ipv6 address is ::12). The VRRPv3 standard supports millisecond timers.

When VRRPv3 is enabled globally, “normal” VRRPv2 configuraiton becomes unavailable. VRRPv2 compatability can however be configured under VRRPv3 interface configuration.

fhrp version vrrp v3
int> vrrp {group} address-family {ipv4|ipv6}
int-vrrp> address {address} {primary|secondary}
int-vrrp> timers advertise {msec} ! 1000ms default
int-vrrp> vrrpv2 ! Enable v2 compatability 

! Delay before FHRP starts after interface comes up
int> fhrp delay {sec}

Redundancy using IPv6 RS/RA

The show ipv6 interface {} command includes most information sent in RA messages. The sending of RA messages is triggered with ipv6 unicast-routing

Router preference: Which gateway to use, higher is better. Notably “High” is 01 while “Low” is 11, which in reality makes the lowest value preferred.

ipv6 nd router-preference {high|low}

Timers:

• Advertisement interval, random value between max and min to avoid sync (200s)

ipv6 nd ra interval {max [min]} | msec {max [min]}

• Router lifetime, maximum time the router should be considered usable as a gateway (5m)

ipv6 nd ra {lifetime} {seconds}

• Valid lifetime, maximum time an address remains valid (30d)
• Preferred lifetime, length of time an address is preferred (7d)

ipv6 nd prefix {pref/len} {valid-time} [preferred-time]

4.5.b Network Time Protocol

The software and hardware clock are separate. In a NTP network the software clock should be updated through NTP and the hardware-clock should regularly be synced with the software clock. The purpose of the hardware clock is to maintain time between boots.

! Software clock
clock timezone {zone} {offset}
clock summer-time {zone} recurring [time of change]
clock set {}

!! Hardware clock
! Enables the hardware clock as a valid time source
clock calendar-valid
! Syncs software with hardware clock
clock read-calendar
! Syncs hardware clock with software clock
clock update-calendar
! Show hardware clock
show calendar 
! show software clock
show clock

Modes:

• Client
• Server
• Symmetric active(mutual peering)

Master, client

The lowest stratum value will be preferred. If multiple servers have the same stratum, the prefer keyword can be used to prefer one of them.

Poll based

! Server
ntp master
ntp source {int}
clock timezone {}
clock calendar-valid ! Hardware clock must be trusted
ntp peer {ip} [version {}] [prefer]

! Client
ntp server....

Broadcast based NTP

Usually not preferred. The benefit of using broadcast NTP is scale(no polling needed)

! Server
ntp master
int> ntp broadcast 

! Client
ntp broadcast version {}
ntp broadcastdelay {}
int> ntp broadcast client

Orphan mode

NTP Orphan mode allows a group of clients to stay in sync with eachother after loss of contant with primary NTP servers.

Full mesh orphan peerings are created for “core” orphan servers, while orphan children are only connected to servers. Clients of the child servers are only connected to their respective server.

ntp server {ip}
ntp peer {ip} 
ntp orphan {stratum}

Access groups

Permissions can be limited by peer IP address through ntp access-group. This is usually not necessary when authentication is in use.

Types:

• ipv4|6 - all or nothing!
• peer - allows time requests and queries, allows full sync
• serve - allows time requests and queries, doesn’t allow sync
• serve-only - only allows time-requests
• query-only - only allows control-queries

Authentication

NTP Authentication is enabled with ntp authenticate, other auth config doesn’t apply until this is enabled. Multiple keys can be trusted inbound but only one used outbound per server.

The following authentication mechanisms are supported:

• md5
• cmac-aes-128
• hmac-sha1
• hmac-sha256

ntp authenticate
ntp authentication-key {n} {type} {key}
ntp trusted-key {num} [ - {num}] ! Allow inbound
ntp server {ip} key {num}

4.5.c DHCP on Cisco IOS

Client, server, relay

Client

int {}
int> ip dhcp client client-id ascii my-test1
int> ip dhcp client class-id my-class-id
int> ip dhcp client lease 0 1 0
int> ip dhcp client hostname sanfran
int> no ip dhcp client request tftp-server-address
int> ip address dhcp

! Specify client options
int> ip dhcp client option {num} {type} {data}

! Specify which options to request
int> ip dhcp client request option {options}

Server

The DHCP lease database can be stored on shared storage with ip dhcp database {url}. This is highly recommended by Cisco.

Logging based on utilization can be achieved with utilization mark {high|low} {percent} log.

ip dhcp pool {pool-name}
dhcp-pool> network {network-id} {cidr}
ip dhcp excluded-address {start-range} {end-range}

Reservations

Reservations can either be created through adding host-pools to the configurations, or as static-mappings added to separate file. The only practical difference is how clean the config file is.

Reservation through dhcp pool:

ip dhcp pool {reservation-name}
dhcp-pool> host {ip-addr}
dhcp-pool> client-name {name}
! Choose one based on whether the
! client includes an identifier.
dhcp-pool> client-identifier {}
dhcp-pool> hardware-address {}  

Reservations through static mapping:

!File format example
*time* Jan 21 2005 03:52 PM
*version* 2
!IP address    Type    Hardware address     Lease expiration
10.0.0.4 /24   1       XXXX.YYYY.ZZZZ 	    Infinite
10.0.0.5 /28   id      XXXX.YYYY.ZZZZ 	    Infinite
10.0.0.2 /21   1       XXXX.YYYY.ZZZZ       Infinite
*end*

! Configuring static mapping
ip dhcp pool {name}
dhcp-pool> origin file {url}

Secondary subnets

Secondary subnets can be configured for a DHCP pool. If secondary subnets exist in a pool, the following rules apply to incoming request:

• The server will look in the primary subnet first, unless the GIADDR of the message matches a secondary subnet.
• When the primary subnet is exhausted, the dhcp server automatically starts assigning addresses from the secondary subnet in the order they were added to the pool.

Option 82

Option 82 can either be used in conjunction with the GIADDR field or by itself. GIADDR is most often set to the broadcast address of the subnet. To assign an IP based on option 82 a device must be mapped to a class, which in turn has it’s own address range defined.

Matching by option 82 only:

ip dhcp class {name}
dhcp-class> relay agent infromation
dhcp-class-relayinfo> relay information hex {pattern}

ip dhcp use class 
ip dhcp pool {name}
dhcp-pool> network {}
dhcp-pool> class {name}
dhcp-pool-class> address range ...

Operation options

! Settings for ping-check before 
! assigning an IP address.
ip dhcp ping packets {}
ip dhcp ping timeout {}

! Ignore bootp, unless forwarded
ip dhcp bootp ignore {} 

Verification

show ip dhcp pool [name]
show ip dhcp binding [address]
show ip dhcp conflict [address]
show ip dhcp database [url]
show ip dhcp server statistics 

Relay

Relay agent sets the GIADDR field and optionally option 82, the source is set to the local subnet broadcast address. Supported with unnumbered interfaces. Achieved with dynamically created static-routes. The dhcp relay functionality is enabled through the ip helper-address interface command. The source address/GIADDR can be set statically through ip dhcp-relay source-interface {}.

Option 82 - Relay information

Inserted by the relaying device. Allows allocating IP addresses by more than just the GIADDR. By default relay-agents will replace option 82 when it receives a dhcp request where it is already set. Unless an interface is configured as trusted, option 82 will be dropped from packets without GIADDR set.

ip dhcp relay information option
ip dhcp relay information policy {drop|keep|replace|encapsulate}

! Set all interfaces as trusted
ip dhcp relay information trust-all

! Configure static subscriber-id for a port
ip dhcp relay information option subscriber-id {string}

! Support for MPLS VPNs
ip dhcp relay information option vpn

The server-id option 82 suboption defines where the DHCP client sends all release and renew packets. This option can be overriden such that the clients never contact the DHCP server directly.

The link selection option 82 suboption allows having separate subscriber subnet and GIADDR. When it is set the DHCP server will use the link selection option instead of the GIADDR to assign leases.

! Enable insertion of server-id + link selection
ip dhcp-relay information option server-override
! Specify source interface for relayed packets
ip dhcp-relay source-interface

IPv6 Relay

IPv6 DHCP relay can use the remote-ID option to provide the DHCPv6 server with information on port, DUID and VLAN. Unlike option 82 in DHCPv4 the remote-ID option isn’t echoed back to the sender.

Relay agents that relay DHCPv6 RELAY-REPLY messages will inspect for the prefix-delegation option and add a static route to the prefix to the local route table.

IPv6 relay doesn’t include many nerd-knobs, which is probably a good thing.

int> ipv6 dhcp relay destination {ipv6-addr} [int]

! Enables VRF aware DHCP relay
ipv6 dhcp relay option vpn

Options

! Option 51
dhcp-pool> lease {days}

! Option 3
dhcp-pool> default-router {address}

! Option 15
dhcp-pool> domain-name {domain}

! Option 6
dhcp-pool> dns server {ip}

! Option 67
dhcp-pool> bootfile {}

! SIADDR, next server in bootstrap chain
dhcp-pool> next-server {}

! To set less common DHCP options:
dhcp-pool> option {n} hex {data}

SLAAC/DHCPv6 interaction

The client will always generate a SLAAC address if the A flag is set in the RA message, even if the M flag is set. To look for DNS servers++ in stateless DHCP the O flag needs to be set.

! Disable A flag
ipv6 nd prefix {prefix} no-autoconfig 
! Enable O flag
ipv6 nd other-config-flag
! Enable M flag
ipv6 nd managed-config-flag

Stateful, stateless DHCPv6

Stateful DHCP configuration

ipv6 dhcp pool {}
dhcp-pool> prefix {}

int> ipv6 dhcp server {pool}
! Disable autoconfiguration(SLAAC)
int> ipv6 nd prefix {prefix} no-autoconfig 
! Enable the client to look for settings in DHCP
int> ipv6 nd other-config-flag
! Allow DHCP to delegate an IP address. 
int> ipv6 nd managed-config-flag

! Add static route to delegated addresses
ipv6 dhcp iana-route-add

Stateless DHCP configuration

! Configure dhcp pool
ipv6 dhcp pool {name}
dhcp-pool> dns-server {}
dhcp-pool> domain {}

int> ipv6 dhcp server {name}
! Make clients look for DHCP
int> ipv6 nd other-config-flag

DHCPv6 Prefix Delegation

Server configuration

! Static prefix
dhcp-pool> prefix-delegation {prefix}
! or named prefix pool
dhcp-pool> prefix-delegation pool {name}

! Add static route to delegated prefix
ipv6 dhcp iapd-route-add
! Default on relay agents

Client configuration

! Enable PD requests
int> ipv6 dhcp client pd {name} 
! Request specific prefix 
int> ipv6 dhcp client pd hint {prefix} 

! Assign IP in delegated prefix
int> ipv6 address {prefix-name} ::{}/64

4.5.d IPv4 Network Address Translation

Static NAT, PAT

! Source NAT
ip nat inside source static {inside-local} {inside-global}

! Source PAT
ip nat inside source static [tcp|udp] {inside-local} {local-port} {inside-global} {global-port}

! Destination NAT

! Outside NAT

Dynamic NAT, PAT

! Dynamic NAT
ip nat pool {name} {start-addr} {end-addr} prefix-length {len}
ip nat inside source list {acl} pool {pool-name}

! Dynamic PAT
ip nat inside source list {acl} int {} overload
! Or 
ip nat pool {name} {start-addr} {end-addr} prefix-length {len}
ip nat inside source list {acl} pool {pool-name} overload

Policy-based NAT, PAT

Uses a route-map as a condition for NAT-ing

route-map {name}....
ip nat .... route-map {name} 

NAT hairpinning isn’t policy-based NAT, but it does at least use policy routing. IOS-XE doesn’t support NVI, making this more complex than it used to be.

NAT hairpinning example:

access-list {acl-name}
! ^ Match all inside addresses
route-map {rm-name} permit
route-map> match {acl-name}
route-map> set-interface {int}
! ^ The inside interface you want to reach

int {}
int> ip nat inside

int {}
int> ip nat outside 
int> ip policy {rm-name}

VRF aware NAT, PAT

VRF to GRT NAT doesn’t differ from reglar GRT-only NAT configuration, hence is not included.

NVI based NAT differs from domain based NAT(regular) in one significant way: it is always routed BEFORE the NAT-ing occurs. In domain based NAT the outbound traffic is routed before NAT, and the inbound traffic is routed after NAT. The result of this is that the VRFs that are to be routed between needs to have routing to eachother.

NOTE: IOS-XE Does NOT support overload when using match-in-vrf

!! GRT to VRF
int Gi1
int> ip nat inside 

int Gi2
int> ip vrf forwarding VRF-A
int> ip nat outside

ip nat source .... int Gi2 vrf VRF-A


!! Intra-VRF
int Gi1
int> ip vrf forwarding VRF-A
int> ip nat inside 

int Gi2
int> ip vrf forwarding VRF-A
int> ip nat outside

ip nat source ... int Gi2 vrf VRF-A match-in-vrf 


!! Inter-VRF with NVI(IOS)
int Lo1
int> ip vrf forwarding VRF-A
int> ip address 10.1.1.1 255.255.255.255

int Lo2 
int> ip vrf forwarding VRF-B
int> ip address 10.2.2.2 255.255.255.255

int Gi1
int> ip vrf forwarding VRF-A
int> ip address 10.1.0.1 255.255.255.255
int> ip nat enable

int Gi2
int> ip vrf forwarding VRF-B
int> ip address 10.2.0.1 255.255.255.255
int> ip nat enable

! Add route from VRF-A to VRF-B address via GRT
ip route vrf VRF-A 10.2.2.2 255.255.255.255 10.2.0.1 global
ip route 10.2.0.0 255.255.255.0 int Gi2

! Add route from VRF-B to VRF-A address via GRT
ip route vrf VRF-B 10.1.1.1 255.255.255.255 10.1.0.1 global
ip route 10.1.0.0 255.255.255.0 int Gi1


!! Inter-VRF without NVI(IOS-L2)
! Let me know if there's a way to do this...

IOS-XE VRF-Aware Software Infrastructure (VASI) NAT

VRF Aware Sofware Infrastructure allows you to “directly connect” two VRFs through virtual interfaces called vasileft and vasiright. The same-numbered vasileft and vasiright interface area connected.

int Lo0
int> ip vrf forwarding VRF-A
int> ip address 10.1.1.1 255.255.255.255

int vasileft 1 
int> ip vrf forwarding VRF-A
int> ip address 10.0.0.0 255.255.255.254
int> ip nat inside 

int vasiright 1 
int> ip vrf forwarding VRF-B
int> ip address 10.0.0.1 255.255.255.254
int> ip nat outside

int Lo1
int> ip vrf forwarding VRF-B
int> ip address 10.2.2.2 255.255.255.255

ip route vrf VRF-A 10.2.2.2 255.255.255.255 vasileft 1 10.0.0.0
ip route vrf VRF-B 10.1.1.1 255.255.255.255 vasiright 1 10.0.0.0
! You can get by with a static route in the "out" direction.

ip nat inside source ... vrf VRF-B

4.6 Network optimization

4.6.a IP SLA

Things that can be measured:

• Delay (both round-trip and one-way)
• Jitter (directional)
• Packet loss (directional)
• Packet sequencing (packet ordering)
• Path (per hop)
• Connectivity (directional)
• Server or website download time
• Voice quality scores

IP SLA Responders rely on IP-SLA Control Protocol from the source to function. IP-SLA Control protocol can be MD5 authenticated. Static ports for the responder can be configured IP SLA Control Protocol isn’t functioning/supported on the sending/receiving device.

The source and target devices both need common NTP server to achieve accurate measurements for all single-direction responder dependent probes. IP SLA overcomes processing delays through using timestamps pre and post processing. This is only available with IP SLA Responders.

Probe

threshold = maximum time for calculating statistics timeout = meximum time for probe frequency = time between each probe run

! Source
ip sla {num}
ip-sla> {type}
ip-sla-jitter> {parameters/options
! Target
ip sla responder

verify-data can be enabled under the ip sla configuration to prevent data corruption.

Scheduling

! Schedule single probe/opertion
ip sla schedule {num} [life {forever | seconds }] [start-time {[hh: mm: ss ] [month day | day month ] | pending | now | after hh: mm: ss }] [ageout {seconds}] [recurring]

Multiple probes/operations can be scheduled at the same time using the ip sla group command. All probes are run within the configured schedule-period and the frequency defaults to the value of schedule-period. Random frequency is recommended when scheduling ip sla probes as a group. This results in more better statistics(in theory). Random schduling is enabled by setting frequency range {min_sec- max_sec}.

ip sla group schedule {group-num} {probe-nums} {schedule-period {schedule-period-range | schedule-together }} frequency ....
show ip sla group schedule

Note: Any IP-SLA with start-time now must be manually restarted after a reload.

Thresholds

ip sla reaction-configuration {reaction-num} react {monitored stat} [action-type option ] [threshold-type {average [number-of-measurements ] | consecutive [occurrences ] | immediate | never | xofy [x-value y-value ]}] [threshold-value upper-threshold lower-threshold ]

! Start IP SLA operation upon triggered reacion
ip sla reaction-trigger {reaction-num} {reaction-operation}

! Enable the device to send SNMP traps from IP SLA Reactions
snmp-server enable traps rtr
snmp-server enable traps syslog

Verification

show ip sla configuration
show ip sla group schedule
show ip sla statistics
show ip sla statistics details
debug ip sla error
debug ip sla trace

ICMP probes

ICMP probes allow for inter-vendor probing as they don’t require a responder on the destination device. UDP jitter and echo probes are preferred over ICMP unless specifically measuring successive packet loss stats, out-of-sequence stats or running path echo probes.

ICMP Jitter

UDP Jitter is preferred for cisco-cisco probes due to improvements in accuracy achieved by using the responder.

ICMP Jitter measures:

• Jitter (source-to-destination and destination-to-source)
• Latency (source-to-destination and destination-to-source)
• Round-trip time latency
• Packet loss
• Successive packet loss
• Out-of-sequence packets (source-to-destination, destination-to-source, and round-trip)
• Late packets

ICMP Timestamp Request (Type 13) and an ICMP Timestamp Reply (Type 14) to achieve directional stats.

ICMP Path jitter can be configured to achieve hop-by-hop jitter testing along the path.

ip-sla> path-jitter 172.31.1.129 source-ip 10.2.30.1 num-packets 12 interval 22

ICMP Echo

The ICMP Echo probe only measures RTT.

ip-sla> icmp-echo {destination-ip-address | destination-hostname } [source-ip {ip-address | hostname } | source-interface interface-name ]

ICMP Path echo can be used to measure hop-by-hop RTT along the path.

ip-sla> path-echo {destination-ip-address | destination-hostname } [source-ip {ip-address | hostname } | source-interface interface-name ]

UDP probes

Jitter probe

Jitter probes can detect:

• per-direction jitter
• per-direction packet-loss
• per-direction delay
• rtt

ip-sla> udp-jitter {destination} {port} [interval] [num-packets]
ip-sla-jitter> request-data-size  	! 10ms
ip-sla-jitter> response-data-size 	! equal to request if not specified

UDP echo

Only checks RTT, reuquires responder.

ip-sla> udp-echo {destination} {port} [source {}] 

Multicast

UDP Jitter probes can be run over multicast by configuring endpoint lists for unicast control communication with the responders. This requires that responders have been configured and are successfully igmp joined before starting.

ip sla endpoint-list type ip {name}
ip-sla-epl> ip-address {address} port {port} ! Unicast addresses of responders

ip-sla> udp-jitter {mcast ip} {port} endpoint-list {name} source-ip {} ...

show ip sla endpoint-list

TCP probes

TCP Connect

TCP Connect probes measures the time to establish a TCP session. The destination can either be a Cisco device with an IP SLA responder or any device running a TCP service

ip-sla> tcp-connect {destination} {port} [source {ip|hostname} [source-port {port}]] 

HTTP

HTTP probes consist of 3 types of measurements:

• DNS lookup­RTT
• TCP Connect
• HTTP transaction time

The http operation goes down when the HTTP code isn’t 200 or the get request otherwise fails. http-raw-request can be used to specify all http parameters.

http get {url} [name-server {ip}] [version version-number ] [source-ip ...] [proxy proxy-url]

4.6.b Tracking object

track {n} ip sla  
track {n} ip route
track {n} int {} line-protocol
track {n} int {} ip routing

Multiple tracked objects can be tracked in a list with track {n} list {bool|treshold}. Stub tracked objects for use with EEM can be created with track {n} stub-object.

4.6.c Flexible Netflow

Tracks flows(streams of packets between a set of source and destination. A unique flow record is created per unique combination of source, destinaton, protocol, ports etc. Flexible netflow allows flow keys to be defined.

Netflow versions:

• v5 - multi-vendor, ipv4 only
• v9 - template-based, multi-protocol
• IPFIX - IETF version
• Flexible - An extension of v9

Netflow components:

• Flow Records, a unique combination of key and non-key fields(defining a flow). All flow records are assigned to flow monitors.
• Flow Monitors, applied to an interface to enable monitoring. Can be different between ingress and egress interface. Records are exported before they’re aged out.
• Flow Exporters, ships flow-records to an external system for analysis.
• Flow Samplers, used to limit/specify which traffic to collect.

! MVP
flow-monior {}
flow-monitor> record netflow ... 
int> ip flow monitor 

! Flow records/UDR
flow record {name}
flow-record> match {} ! Define key field
flow-record> collect {} ! Define non-key field

! Configure exporter
flow exporter {name}
flow-exporter> destination {ip}
flow-exporter> export-protocol {netflow-v5|netflow-v9|ipfix}
flow-exporter> transport ...

! Create monitor
flow monitor {name}
flow-monitor> record {name}
flow-monitor> exporter {name}
flow-monitor> cache ...

! Apply monitor to interface
int> ip[v6] flow monitor {name} {input|output}

4.7 Network operations

4.7.a Traffic capture

SPAN

monitor session {num} filter {acl}
monitor session {num} source int {} [rx|tx|both]
monitor session {num} destin int {}

RSPAN

Forwarding all captured frames through a VLAN, to be analyzed/captured on another switch. RSPAN is also supported on IOS-XE routers.

vlan {n}
 remote-span

! Source device
monitor session {num} source int {} [rx|tx|both]
monitor session {num} destin remote vlan {num}

! Destination device
monitor session {num} source vlan {num} 
monitor session {num} destin int {}

ERSPAN

Encapsulated Remote SPAN allows routing the captured traffic through the use of GRE tunnels. “RSPAN on L3”

! Source device 
monitor session {n} type erspan-source
erspan-src> source {cpu | drop-cause {cause} | interface | vlan }
erspan-src> filter...
erspan-src> destination
erspan-src-dst> vrf {}
erspan-src-dst> ip address {}
erspan-src-dst> origin ip address 
erspan-src-dst> erspan-id ! Identify/Select flow on remote end
erspan-src-dst> no shut

! Destination device
monitor session {n} type erspan-destination
erspan-dst> destination interface {int}
erspan-dst> source
erspan-dst-src> vrf {}
erspan-dst-src> ip address {} ! Which address to receive on
erspan-dst-src> erspan-id {} ! Select flow
erspan-dst-src> no shut

Embedded Packet Capture

! 1. Configure options
monitor capture buffer access-list
monitor capture {} limit {}
monitor capture {} int {} {in|out|both}

! 2. Create buffer
monitor capture {name} buffer circular size {b}

! 3. Run and export
monitor capture {name} start
monitor capture {name} stop
monitor capture {name} export {location}

! 4. Verify/show
show monitor capture {name} buffer dump ! Show hex dump
show monitor capture {name} parameter   ! Show options
debug epc ...

4.7.b Cisco IOS-XE troubleshooting tools

Packet Trace

Levels of inspection:

• Accounting, simple counters. Runs until stopped
• Summary, input and output interfaces, final packet states, punted/dropped etc.
• Path data - Highly detailed, packet copy, feature invocation array(FIA).

Both summary and path data levels cannot be run indefinetely due to large resource requirements. Packet trace puts a load on the data plane, use conditions well if resources are contstrained.

!! Set up collection. 
debug platform packet-trace \
packet {num} [fia-trace|summary-only] \
[circular][data-size {}]
! circular saves most recent packets
! data-size can be set above 2048 to capture more. 

!! Copy packet contents
debug platform packet-trace copy packet {in|out|both} [L2|L3|L4] [size {bytes}]

!! limits collection of specific packet treatment
debug platform packet-trace [drop|inject|punt]

!! Configure conditions
debug platform condition ...

!! Run trace and view results
debug platform condition [start|stop]
show platform packet-trace {config|statistiscs|summary|packet}

!! Cleanup
clear platform condition all ! removes all config
clear platform packet-trace statistics 
clear platform packet-trace configuration 

Conditional debugger (debug platform condition)

debug platform condition ipv4 {subnet} {in/out}
debug platform condition ipv4 access-list {n} {ingress/egress}
debug platform condition interface {int} {ingress/egress}
debug platform condition mpls {label} {ingress/egress}
debug platform condition ingress

Copied from configuration guide

Troubleshooting

To be populated…

Study resources

The INE CCIE Enterprise infrastructure learning track

Books used, ranked by most value for time spent:

• CCIE Routing and Switching v5.0 Official Cert Guide, Volume 2, 5th Edition
• CCIE Routing and Switching v5.1 Foundations: Bridging the Gap Between CCNP and CCIE
• Cisco CCIE Routing and Switching v5.0 Configuration Practice Labs, 3rd Edition
• Cisco CCIE Routing and Switching v5.0 Troubleshooting Practice Labs, 2nd Edition
• Routing TCP/IP, Volume II: CCIE Professional Development, 2nd Edition

The CCIE Enterprise Infrastructure Foundation book by Narbik Kocharians hasn’t been released at the time of writing this, but i suspect it will also be a very good resource for the EI.

I have also used the IOS XE 16.12.x configuration guides extensively.

Various links I’ve found useful:

• https://www.valarnet.com/posts/nat-virtual-interface-and-vrfs/
• https://community.cisco.com/t5/security-knowledge-base/nat-with-vrf-ios-vs-ios-xe/ta-p/3156555

Got feedback or a question?
Feel free to contact me at hello@torbjorn.dev