NetBSD Kernel Topics:

Transcription

1 NetBSD Kernel Topics: IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous January 9, 2001 Router Plugins (Crossbow) Washington 1

2 IP Goals Understand the structure of IP processing in NetBSD Understand the IP packet format Become familiar with IP packet field access mbufs» accessing src and dst addresses, etc Understand the mbuf structure Become familiar with mbuf access routines» packet size» data location Stevens TCP/IP books impress upon you the importance of these ref. books January 9, 2001 Router Plugins (Crossbow) Washington 2

3 IPv6 What we are NOT going to talk about Everything we talk about will be in terms of IPv4 Router Plugins also supports IPv6 The code can be confusing IP Options» use IPv6 structures since they are superset of IPv4 Fast Forward Path Details of interaction between device drivers and IP actually, we ll talk about some of this Packet Scheduler Routing protocols daemons: routed, rsvpd, January 9, 2001 Router Plugins (Crossbow) Washington 3

4 NetBSD Kernel Topics: IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous January 9, 2001 Router Plugins (Crossbow) Washington 4

5 Vocabulary: Terms Used in Stevens message: Transport protocol data given to IP datagram: message + IP header fragment: if datagram is too large for network: IP splits it. Each fragment contains its own IP header packet: fragment or datagram small enough for network frame: packet + data-link layer header January 9, 2001 Router Plugins (Crossbow) Washington 5

6 Protocol messages, IP Packets, Fragments, message IP UDP Application data fragment 1 datagram IP UDP application data packet IP Application data IP Application data fragment 2 fragment 3 link IP UDP application data link IP Application data link IP Application data frame Derived from Figure 8.7 from Wright/Stevens, Vol. 2 Washington January 9, 2001 Router Plugins (Crossbow) 6

7 IP Packet Format Vers HLen TOS Total length ID Flags Fragment Offset TTL Protocol Header Checksum Source Address Destination Address Source Port Destination Port... IP Header (with no Options fields) Transport Protocol (e.g. TCP, UDP) Header Packet Payload January 9, 2001 Router Plugins (Crossbow) Washington 7

8 IPv4 Forwarding in the NetBSD Kernel (APIC) T R A N S P O R T tcp/udp_input socket layer tcp/udp_output N E T W O R K ip6_input ipintr ip6_forward ip_forward ip6_output ip_output ipintrq: D A T A atmc_input atmc_output PS_Q packet_scheduler U P P E R L I N K apic_isr apic_intr apic_devoutput apic_start L O W E R input packets interrupts output packets January 9, 2001 Router Plugins (Crossbow) Washington 8

9 User IP in the NetBSD Kernel User Program T R A N S P O R T tcp/udp_input socket layer Waits in Socket Buffer for User Program to pick up tcp/udp_output N E T W O R K ipintr ipintrq: ip_forward Software Interrupt ip_output D A T A atmc_input atmc_output PS_Q packet_scheduler U P P E R L I N K apic_isr apic_intr apic_devoutput apic_start L O W E R input packets interrupts output packets Washington January 9, 2001 Router Plugins (Crossbow) 9

10 IP Packet Handling in the CB Kernel User Space Kernel T R socket layer A This Where NHere is Where are is the the gate are route rest that the of lookup gates? you the will gates? done? use. S P O R T tcp/udp_input tcp/udp_output N E T W O R K ip6_input ipintr ip6_forward ip_forward ip6_output ip_output sgate dgate SW Interrupt ipintrq: HW Interrupt D A T A atmc_input atmc_output PS_Q packet_scheduler U P P E R L I N K apic_isr apic_intr apic_devoutput apic_start L O W E R input packets interrupts output packets January 9, 2001 Router Plugins (Crossbow) Washington 10

11 Important Source Files (usr/src/sys/netinet/ip*) ip.h struct ip { } #define s ip_input.c ipintr() (This is ip_input routine) ip_forward() ip_output.c ip_output() January 9, 2001 Router Plugins (Crossbow) Washington 11

12 IP Packet Handling in CB Kernel with APIC Device Driver -- Device Dependent apic_intr()» read INTR_ACK: 0 => not for us, return» apic_isr() apic_isr()» read NOTIFY_LIST, next rcv channel needing attention Process descriptors for RCV channel swap words in each received mbuf (APIC BUG) link mbufs until we find end of frame verify CRC (just check flag set by APIC) atmc_input(packet) repeat repeat ipintr ip_forward ip_output ipintrq: atmc_input atmc_output apic_isr apic_intr apic_devoutput January 9, 2001 Router Plugins (Crossbow) Washington 12

13 IP Packet Handling in CB Kernel (continued) Device Driver - Dev. Independent atmc_input(): LLC/SNAP processing (if needed)» extract type (IP/RATM/ ) AAL5 processing» extract length» strip trailer aiu_getafix(packet) (flow table will be discussed later)» aiu_getafix stores a ptr to FTE in packet Enqueue in IP Queue ipintr ipintrq: ip_forward ip_output atmc_input atmc_output apic_isr apic_intr apic_devoutput January 9, 2001 Router Plugins (Crossbow) Washington 13

14 IP Packet Handling in CB Kernel (continued) IP Input -- ipintr(): Get next packet from IP Queue Do some basic checks, header, length, checksum process IP options aiu_dgate(&m,2) Router Plugins Dynamic Gate #2 if packet is not for us and we can forward it:» forward packet ip_forward() -- upcoming slide...» return aiu_dgate(&m,4) Router Plugins Dynamic Gate #4 protocol specific input routine e.g. tcp_input() ipintr ipintrq: atmc_input ip_forward ip_output atmc_output apic_isr apic_intrapic_devoutput January 9, 2001 Router Plugins (Crossbow) Washington 14

15 IP Packet Handling in CB Kernel (continued) IP Forwarding -- ip_forward(): decide if we need to send any redirects to sender aiu_dgate(&m,5) Router Plugins Dynamic Gate #5 route lookup ip_output() Update for Fred s new Gate also check details of if/when route lookup is done ipintr ipintrq: ip_forward ip_output atmc_input atmc_output apic_isr apic_intr apic_devoutput January 9, 2001 Router Plugins (Crossbow) Washington 15

16 IP Packet Handling in CB Kernel (continued) IP Output -- ip_output() aiu_dgate(&m,3) Router Plugins Dynamic Gate #3 if (no route yet)» get route check for special processing» ANEP Options anep_output() send on interface ipintr ip_forward ip_output ipintrq: atmc_input apic_isr apic_intr atmc_output apic_devoutput if DAN then afd_handle_dan_packet() :ACTIVE PROCESSING January 9, 2001 Router Plugins (Crossbow) Washington 16

17 IP Packet Handling in CB Kernel (continued) Device Driver Device Indep.-- atmc_output(): LLC/SNAP processing AAL5 processing (if needed) if packet_scheduling: enqueue for PS if!packet_scheduling: send to devoutput() Device Driver Device Dep. -- apic_devoutput() configure an APIC descriptor for each mbuf in packet Resume APIC TX channel ipintr ipintrq: ip_forward ip_output atmc_input atmc_output apic_isr apic_intr apic_devoutput January 9, 2001 Router Plugins (Crossbow) Washington 17

18 Note about Addresses The address structures in the Crossbow Kernel are IPv6 IPv6 address structures are a superset of IPv4 IPv6 address are 128 bits We will be using IPv4 addresses e.g IPv4 addresses are 32 bits Notation for using IPv4 address in IPv6: Use double colon before address: :: Double colon tells the utilities to set everything to the left to 0 s January 9, 2001 Router Plugins (Crossbow) Washington 18

19 NetBSD Kernel Topics IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous January 9, 2001 Router Plugins (Crossbow) Washington 19

20 The mbuf Data Structure mbuf{} mbuf{} mbuf{} mbuf{} m_next m_nextpkt m_len m_data m_type m_flags MT_xxx 0 MT_xxx M_PKTHDR MT_xxx M_EXT MT_xxx M_PKTHDR M_EXT m_pkthdr.len m_pkthdr.rcvif m_ext.ext_buf m_ext.ext_free m_ext.ext_size Normal mbuf Normal mbuf with Packet Hdr 2048-byte cluster Cluster mbuf 2048-byte cluster Cluster mbuf with Packet Hdr January 9, 2001 Router Plugins (Crossbow) Washington 20

21 Highlight: (add Stevens like figures ) m_next vs m_nextpkt (figure 2.2 page 35)» m_next: points to next mbuf in THIS packet» m_nextpkt: points to the first mbuf of the NEXT packet m_len vs. m_pkthdr.len m_ext.ext_buf vs. m_data Which mbuf structure we use with APIC Example: Sorry: This was just one do m_pullup of my notes slides that get data pointer should have been removed print out src addr, dst addr before and protocol printing fields get Transport header pointer print out dst port January 9, 2001 Router Plugins (Crossbow) Washington 21

22 The mbuf Data Structure m_next m_nextpkt m_len m_data m_type m_flags mbuf{} 0 mbuf{} MT_DATAMT_xxx M_PKTHDR mbuf{} MT_xxx M_EXT mbuf{} MT_xxx M_PKTHDR M_EXT m_pkthdr.len Sorry: This was just one of my notes slides that should have been removed before printing m_pkthdr.rcvif m_ext.ext_buf m_ext.ext_free m_ext.ext_size 2048-byte cluster 2048-byte cluster January 9, 2001 Router Plugins (Crossbow) Washington 22

23 The mbuf Data Structure (example) head of queue tail of queue mbuf{} m_next m_nextpkt m_len m_data m_type next mbuf in chain 42 MT_DATA m_flags M_PKTHDR m_pkthdr.len 192 mbuf m_pkthdr.rcvif NULL packet header mbuf{} next mbuf in chain m_next m_nextpkt NULL m_len 100 m_data m_type MT_DATA m_flags 0 m_pkthdr.len m_pkthdr.rcvif mbuf{} m_next m_nextpkt m_len m_data m_type m_flags 50 bytes of data NULL NULL 50 MT_DATA 0 Ethernet header, IP header, UDP header 100 bytes of data next packet in queue January 9, 2001 Router Plugins (Crossbow) Washington 23

24 The mbuf Data Structure (example, continued) tail of queue mbuf{} m_next m_nextpkt m_len m_data m_type NULL 54 MT_HEADER m_flags M_PKTHDR m_pkthdr.len 1514 mbuf m_pkthdr.rcvif NULL packet header Ethernet header, IP header, TCP header next mbuf in chain mbuf{} m_next NULL m_nextpkt NULL m_len 1460 m_data m_type MT_DATA m_flags M_EXT m_pkthdr.len m_pkthdr.rcvif m_ext.ext_buf m_ext.ext_free m_ext.ext_size byte cluster 1460 bytes of data January 9, 2001 Router Plugins (Crossbow) Washington 24

25 NetBSD Kernel Topics IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous January 9, 2001 Router Plugins (Crossbow) Washington 25

26 Loadable Kernel Modules Mechanism in NetBSD to dynamically load code into running kernel NetBSD System Utilities: modload(8) modunload(8) modstat(8) Module Types supported: System Call modules» When unloaded, returns to original system call» Any system call can be replace» Take care when replacing ioctl(2) since LKM uses it to load/unload modules!! Virtual File System modules Device Driver modules: Block and character device drivers Execution Interpreters: For binaries not normally usable by OS Miscellaneous modules» No defined interfaces» Up to user/developer to provide hooks to get to the code» This is what Router Plugins uses January 9, 2001 Router Plugins (Crossbow) Washington 26

27 Loadable Kernel Modules: modload Open /dev/lkm ioctl s will be performed on the open file descriptor Prelink module, open it and calculate size info system( ld -A /netbsd -e _<entry> -o <outfile> -T <addr=0> <module> ); Reserve Kernel memory: ioctl(lkm_fd, LMRESERV, size_info) returns kernel load address Relink at kernel load address Open relinked module and load it into kernel ioctl(lkm_fd, LMLOADBUF,...) Adjusting symbol table entry pointers and load it. sync(); Call the module s entry function to test it. Post-install if called for... ioctl(lkm_fd, LMSTAT,...) January 9, 2001 Router Plugins (Crossbow) Washington 27

28 NetBSD Kernel Topics IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous January 9, 2001 Router Plugins (Crossbow) Washington 28

29 8 levels spl0 normal Interrupts splsoftclock low-priority clock processing splnet network protocol processing spltty terminal I/O splbio disk and tape I/O splimp network device I/O splclock high-priority clock processing splhigh all interrupts blocked Typical usage: int i; i = splimp(); [do something at device I/O level] splx(i); January 9, 2001 Router Plugins (Crossbow) Washington 29

30 NetBSD Kernel Topics IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous January 9, 2001 Router Plugins (Crossbow) Washington 30

31 Other kernel details you must know No floating point No #include in header files More caveats and info from Fred later January 9, 2001 Router Plugins (Crossbow) Washington 31