How many of you have never built a NetBSD kernel?

Transcription

1 A Smart Port Card Tutorial - The Exercises John DeHart Washington University jdd@arl.wustl.edu 1 Question? How many of you have never built a NetBSD kernel? 2 page 1

2 Exercises Outline Download a kernel Build a kernel for the SPC Modify a file system Build a file system from scratch Configure an SPC for IP connectivity Install the SPC hardware in a Switch Use completely separate set of switches from other exercises Do this as you have free time throughout SATURDAY 3 Open Exercises: If time permits... things we have not done yet things we can discuss and/or start working on. Building a kernel - you add the SPC mods Download program using AAL5 frames Investigate Linux kernel for SPC 4 page 2

3 Setup We need to divide into 7 Groups: Each group gets its own: login (spc1, spc2, spc3, spc7 passwords: Spc1Tut, Spc2Tut, ) switch with SPC already installed on Port 4 PC with PC APIC connected to Port 3 /usr/spc directory tree WUGS Switch Port 1 Port 0 Port 2 Port 7 APIC Fiber Pair 2 Serial Cables Port 3 SPC/ Port 4 Port 6 Port 5 5 Directory Tree for each Group: /usr/spc/netbsd NetBSD source directories /usr/spc /bin WUGS and SPC utilities binaries /usr/spc /boot_loader Source for SPC boot loader /usr/spc /configuration GBNSC config files and Jammer scripts /usr/spc /download Directory from which to download kernels /usr/spc /root_wdir Directory for portion of kernel build done as root /usr/spc /spc_utilities Source code for SPC utilities 6 page 3

4 Directory Tree for each Group: WUGS Utilities: GBNSC Jammer GBNSC config file Jammer script for configuring switch SPC Utilities: resetspc download_apic download_eni mknl: make symbol-only netbsd mkimg: make BSS image download directory pre-built bootloader boot.out bit file Pre-built kernel spckernel/netbsd.bss.orig SPC kernel source tree non-spc kernel source tree Scripts for building kernel SPC Filesystem 7 Exercise 1: Kernel Downloading page 4

5 Exercise 1 - Download and boot a Kernel Reset switch Run GBNSC and Jammer to configure switch: as root: > cd /usr/spc/configuration > sh config.apic.connections as regular user: > cd /usr/spc/configuration >../bin/gbnsc -r -a config.switch.apic & >../bin/jammer 0.1 wooster 3551 SPC_Control.js 9 Exercise 1 - continued Configure the PC (/etc/remote) so you can use tip: spc1:dv=/dev/tty01:dc:br#9600:pa=none spc0:dv=/dev/tty00:dc:br#9600:pa=none Start the tip processes in two separate windows as root: > tip spc0 > tip spc1 10 page 5

6 Exercise 1 - continued Download bootloader and kernel > cd /usr/spc/download >../bin/download_apic -a download will take 4-5 minutes make sure you see this fairly quickly: boot loader started, sleep(2) Sending entire kernel...sendfile(filename=netbsd.bss, load_addr = 0x100000, start = 0x0, sendlen = 0x ) while you re waiting look at the scripts and source code: /usr/spc/configuratoin/spc_control.js Look at the WUGS connections to get to the SPC /usr/spc/spc_utilities/download_apic/main.cc 11 Exercise 1 - continued Boot to multi-user mode When you see this: Enter pathname of shell or RETURN for sh: hit <return> Then at # prompt, type exit Log in as root (no password) on the serial console Use Terminal Type: vt100 Notice which port is the console port. tty00 or tty01? Check the size of the netbsd kernel in the file system: > ls -l /netbsd > du -s /netbsd Check the size of the file system: > df -k / While looking at your watch, try this: > sleep 20 what is the effect of these SPCs running at 25 MHz? Why? 12 page 6

7 Send a reset control cell Exercise 1 - continued > /usr/spc/bin/resetspc -a See that the serial console is now dead Why does resetspc hang? Look at code... Kill it with a <ctrl-c> Download bootloader and kernel again See boot messages in tip window 13 Exercise 2: Kernel and Filesystem Building page 7

8 Exercise 2A - Building a kernel Do these steps as regular user (i.e. not as root) Make sure the kern and pci source code is configured: > cd /usr/spc/netbsd/usr/src/sys/kern > make init_sysent.c > cd../dev/pci > make -f Makefile.pcidevs Kernel config file provided: /usr/spc/netbsd/usr/src/sys/arch/i386/conf/spc_24mb > cd /usr/spc/netbsd/usr/src/sys/arch/i386/conf > config SPC_24MB > cd../compile/spc_24mb > make depend 15 Exercise 2A - Building a kernel (con t) Three shell scripts are provided: mk.1, mk.2, mk.3 Its easier for this part if you have two windows open root window in directory /usr/spc/root_wdir regular user window in directory: /usr/spc/netbsd/usr/src/sys/arch/i386/compile/spc_24mb Stage 1: mk.1: run as regular user performs make to build kernel may take several minutes: take a look at mk.1, mk.2, mk.3 while you are waiting where does the kernel file end up when it is ready for download? > cd /usr/spc/netbsd/usr/src/sys/arch/i386/compile/spc_24mb/ >../mk.1 16 page 8

9 Exercise 2A - Building a kernel (con t) Stage 2: mk.2 run as root inserts symbol-only copy of /netbsd into filesystem > cd /usr/spc/root_wdir >./mk.2 Stage 3: mk.3 run as regular user copies filesystem into netbsd kernel file takes a couple of minutes... > cd /usr/spc/netbsd/usr/src/sys/arch/i386/compile/spc_24mb/ >../mk.3 Download bootloader and kernel again don t forget to reset SPC and if you use resetspc, don t forget to kill it with <ctrl-c> 17 Exercise 2B - Modifying a kernel Modify the kernel Change kernel so it boots to multi-user mode look in sys/arch/i386/i386/md_root.c (search for SINGLE) Change timing of clock in system FPGA look at: sys/arch/i386/isa/timerreg.h (#define TIMER_FREQ) add a kernel config file option to change TIMER_FREQ If needed, configure switch as in Exercise 1 Rebuild your kernel You will need to do a > make clean in the compile/spc_24mb directory If needed, configure switch as in Exercise 1 If needed, send reset control cell to SPC Download bootloader and kernel Log in via serial port Check the file system and verify that it is what you built 18 page 9

10 Exercise 2C - Modify a file system Modify the SPC Filesystem (as root, be careful...) > cd /usr/spc/root_wdir > vnconfig -t spc24mb -v -c /dev/vnd0d SPC24MB.fs > mount /dev/vnd0d /mnt > cd /mnt/etc Put something in it that will tell you that it is yours e.g. a file in /mnt/etc/<your_name> Change console to other TTY port modify /mnt/etc/ttys (in SPC filesystem) also change default terminal type for non-console tty to vt100 also need to change the kernel config file and re-run config > umount /mnt > vnconfig -u /dev/vnd0d 19 Exercise 2C - Modify a file system (con t) Rebuild your kernel If needed, configure switch as in Exercise 1 If needed, send reset control cell to SPC Download bootloader and kernel Log in via serial port Check the file system and verify that it is what you built 20 page 10

11 Exercise 2D - Build a file system Build your own Filesystem from scratch Edit /etc/disktab to create a new sized disk: 18MB. spc18mb spc18mbkernspc Kernel FileSystem Disk:\ :ty=simulated:se#512:nt#16:rm#300:ns#27:nc#80:\ :pa#34560:oa#0:ba#4096:fa#512:ta=4.2bsd:\ :pb#34560:ob#0:\ :pc#34560:oc#0: > cd /usr/spc/root_wdir Use dd(1) to create a file > dd if=/dev/zero of=<filename> count=<numblocks> Use vnconfig(1) to configure the file as a vnode disk > vnconfig -t <disktab entry> -v -c /dev/vnd0d <filename> Use disklabel (1) to give it a label > disklabel -rw /dev/vnd0d <disktab entry> <label> > disklabel -r /dev/vnd0d Use newfs(1) to put a filesystem on the disk > newfs -m 0 -o space -i c 54 -T <disktab entry> /dev/rvnd0a 21 Exercise 2D - Build a file system (con t) Mount it. > mount /dev/vnd0a /mnt Populate the disk: Use mknod(1) to make devices in /dev directory see next page for details! Mount the original SPC filesystem we used before on /mnt1 > vnconfig -t spc24mb -v -c /dev/vnd1d <original filesystem filename> > mount /dev/vnd1d /mnt1 if /mnt1 does not exists you may have to create it with mkdir Look through /mnt1/* and populate your filesystem from there Useful command, this does a tar(1) from directory to directory: > cd <dir1>; tar cf -. (cd <dir2>; tar xfbp -) Unmount > umount /mnt > vnconfig -u /dev/vnd0d > umount /mnt1 > vnconfig -u /dev/vnd1d 22 page 11

12 Exercise 2D - (con t) : mknod(1) Devices > cd /mnt > mkdir dev > cd dev > mknod bpf0 c 23 0 > mknod bpf1 c 23 1 > mknod bpf2 c 23 2 > mknod bpf3 c 23 3 > mknod console c 0 0 > mknod drum c 4 0 > mknod io c 2 14 > mknod klog c 7 0 > mknod kmem c 2 1 > mknod lkm c 28 0 > mknod mem c 2 0 > mknod null c 2 2 > mknod ptyp0 c 6 0 > mknod ptyp1 c 6 1 > mknod ptyp2 c 6 2 > mknod ptyp3 c 6 3 > mknod rmd0a c 24 0 > mknod rmd0b c 24 1 > mknod rmd0c c 24 2 > mknod stderr c 22 2 > mknod stdin c 22 0 > mknod stdout c 22 1 > mknod tty c 1 0 > mknod tty00 c 8 0 > mknod tty01 c 8 1 > mknod ttyp0 c 5 0 > mknod ttyp1 c 5 1 > mknod ttyp2 c 5 2 > mknod ttyp3 c 5 3 > mknod zero c 2 12 > mknod md0a b 17 0 > mknod md0b b 17 1 > chmod 600 console > chmod 640 drum > chmod 640 io > chmod 600 klog > chmod 640 kmem > chmod 640 mem > chmod 666 null > chmod 640 md0a > chmod 640 md0b > chmod 640 md0c > chmod 640 rmd0a > chmod 640 rmd0b > chmod 640 rmd0c > chmod 666 stderr > chmod 666 stdin > chmod 666 stdout > mknod md0c b 17 2 > chmod 666 tty > chmod 644 tty00 > chmod 644 tty01 > chmod 666 zero 23 Exercise 2D - (con t) : mknod(1) Devices > cd /mnt > mkdir dev > cd dev > mknod bpf0 c 23 0 > mknod bpf1 c 23 1 > mknod bpf2 c 23 2 > mknod bpf3 c 23 3 > mknod console c 0 0 > mknod drum c 4 0 > mknod io c 2 14 > mknod klog c 7 0 > mknod kmem c 2 1 > mknod lkm c 28 0 > mknod mem c 2 0 > mknod null c 2 2 > mknod ptyp0 c 6 0 > mknod ptyp1 c 6 1 > mknod ptyp2 c 6 2 > mknod ptyp3 c 6 3 > mknod rmd0a c 24 0 > mknod rmd0b c 24 1 > mknod rmd0c c 24 2 > mknod stderr c 22 2 > mknod stdin c 22 0 > mknod stdout c 22 1 > mknod tty c 1 0 > mknod tty00 c 8 0 > mknod tty01 c 8 1 > mknod ttyp0 c 5 0 > mknod ttyp1 c 5 1 > mknod ttyp2 c 5 2 > mknod ttyp3 c 5 3 > mknod zero c 2 12 > mknod md0a b 17 0 > mknod md0b b 17 1 > chmod 600 console > chmod 640 drum > chmod 640 io > chmod 600 klog > chmod 640 kmem > chmod 640 mem > chmod 666 null > chmod 640 md0a > chmod 640 md0b > chmod 640 md0c > chmod 640 rmd0a > chmod 640 rmd0b > chmod 640 rmd0c > chmod 666 stderr > chmod 666 stdin > chmod 666 stdout OK, OK. There is a script for this. /usr/spc/bin/mknods > mknod md0c b 17 2 > chmod 666 tty > chmod 644 tty00 > chmod 644 tty01 > chmod 666 zero 24 page 12

13 Exercise 2D - Build a file system (con t) Rebuild your kernel Change the kernel config file Look through the make scripts to change filesystem Re-config and re-make the kernel If needed, configure switch as in Exercise 1 If needed, send reset control cell to SPC Download bootloader and kernel Log in via serial port Check the file system and verify that it is what you built Keep it running while you go on to next exercise Exercise 3: IP Configuration page 13

14 Exercise 3A - IP Between SPC and PC Save any changes you make for the SPC in your filesystem Configure an IP address for the SPC s APIC > ifconfig Configure an IP address for the PC s APIC Configure an APIC connection on each > atm_ifconfig Add routes on PC and SPC: > route add... Configure the necessary WUGS connection(s) 27 Exercise 3A - IP Address Assignments Team PC APIC IP Address SPC APIC IP Address > cat filename telnet postscript.cs page 14

15 Exercise 3A - IP Between SPC and PC Try to ping Iterate until you can ping from PC to SPC Tools for debugging the IP connection: netstat APIC readcounts utility /usr/local/bin/readcounts on both PC and SPC WUGS cell counters (Jammer) Turn on telnetd edit /etc/inetd.conf on SPC kill -HUP <inetd pid> iterate until you can telnet from PC to SPC Be patient on some machines telnet seems to take a while to connect. Not sure why yet Exercise 3A - IP Between SPC and PC Make sure you have saved all your changes Reset switch Rebuild kernel with your newly modified filesystem and re-download. Does it still work page 15

16 Exercise 3B - IP Between Neighbor SPCs Use SPCs as routers: Domain PC PC Domain SPC SPC Exercise 3B - IP Between Neighbor SPCs Save any changes you make for the SPC in your filesystem Configure an IP address alias for the SPC s APIC to put it in the neighboring SPC s domain > ifconfig alias Remove the network route that the above caused: > route delete -net Configure an APIC connection on each SPC > atm_ifconfig Add routes on PC and SPC: > route add... Configure the necessary WUGS connection(s) 32 page 16

17 Exercise 3B - IP Between Neighbor SPCs Try to ping Iterate until you can ping from PC to neighbor PC Tools for debugging the IP connection: netstat APIC readcounts utility (/usr/local/bin/readcounts on SPC) WUGS cell counters Make sure you have saved all your changes Reset switch Rebuild kernel with your newly modified filesystem and re-download. Does it still work Exercise 4: Hardware Installation page 17

18 Exercise 4 - Installing an SPC in a Switch Power Off Open Switch Removing Glink Line Card from Port 4 or 5 Install SPC Install Serial Cables, feed in through front panel Re-Install Glink Line Card on top of SPC Close Switch Connect serial cables to PC Power On Download and Boot Kernel Tom will provide a Slide or two 35 Open Exercises page 18

19 Open Exercises: SPC Mods for NetBSD Do this first if you are interested in SPC/Linux Compare kernel config files: /usr/spc/netbsd/usr/src/sys/arch/i386/conf/spc_24mb /usr/spc/netbsd/usr/src/sys/arch/i386/conf/apic_1_4_1 Note the major differences Find the network devices in both. Anything interesting? Compare source files: /usr/spc/netbsd/usr.orig /usr/spc/netbsd/usr Directories where there are differences: sys/arch/i386/i386 sys/arch/i386/isa sys/dev/ic sys/dev/pci sys/ Open Exercises: AAL5 Download Program currently we download via APIC control cells 16 bytes of data per cell send one cell, wait for response cell before sending next one slow: 4-5 minutes for a 24MB kernel Preferred method would be AAL5 Use APIC Control cells to configure AAL5 descriptors Send Kernel in AAL5 frames to SPC Read it back to verify? 38 page 19

20 Open Exercises: Linux Kernel for the SPC ONLY if you are already familiar with NetBSD/SPC mods ONLY if you are already VERY familiar with Linux Kernel Changes: Memory Disk Serial port console APIC Configuration by APIC driver System FPGA impact: Timer configuration RTC requirements Serial port configuration Static IRQ assignment Download and Boot Boot loader Other things that Linux expects from the BIOS or bootloader? 39 page 20