Neutron 技術深入探討 /6/2 Lane

Size: px

Start display at page:

Download "Neutron 技術深入探討 /6/2 Lane"

Violet Carson
6 years ago
Views:

1 Neutron 技術深入探討 -2017/6/2 Lane

2 2

3 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent NameSpace Access control Metadata Agent NAT 3

4 4

5 Architecture 5

6 Architecture 6

7 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent NameSpace Access control Metadata Agent NAT 7

8 Provider Self-Service 8

9 9

10 # virsh list Id Name State instance running # virsh dumpxml 1 <interface type='bridge'> <mac address='fa:16:3e:49:d2:29'/> <source bridge='qbrcef0186b-d3'/> <target dev='tapcef0186b-d3'/> <model type='virtio'/> <driver name='qemu'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface>... 10

11 # brctl show bridge name bridge id STP enabled interfaces qbrcef0186b-d c26acc95f1ba no qvbcef0186b-d3 tapcef0186b-d3 11

12 # ip l 6: qbrcef0186b-d3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT qlen : qvocef0186b-d3@qvbcef0186b-d3: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP mode DEFAULT qlen : qvbcef0186b-d3@qvocef0186b-d3: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1450 qdisc noqueue master qbrcef0186b-d3 state UP mode DEFAULT qlen : tapcef0186b-d3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master qbrcef0186b-d3 state UNKNOWN mode DEFAULT qlen

13 Compute Node # ovs-vsctl show... Bridge br-int... Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal Port "qvocef0186b-d3" tag: 1 Interface "qvocef0186b-d3"... Network Node # ovs-vsctl show... Bridge br-int... Port "tapbb3b53c1-41" tag: 2 Interface "tapbb3b53c1-41" type: internal

14 # ps aux grep dnsmasq. nobody ? S 11:38 0:00 dnsmasq --no-hosts --no-resolv --strict-order --except-interface=lo --pid-file=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/host --addn-hosts=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/opts --dhcp-leasefile=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/leases --dhcp-match=set:ipxe,175 --bind-interfaces --interface=tapbb3b53c dhcp-range=set:tag0, ,static,86400s --dhcp-option-force=option:mtu, dhcp-lease-max=256 --conf-file= --domain=openstacklocal # ip netns qdhcp-fa8aaa a98-9b5c-b974ae9ebfbb qrouter e-dea f285de38fd qrouter-c6766baf-e8dc-40a3-9f29-ff95a01b5c

15 Compute Node # ovs-vsctl show... Bridge br-tun Controller "tcp: :6633" is_connected: true fail_mode: secure Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun}... Network Node # ovs-vsctl show... Bridge br-tun Controller "tcp: :6633" is_connected: true fail_mode: secure Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port br-tun Interface br-tun type: internal Port "vxlan-ac160491" Interface "vxlan-ac160491" type: vxlan options: {df_default="true", in_key=flow, local_ip=" ", out_key=flow, remote_ip=" "} 15...

16 # ovs-vsctl show Bridge br-tun Controller "tcp: :6633" is_connected: true fail_mode: secure Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port br-tun Interface br-tun type: internal Port "vxlan-ac160491" Interface "vxlan-ac160491" type: vxlan options: {df_default="true", in_key=flow, local_ip=" ", out_key=flow, remote_ip=" "}... 16

17 # ovs-vsctl show Bridge br-int... Port "tapbb3b53c1-41" tag: 2 Interface "tapbb3b53c1-41" type: internal Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "qr-5263d8eb-71" tag: 2 Interface "qr-5263d8eb-71" type: internal Port "qr-451e14ac-c6" tag: 1 Interface "qr-451e14ac-c6" type: internal... 17

18 # ip netns exec qrouter e-dea f285de38fd ip r default via dev qg-71ded6b7-d /24 dev qr-5263d8eb-71 proto kernel scope link src /28 dev qg-71ded6b7-d1 proto kernel scope link src

19 # ovs-vsctl show... Bridge br-ex Port "qg-aa24cf30-a1" Interface "qg-aa24cf30-a1" type: internal Port "qg-71ded6b7-d1" Interface "qg-71ded6b7-d1" type: internal Port br-ex Interface br-ex type: internal... 19

20 20

21 21

22 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent NameSpace Access control Metadata Agent NAT 22

23 Tunneling port br-ex br-int br-tun br-tun router 23

24 Tunneling # ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0xb829c9b8de45a59e, duration= s, table=0, n_packets=33, n_bytes=3130, idle_age=5205, priority=1,in_port=1 actions=resubmit(,2) cookie=0xb829c9b8de45a59e, duration= s, table=0, n_packets=0, n_bytes=0, idle_age=14059, priority=1,in_port=2 actions=resubmit(,4) cookie=0xb829c9b8de45a59e, duration= s, table=0, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=drop cookie=0xb829c9b8de45a59e, duration= s, table=2, n_packets=0, n_bytes=0, idle_age=14146, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20) cookie=0xb829c9b8de45a59e, duration= s, table=2, n_packets=33, n_bytes=3130, idle_age=5205, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22) cookie=0xb829c9b8de45a59e, duration= s, table=3, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=drop cookie=0xb829c9b8de45a59e, duration= s, table=4, n_packets=0, n_bytes=0, idle_age=14084, priority=1,tun_id=0x4a actions=mod_vlan_vid:1,resubmit(,10) cookie=0xb829c9b8de45a59e, duration= s, table=4, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=drop cookie=0xb829c9b8de45a59e, duration= s, table=6, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=drop cookie=0xb829c9b8de45a59e, duration= s, table=10, n_packets=0, n_bytes=0, idle_age=14146, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xb829c9b8de45a59e,nxm_of_vlan_tci[0..11],nxm_of_eth_dst[]=nxm_of_eth _SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1 cookie=0xb829c9b8de45a59e, duration= s, table=20, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=resubmit(,22) cookie=0xb829c9b8de45a59e, duration= s, table=22, n_packets=16, n_bytes=1640, idle_age=5205, hard_age=14058, dl_vlan=1 actions=strip_vlan,set_tunnel:0x4a,output:2 cookie=0xb829c9b8de45a59e, duration= s, table=22, n_packets=17, n_bytes=1490, idle_age=5232, priority=0 actions=drop 24

25 Tunneling # ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0xb829c9b8de45a59e, duration= s, table=0, n_packets=33, n_bytes=3130, idle_age=5205, priority=1,in_port=1 actions=resubmit(,2) cookie=0xb829c9b8de45a59e, duration= s, table=0, n_packets=0, n_bytes=0, idle_age=14059, priority=1,in_port=2 actions=resubmit(,4) cookie=0xb829c9b8de45a59e, duration= s, table=0, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=drop cookie=0xb829c9b8de45a59e, duration= s, table=2, n_packets=0, n_bytes=0, idle_age=14146, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20) cookie=0xb829c9b8de45a59e, duration= s, table=2, n_packets=33, n_bytes=3130, idle_age=5205, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22) cookie=0xb829c9b8de45a59e, duration= s, table=3, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=drop cookie=0xb829c9b8de45a59e, duration= s, table=4, n_packets=0, n_bytes=0, idle_age=14084, priority=1,tun_id=0x4a actions=mod_vlan_vid:1,resubmit(,10) cookie=0xb829c9b8de45a59e, duration= s, table=4, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=drop cookie=0xb829c9b8de45a59e, duration= s, table=6, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=drop cookie=0xb829c9b8de45a59e, duration= s, table=10, n_packets=0, n_bytes=0, idle_age=14146, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xb829c9b8de45a59e,nxm_of_vlan_tci[0..11],nxm_of_eth_dst[]=nxm_of_eth _SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1 cookie=0xb829c9b8de45a59e, duration= s, table=20, n_packets=0, n_bytes=0, idle_age=14146, priority=0 actions=resubmit(,22) cookie=0xb829c9b8de45a59e, duration= s, table=22, n_packets=16, n_bytes=1640, idle_age=5205, hard_age=14058, dl_vlan=1 actions=strip_vlan,set_tunnel:0x4a,output:2 cookie=0xb829c9b8de45a59e, duration= s, table=22, n_packets=17, n_bytes=1490, idle_age=5232, priority=0 actions=drop 25

26 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent NameSpace Access control Metadata Agent NAT LBaaS 26

27 DHCP agent # ps aux grep dnsmasq. nobody ? S 11:38 0:00 dnsmasq --no-hosts --no-resolv --strict-order --except-interface=lo --pid-file=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/host --addn-hosts=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/opts --dhcp-leasefile=/var/lib/neutron/dhcp/fa8aaa a98-9b5c-b974ae9ebfbb/leases --dhcp-match=set:ipxe,175 --bind-interfaces --interface=tapbb3b53c dhcp-range=set:tag0, ,static,86400s --dhcp-option-force=option:mtu, dhcp-lease-max=256 --conf-file= --domain=openstackloca 27

28 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent NameSpace Access control Metadata Agent NAT 28

29 ML2 plug-in Type drivers Flat, VLAN, VXLAN, GRE Mechanism drivers Open vswitch, Linux Bridge, L2 population 29

30 ML2 plug-in 30

31 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent Name Space Access control Metadata Agent Floating 31

32 Name Space # ip net qrouter-aa7d326c-38e6-4ad da2442a6 qdhcp-71487f16-4ea3-470c-95ec-9d5b4cebecfc 32

33 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent Name Space Access control Metadata Agent Floating 33

34 Access control # iptables -S neutron-openvswi-i2ba N neutron-openvswi-i2ba A neutron-openvswi-i2ba m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN -A neutron-openvswi-i2ba s /32 -p udp -m udp --sport 67 -m udp --dport 68 -j RETURN -A neutron-openvswi-i2ba m set --match-set NIPv415079a41-5f60-4cd0-870d- src -j RETURN -A neutron-openvswi-i2ba p tcp -m tcp --dport 22 -j RETURN -A neutron-openvswi-i2ba m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP -A neutron-openvswi-i2ba m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback 34

35 Access control INPUT neutron-openvswi-i2ba OUTPUT neutron-openvswi-o2ba Allow traffic from defined IP/MAC pairs neutron-openvswi-s2ba

36 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent Name Space Access control Metadata Agent Floating 36

37 Metadata Agent qdhcp neutron metadata agent instance nova metadata agent qrouter 37

38 Metadata Agent ssh password: cubswin:) curl 38

39 Agenda Architecture L2 Agent Tunneling DHCP agent ML2 plug-in L3 Agent Name Space Access control Metadata Agent Floating 39

40 Floating # ip net exec qrouter-aa7d326c-38e6-4ad da2442a6 iptables -t nat -S... -A neutron-l3-agent-output -d /32 -j DNAT --to-destination A neutron-l3-agent-postrouting! -i qg-cb3ca0f0-ce! -o qg-cb3ca0f0-ce -m conntrack! --ctstate DNAT -j ACCEPT -A neutron-l3-agent-prerouting -d /32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports A neutron-l3-agent-prerouting -d /32 -j DNAT --to-destination A neutron-l3-agent-float-snat -s /32 -j SNAT --to-source A neutron-l3-agent-snat -j neutron-l3-agent-float-snat -A neutron-l3-agent-snat -o qg-cb3ca0f0-ce -j SNAT --to-source A neutron-l3-agent-snat -m mark! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat 40

41 迎棧科技股份有限公司 Thank You! 41

42 課後討論分享 *OpenStack core project以外相關的project可以到stack forge上找 *OpenStack與HPC相關的技術還在討論中這裡可以看到相關討論 c 42

Deep Dive into OpenStack Networking

Deep Dive into OpenStack Networking Damrongsak Reetanon Solutions Architect Red Hat (Thailand) September 22, 2016 Deep Dive into OpenStack Networking Deep Dive into OpenStack