Troubleshooting the Wide Area Bonjour Application

Transcription

1 Troubleshooting the Wide Area Bonjour Application Troubleshooting Scenarios, page 1 Troubleshooting Using Logs, page 5 Troubleshooting Using Keywords, page 13 Troubleshooting Scenarios Troubleshooting Scenarios assists you in resolving some of the issues you might encounter while working on the Wide Area Bonjour application. If a issue in your application is similar to one of the following scenarios described in this section, use the associated workarounds to resolve the issue. Wide Area Bonjour application does not show a specific SDG Agent Step 1 Verify that the SDG agent's controller-source-interface IP Address is configured in at least one service filter as the source or query SDG agent in the Configuration tab, as shown in the below figure. To go this page, click Service Filter from the topology and click Edit option from Actions column. 1

2 SDG Agent State is Up but its announced Services are missing Troubleshooting the Wide Area Bonjour Application Step 2 Ensure that the SDG agent is properly configured in GUI. Note For more details on configuring SDG agent in service filter, see Configuring the Filer Manually section in Configuring the Wide Area Bonjour Application. SDG Agent State is Up but its announced Services are missing For this scenario, verify Wide Area Bonjour configuration in the Configuration tab, as shown in the below figure. To go to this page, click Service Filter from the topology and click Edit option from Actions column. Note For more details on configuring SDG agent in service filter, see Configuring the Filer Manually section in Configuring the Wide Area Bonjour Application. 2

3 Troubleshooting the Wide Area Bonjour Application Query SDG Agent's state is Up, but queries does not return any results from remote SDG Agents Query SDG Agent's state is Up, but queries does not return any results from remote SDG Agents For this scenario, verify Wide Area Bonjour configuration in the Configuration tab, as shown in the below figure. To go to this page, click Service Filter from the topology and click Edit option from Actions column. Note For more details on configuring SDG agent in service filter, see Configuring the Filer Manually section in Configuring the Wide Area Bonjour Application. Query SDG Agent's State is Up but query is failing For this scenario, analyze Wide Area Bonjour log and check what occurrences takes place when the service is queried. The following section displays Wide Area Bonjour logs and sample outputs. Logging into Wide Area Bonjour App Step 1 Enter the appropriate credentials to login (SSH) to APIC-EM as shown below: $ ssh grapevine@ The authenticity of host ' ( )' can't be established. ECDSA key fingerprint is 6c:63:dc:28:e8:dd:38:05:14:df:d2:88:06:19:a7:b4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ' ' (ECDSA) to the list of known hosts. grapevine@ 's password:****** Welcome to the Cisco APIC-EM Appliance - Powered by Grapevine 3

4 Query SDG Agent's State is Up but query is failing Troubleshooting the Wide Area Bonjour Application System information as of Sun Mar 12 00:16:59 UTC 2017 System load: 2.14 Users logged in: 1 Usage of /: 2.0% of 1.55TB IP address for eth0: Memory usage: 48% IP address for eth1: Swap usage: 0% IP address for grape-br0: Processes: 659 APIC-EM Version: Grapevine Version: dev1032-g32a1080 Step 2 Step 3 Last login: Sat Mar 11 21:29: from dhcp cisco.com (grapevine) [Sun Mar 12 00:17:00 UTC] grapevine@ (grapevine-root-1) ~ $ Check for the Link Local IP address of the Wide Area Bonjour App. Follow these steps, as shown in the below output. a) Enter the grape instance status command. This lists all grapevine services and their link local IP addresses. b) Check for the service 'sdg -service' that is in running state. Note If you have older versions of Wide Area Bonjour app in disabled state, they may be listed and will not be in 'running' state. c) Note down the link local IP address of the grapevine service ' sdg-service' (in the following output, it is ). $ grape instance status SERVICE VERSION STATE CLIENT IP UPTIME access-policy-programmer-service running 1ac5b5d cde-9a95-2b0af8edc days,3:12:18 apic-em-event-service running 0915c4d6-ffee-43d7-9f2c-63d c days,3:12:57 apic-em-inventory-manager-service running 9e91d331-ec41-4dc8-aef4-21f4680fcdff days,21:30: <snip> <snip> <snip> <snip> <snip> sdg-service running e919a7d2-f123-4a61-b4bd-e184a6611c6b days,21: <snip> <snip> <snip> <snip> <snip> To log into Wide Area Bonjour App via Secure Shell(SSH), enter the SSH. Note SSH is the IP address of the SDG Service and credentials are not required to enter. $ ssh Welcome to Ubuntu LTS (GNU/Linux generic x86_64) * Documentation: Last login: Sat Mar 11 21:59: from grapevine-root-1 (grapevine) [Sun Mar 12 00:48:54 UTC] grapevine@ (grapevine-client-aa6eb99b-65c f3542eb3ae) ~ Now, you are logged into Wide Area Bonjour application. 4

5 Troubleshooting the Wide Area Bonjour Application Troubleshooting Using Logs Step 4 Enter the following command for a running display of the log file of the Wide Area Bonjour App. tail -f /var/log/grapevine/services/sdg-service/<sdg Service Version>/sdg-service.log Replace <SDG-service Version> with the sdg-service version (see Step 3). You can troubleshoot Wide Area Bonjour using the following log: $ tail -f /var/log/grapevine/services/sdg-service/ /sdg-service.log :59:27,814 DEBUG nioeventloopgroup-4-24 c.c.sdg.packet.util.packetutilities Initializing PAYLOAD buffer with size :59:27,814 DEBUG nioeventloopgroup-4-24 c.c.sdg.packet.util.packetutilities Filling Payload buffer: Main Buffer offset is at <snip> <snip> <snip> To stop displaying, press CTRL+C. To capture the output in a file, enter the following command: $ tail -f /var/log/grapevine/services/sdg-service/ /sdg-service.log tee <file name> The log has wide information that can be decoded by Cisco engineers familiar with the internals. However, it may provide guidance to operational personnel on the data sent to APIC-EM by a service announcement or query, and for checking if it matches what is expected by APIC-EM (Wide Area Bonjour). The following questions are addressed: Is the announcement/query from a VLAN reaching Wide Area Bonjour? (if not, the log will not show the packet being received). Is the query received, but dropped because it does not match configured service-filter? If a query is received and matched policies, then did Wide Area Bonjour find services and responded? Troubleshooting Using Logs You can access and view logs on the controller to troubleshoot issues with the Wide Area Bonjour application. Step 1 Step 2 Step 3 Using a Secure Shell (SSH) client, log into the host (physical or virtual) with the IP address that you specified using the configuration wizard. Note The IP address to enter for the SSH client is the IP address that you configured for the network adapter. This IP address connects the host to the external network. When prompted, enter your Linux username ('grapevine') and password for SSH access. Issue the following command on the controller: grape instance status grep sdg 5

6 Analyzing Wide Area Bonjour Logs Troubleshooting the Wide Area Bonjour Application Step 4 Step 5 Using SSH again, connect to the specific container with the IP address of the service returned in above command output. Navigate and view the logs. The logs are located in "/var/log/grapevine/services/sdg-service/<version>". A maximum of 11 files should be available under this directory with name sdg-service.log*. Important If there are multiple sdg-service instances returned in the above steps, then perform the above steps again for each service instance to collect all of the logs. Analyzing Wide Area Bonjour Logs For advanced troubleshooting, it is significant to check for important output phrases in log file. Consider that a new query is received (for _universal._sub._ ipps._ tcp.local service-type). See the example below: [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _universal._sub._ipps._tcp.local.]] question:[pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _universal._sub._ipps._tcp.local.] :26:25,725 INFO pool-2-thread-1 c.c.s.p.p.queryprocessorimpl ----RECIEVED NEW QUERY. START PROCESSING ---- Wide Area Bonjour obtains the following information from the query (or announcement) for matching with its configured policy. Policy Matcher Input [ip= , network= , mask=24, servicetype=_airplay._tcp.local.] The log above displays the following information: SDG Agent's IP address Network: Subnet of the SDG Agent VLAN from which the query (or announcement) is received. Subnet mask The end-point is querying (or announcing this service) An ATV will announce airplay and roap. A printer will announce _ipp._tcp.local., and/or _ipps._tcp.local., _printer._tcp.local and so on. A new announcement is received (for._raop._tcp.local service-type, in this example below): [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _raop._tcp.local. ttl: '4500/4500' alias: '6C94F8F121B5@RTP-ATV-2._raop._tcp.local.'] additionals: [Service@ type: TYPE_SRV index 33, class: CLASS_IN index 1, name: 6C94F8F121B5@RTP-ATV-2._raop._tcp.local. ttl: '120/120' server: 'RTP-ATV-2.local.:7000'] [IPv4Address@ type: TYPE_A index 1, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: ' '] [IPv6Address@ type: TYPE_AAAA index 28, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: '2001:10:101:18:145c:d3b5:8521:7f80'] [IPv6Address@ type: TYPE_AAAA index 28, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: '2001:10:101:18:cf86:d5e3:df1d:6834'] 6

7 Troubleshooting the Wide Area Bonjour Application Analyzing Wide Area Bonjour Logs type: TYPE_TXT index 16, class: CLASS_IN index 1, name: ttl: '4500/4500' text: 'cn=0,1,2,3^gda=tr...']] answer: type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _raop._tcp.local. ttl: '4500/4500' alias: additional: type: TYPE_SRV index 33, class: CLASS_IN index 1, name: ttl: '120/120' server: 'RTP-ATV-2.local.:7000'] additional: type: TYPE_A index 1, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: ' '] additional: type: TYPE_AAAA index 28, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: '2001:10:101:18:145c:d3b5:8521:7f80'] additional: type: TYPE_AAAA index 28, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: '2001:10:101:18:cf86:d5e3:df1d:6834'] additional: type: TYPE_TXT index 16, class: CLASS_IN index 1, name: ttl: '4500/4500' text: 'cn=0,1,2,3^gda=tr...'] Result of matching the query with configured policy: Every query is matched with query sections of all service filters until a match is found. The matching result for each configured policy is logged. The below table displays the following information from the above Figure: S.No Description Query matched with service-filter (can be 'unmatched') Matching Service-filter name The rest of the output specifies the service-filter configuration 7

8 Analyzing Wide Area Bonjour Logs Troubleshooting the Wide Area Bonjour Application The following log shows the successful announcement processing output. Successful Processing of an ATV announcement (for _roap._tcp.local. Processing for _.airplay_.tcp.local not shown) :26:50,381 INFO pool-1-thread-5 c.c.s.p.p.announcementprocessorimpl GOT NEW ANNOUNCEMENT. STARTING PROCESSING :26:50,384 DEBUG nioeventloopgroup-4-16 c.c.sdg.packet.util.packetutilities Message length is from PACKET HEADER is 380 while buffer length is :26:50,384 DEBUG nioeventloopgroup-4-16 c.c.sdg.packet.util.packetutilities TLVs is :26:50,384 DEBUG nioeventloopgroup-4-16 c.c.sdg.packet.util.packetutilities TLV - Type : 5 Length : 4 Value : 0a65ff <snip> <snip> :26:50,385 DEBUG pool-3-thread-3 c.c.s.p.h.packetrecieveworkerthread -- RECIEVED PACKET :26:50,385 DEBUG pool-3-thread-3 c.c.s.p.h.packetrecieveworkerthread dns[response,:-1, length=356, id=0x0, flags=0x8000:r, answers=1, additionals=5 answers: [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _raop._tcp.local. ttl: '4500/4500' alias: '6C94F8F121B5@RTP-ATV-2._raop._tcp.local.'] additionals: [Service@ type: TYPE_SRV index 33, class: CLASS_IN index 1, name: 6C94F8F121B5@RTP-ATV-2._raop._tcp.local. ttl: '120/120' server: 'RTP-ATV-2.local.:7000'] [IPv4Address@ type: TYPE_A index 1, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: ' '] [IPv6Address@ type: TYPE_AAAA index 28, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: '2001:10:101:18:145c:d3b5:8521:7f80'] [IPv6Address@ type: TYPE_AAAA index 28, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: '2001:10:101:18:cf86:d5e3:df1d:6834'] [Text@ type: TYPE_TXT index 16, class: CLASS_IN index 1, name: 6C94F8F121B5@RTP-ATV-2._raop._tcp.local. 8

9 Troubleshooting the Wide Area Bonjour Application Analyzing Wide Area Bonjour Logs ttl: '4500/4500' text: 'cn=0,1,2,3^gda=tr...']] answer: type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _raop._tcp.local. ttl: '4500/4500' alias: additional: type: TYPE_SRV index 33, class: CLASS_IN index 1, name: ttl: '120/120' server: 'RTP-ATV-2.local.:7000'] additional: type: TYPE_A index 1, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: ' '] additional: type: TYPE_AAAA index 28, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: '2001:10:101:18:145c:d3b5:8521:7f80'] additional: type: TYPE_AAAA index 28, class: CLASS_IN index 1, name: RTP-ATV-2.local. ttl: '120/120' address: '2001:10:101:18:cf86:d5e3:df1d:6834'] additional: type: TYPE_TXT index 16, class: CLASS_IN index 1, name: ttl: '4500/4500' text: 'cn=0,1,2,3^gda=tr...'] Successful Processing of an ATV announcement : :26:50,385 DEBUG pool-1-thread-5 c.c.s.p.p.announcementprocessorimpl Anouncement Packet is from a valid SDGNode :26:50,385 INFO pool-1-thread-5 c.c.s.p.p.announcementprocessorimpl Pointer service name is RTP-ATV :26:50,386 DEBUG pool-1-thread-5 c.c.s.p.p.announcementprocessorimpl IPAddress of service RTP-ATV-2 is , 2001:10:101:18:145c:d3b5:8521:7f :26:50,386 DEBUG pool-1-thread-5 c.c.s.p.p.announcementprocessorimpl MACAddress of service RTP-ATV-2 is 6c:94:f8:f1:21:b :26:50,386 INFO pool-1-thread-5 c.c.s.p.p.announcementprocessorimpl Total services in this announcement 1 Note ATV announces _airplay._tcp.local similar to the above processing. Successful Query Processing Log output: Successful Processing of an query for airplay: :59:25,745 DEBUG pool-3-thread-5 c.c.s.p.h.packetrecieveworkerthread --- RECIEVED PACKET :59:25,746 DEBUG pool-3-thread-5 c.c.s.p.h.packetrecieveworkerthread dns[query,:-1, length=37, id=0xae53, flags=0x100, questions=1 questions: [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _airplay._tcp.local.]] question: [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _airplay._tcp.local.] :59:25,746 INFO pool-2-thread-1 c.c.s.p.p.queryprocessorimpl --RECIEVED NEW QUERY. START PROCESSING :59:25,749 DEBUG pool-2-thread-1 c.c.s.r.engine.impl.policymatcher PolicyMatcher: Starting to get ALL matching rules :59:25,749 DEBUG pool-2-thread-1 c.c.s.r.engine.impl.policymatcher Policy Matcher Input [ip= , network= , mask=24, servicetype=_airplay._tcp.local.] Mar 12, :59:25 AM ch.maxant.rules.engine getmatchingrules INFO: matched: com.cisco.sdg.rule.engine.result.pod-5-atv-{ ( ( input.ip == ' ' input.network == ' ' input.mask == '24' ) ( input.ip == ' ' input.network == '2001:10:106:221:0:0:0:0' input.mask == '64' ) ) ( input.servicetype == '_raop._tcp.local.' input.servicetype == '_airplay._tcp.local.' ) } Mar 12, :59:25 AM ch.maxant.rules.engine getmatchingrules 9

10 Analyzing Wide Area Bonjour Logs Troubleshooting the Wide Area Bonjour Application INFO: unmatched: com.cisco.sdg.rule.engine.result.pod-5-printer-{ ( ( input.ip == ' ' input.network == ' ' input.mask == '24' ) ( input.ip == ' ' input.network == '2001:10:106:221:0:0:0:0' input.mask == '64' ) ) ( input.servicetype == '_printer._tcp.local.' input.servicetype == '_ipps._tcp.local.' input.servicetype == '_ipp._tcp.local.' ) } :59:25,750 INFO pool-2-thread-1 c.c.s.r.engine.impl.policymatcher PolicyMatcher: ALL MATCHED RULES [7f1c7bb1-a b4d9-50cc94f6e530] :59:25,750 DEBUG pool-2-thread-1 c.c.s.p.c.manager.sdgcachemanager Getting services for policyids [7f1c7bb1-a b4d9-50cc94f6e530] :59:25,750 DEBUG pool-2-thread-1 c.c.s.p.c.manager.sdgcachemanager Obtained 2 services for policyids [7f1c7bb1-a b4d9-50cc94f6e530] :59:25,750 DEBUG pool-2-thread-1 c.c.s.p.p.queryprocessorimpl Matching for service type _airplay._tcp.local. as the type is _airplay._tcp.local. and subtype is in the question :59:25,750 DEBUG pool-2-thread-1 c.c.s.p.p.queryprocessorimpl Matching for service type _airplay._tcp.local. as the type is _airplay._tcp.local. and subtype is in the question :59:25,750 INFO pool-2-thread-1 c.c.s.p.p.queryprocessorimpl Found 1 services to respond. Sending :59:25,750 INFO EventLogProcessor com.cisco.sdg.packet Sun Mar 12 04:59:25 UTC 2017 TX QUERY RESPONDED Processed the query packet for service name _airplay._tcp.local. from SDG Node Responded with 1 service instances :59:25,753 INFO pool-2-thread-1 c.c.s.p.p.queryprocessorimpl -- COMPLETED QUERY PROCESSING :59:25,754 DEBUG pool-5-thread-5 c.c.s.p.m.queryresponsesender Sending the response to IP port: 9991 data length: 1 Unsuccessful Query Processing Log output: Unsuccessful Processing of an query for airplay: The query matched configured policy, but there are no ATVs available :14:40,754 DEBUG pool-3-thread-6 c.c.s.p.h.packetrecieveworkerthread ---RECIEVED PACKET :14:40,754 DEBUG pool-3-thread-6 c.c.s.p.h.packetrecieveworkerthread dns[query,:-1, length=34, id=0xa7a2, flags=0x100, questions=1 questions: [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _raop._tcp.local.]] question: [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _raop._tcp.local.] :14:40,755 DEBUG pool-2-thread-5 c.c.s.r.engine.impl.policymatcher PolicyMatcher: Starting to get ALL matching rules :14:40,755 DEBUG pool-2-thread-5 c.c.s.r.engine.impl.policymatcher Policy Matcher Input [ip= , network=2001:10:106:221:0:0:0:0, mask=64, servicetype=_airplay._tcp.local.] Mar 12, :14:40 AM ch.maxant.rules.engine getmatchingrules INFO: matched: com.cisco.sdg.rule.engine.result.pod-5-atv-{ ( ( input.ip == ' ' input.network == ' ' input.mask == '24' ) ( input.ip == ' ' input.network == '2001:10:106:221:0:0:0:0' input.mask == '64' )) ( input.servicetype == '_raop._tcp.local.' input.servicetype == '_airplay._tcp.local.' ) } :14:40,755 INFO pool-2-thread-3 c.c.s.p.p.queryprocessorimpl -- RECIEVED NEW QUERY. START PROCESSING Mar 12, :14:40 AM ch.maxant.rules.engine getmatchingrules INFO: unmatched: com.cisco.sdg.rule.engine.result.pod-5-printer-{ ( ( input.ip == ' ' input.network == ' ' input.mask == '24' ) ( input.ip == ' ' input.network == '2001:10:106:221:0:0:0:0' input.mask == '64' ) ) ( input.servicetype == '_printer._tcp.local.' input.servicetype == '_ipps._tcp.local.' input.servicetype == '_ipp._tcp.local.' ) } :14:40,755 INFO pool-2-thread-5 c.c.s.r.engine.impl.policymatcher 10

11 Troubleshooting the Wide Area Bonjour Application Analyzing Wide Area Bonjour Logs PolicyMatcher: ALL MATCHED RULES [7f1c7bb1-a b4d9-50cc94f6e530] :14:40,755 DEBUG pool-2-thread-5 c.c.s.p.c.manager.sdgcachemanager Getting services for policyids [7f1c7bb1-a b4d9-50cc94f6e530] :14:40,756 DEBUG pool-2-thread-5 c.c.s.p.c.manager.sdgcachemanager Obtained 0 services for policyids [7f1c7bb1-a b4d9-50cc94f6e530] :14:40,756 INFO pool-2-thread-5 c.c.s.p.p.queryprocessorimpl Found 0 services to respond. Sending :14:40,756 INFO EventLogProcessor com.cisco.sdg.packet Sun Mar 12 05:14:40 UTC 2017 RX QUERY DROPPED Dropped the query for service _airplay._tcp.local. as no policies matched for source IP , subnet 2001:10:106:221:0:0:0:0 and mask :14:40,756 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities Message length is from PACKET HEADER is 7 while buffer length is :14:40,756 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities TLVs is :14:40,757 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities TLV - Type : 22 Length : 4 Value : 0a64ff :14:40,757 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities Initializing PAYLOAD buffer with size :14:40,757 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities Filling Payload buffer: Main Buffer offset is at :14:40,757 DEBUG nioeventloopgroup-4-29 c.c.s.p.h.tcppacketeventqueuer Done processing the packet in 3ms :14:40,757 DEBUG ThreadManager-Default-45 c.c.s.p.h.newdatapacketnotificationhandler About to handle packet com.cisco.sdg.packet.model.datapacket@5f :14:40,757 INFO ThreadManager-Default-45 c.c.s.p.h.packetrecievehandler PacketRecieveHandler: SDG Packet Recieved from DPS com.cisco.sdg.packet.model.datapacket@5f :14:40,758 DEBUG pool-2-thread-3 c.c.s.r.engine.impl.policymatcher PolicyMatcher: Starting to get ALL matching rules :14:40,758 INFO pool-2-thread-2 c.c.s.p.p.queryprocessorimpl OOPS! NO RESPONSES FOUND TO BE REPLIED :14:40,758 DEBUG pool-2-thread-3 c.c.s.r.engine.impl.policymatcher Policy Matcher Input [ip= , network=2001:10:106:221:0:0:0:0, mask=64, servicetype=_raop._tcp.local.] :14:40,758 INFO pool-2-thread-2 c.c.s.p.p.queryprocessorimpl - COMPLETED QUERY PROCESSING.-- Unsuccessful Processing of an query for airplay: The query did not match configured policy :33:40,784 DEBUG pool-3-thread-3 c.c.s.p.h.packetrecieveworkerthread --- RECIEVED PACKET :33:40,784 DEBUG pool-3-thread-3 c.c.s.p.h.packetrecieveworkerthread dns[query,:-1, length=37, id=0x508f, flags=0x100, questions=1 questions: [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _airplay._tcp.local.]] question: [Pointer@ type: TYPE_PTR index 12, class: CLASS_IN index 1, name: _airplay._tcp.local.] :33:40,785 INFO pool-2-thread-5 c.c.s.p.p.queryprocessorimpl RECIEVED NEW QUERY. START PROCESSING :33:40,786 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities Message length is from PACKET HEADER is 7 while buffer length is :33:40,786 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities TLVs is :33:40,786 DEBUG nioeventloopgroup

12 Analyzing Wide Area Bonjour Logs Troubleshooting the Wide Area Bonjour Application c.c.sdg.packet.util.packetutilities TLV - Type : 22 Length : 4 Value : 0a64ff :33:40,786 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities Initializing PAYLOAD buffer with size :33:40,786 DEBUG nioeventloopgroup-4-29 c.c.sdg.packet.util.packetutilities Filling Payload buffer: Main Buffer offset is at :33:40,786 DEBUG nioeventloopgroup-4-29 c.c.s.p.h.tcppacketeventqueuer Done processing the packet in 2ms :33:40,786 DEBUG ThreadManager-Default-20 c.c.s.p.h.newdatapacketnotificationhandler About to handle packet com.cisco.sdg.packet.model.datapacket@7d97c9b :33:40,786 INFO ThreadManager-Default-20 c.c.s.p.h.packetrecievehandler PacketRecieveHandler: SDG Packet Recieved from DPS com.cisco.sdg.packet.model.datapacket@7d97c9b :33:40,788 DEBUG pool-2-thread-5 c.c.s.r.engine.impl.policymatcher PolicyMatcher: Starting to get ALL matching rules :33:40,788 DEBUG pool-2-thread-5 c.c.s.r.engine.impl.policymatcher Policy Matcher Input [ip= , network=2001:10:106:221:0:0:0:0, mask=64, servicetype=_airplay._tcp.local.] Mar 12, :33:40 AM ch.maxant.rules.engine getmatchingrules INFO: unmatched: com.cisco.sdg.rule.engine.result.pod-5-atv-{ ( ( input.ip == ' ' input.network == ' ' input.mask == '24' ) ( input.ip == ' ' input.network == '2001:10:106:222:0:0:0:0' input.mask == '64' ) ) ( input.servicetype == '_raop._tcp.local.' input.servicetype == '_airplay._tcp.local.' ) } Mar 12, :33:40 AM ch.maxant.rules.engine getmatchingrules INFO: unmatched: com.cisco.sdg.rule.engine.result.pod-5-printer-{ ( ( input.ip == ' ' input.network == ' ' input.mask == '24' ) ( input.ip == ' ' input.network == '2001:10:106:221:0:0:0:0' input.mask == '64' ) ) ( input.servicetype == '_printer._tcp.local.' input.servicetype == '_ipps._tcp.local.' input.servicetype == '_ipp._tcp.local.' ) } :33:40,789 INFO pool-2-thread-5 c.c.s.r.engine.impl.policymatcher PolicyMatcher: ALL MATCHED RULES [] :33:40,789 DEBUG pool-2-thread-5 c.c.s.p.c.manager.sdgcachemanager Getting services for policyids [] :33:40,789 DEBUG pool-2-thread-5 c.c.s.p.c.manager.sdgcachemanager Obtained 0 services for policyids [] :33:40,789 INFO pool-2-thread-5 c.c.s.p.p.queryprocessorimpl Found 0 services to respond. Sending :33:40,789 INFO EventLogProcessor com.cisco.sdg.packet Sun Mar 12 05:33:40 UTC 2017 RX HELLO PROCESSED Recieved keep alive packet from SDG Node IP :33:40,789 INFO EventLogProcessor com.cisco.sdg.packet Sun Mar 12 05:33:40 UTC 2017 RX QUERY DROPPED Dropped the query for service _airplay._tcp.local. as no policies matched for source IP , subnet 2001:10:106:221:0:0:0:0 and mask :33:40,790 DEBUG ThreadManager-Default-4 c.c.s.p.h.newdatapacketnotificationhandler About to handle packet com.cisco.sdg.packet.model.datapacket@7d97c9b :33:40,790 INFO EventLogProcessor com.cisco.sdg.packet Sun Mar 12 05:33:40 UTC 2017 TX HELLO ACKED Sent keep alive packet acknowledgement to SDG Node IP :33:40,790 DEBUG ThreadManager-Default-4 c.c.s.p.h.udptcppacketsendhandler Done Sending packet on / and port 9991 and type TCP :33:40,790 DEBUG nioeventloopgroup-4-29 c.c.s.p.h.udptcppacketsendhandler Future DefaultChannelPromise@6dce43ad(success) completed with status true 12

13 Troubleshooting the Wide Area Bonjour Application Troubleshooting Using Keywords :33:40,792 INFO pool-2-thread-5 c.c.s.p.p.queryprocessorimpl OOPS! NO RESPONSES FOUND TO BE REPLIED :33:40,792 INFO pool-2-thread-5 c.c.s.p.p.queryprocessorimpl -- COMPLETED QUERY PROCESSING.- Unsuccessful Processing airplay announcement: The announcements did not match the source service-filter :43:50,384 INFO pool-1-thread-4 c.c.s.p.p.announcementprocessorimpl Matching policy for recieved announcement: servicetype _airplay._tcp.local. name RTP-ATV-2 mac address 6c:94:f8:f1:21:b :43:50,384 DEBUG pool-1-thread-4 c.c.s.r.engine.impl.policymatcher PolicyMatcher: Starting to get ALL matching rules :43:50,384 DEBUG pool-1-thread-4 c.c.s.r.engine.impl.policymatcher Policy Matcher Input [ip= , network= , mask=28, servicetype=_airplay._tcp.local.] :43:50,384 INFO pool-1-thread-5 c.c.s.p.p.announcementprocessorimpl --- GOT NEW ANNOUNCEMENT. STARTING PROCESSING Mar 12, :43:50 AM ch.maxant.rules.engine getmatchingrules INFO: unmatched: com.cisco.sdg.rule.engine.result.pod-5-atv-{ ( ( input.ip == ' ' input.network == ' ' input.mask == '28' ) ( input.ip == ' ' input.network == '2001:10:101:19:0:0:0:0' input.mask == '64' ) ) ( input.servicetype == '_raop._tcp.local.' input.servicetype == '_airplay._tcp.local.' ) } Mar 12, :43:50 AM ch.maxant.rules.engine getmatchingrules INFO: unmatched: com.cisco.sdg.rule.engine.result.pod-5-printer-{ ( ( input.ip == ' ' input.network == ' ' input.mask == '28' ) ( input.ip == ' ' input.network == '2001:10:101:77:0:0:0:0' input.mask == '64' ) ) ( input.servicetype == '_printer._tcp.local.' input.servicetype == '_ipps._tcp.local.' input.servicetype == '_ipp._tcp.local.' ) } :43:50,384 INFO pool-1-thread-4 c.c.s.r.engine.impl.policymatcher PolicyMatcher: ALL MATCHED RULES [] :43:50,384 INFO pool-1-thread-4 c.c.s.p.p.announcementprocessorimpl No matching policies for valid announcements found. DROPPING THE ANNOUNCEMENT :43:50,385 INFO pool-1-thread-4 c.c.s.p.p.announcementprocessorimpl COMPLETED ANNOUNCEMENT PROCESSING :43:50,385 INFO EventLogProcessor com.cisco.sdg.packet Sun Mar 12 05:43:50 UTC 2017 RX ANNOUNCEMENT DROPPED Dropped the announcement of service RTP-ATV-2 of type _airplay._tcp.local. as no policies matched for source IP , subnet and mask 28 From the above log, No service filters matches the announcement (the subnets did not match). Troubleshooting Using Keywords You can troubleshoot the Wide Area Bonjour application using the Linux grep command and specific keywords. To troubleshoot with these specific keywords, first open a Secure Shell (SSH) client and log into the host (physical or virtual) running the Cisco APIC-EM controller. When prompted, enter your Linux username ('grapevine') and password for SSH access. Once SSH access is established use the Linux grep command with the keywords described in the following table. 13

14 Troubleshooting Using Keywords Troubleshooting the Wide Area Bonjour Application Table 1: Troubleshooting Keywords Keyword QUERY ANNOUNCEMENT Command tail f {path to sdg-service.log} grep QUERY tail f {path to sdg-service.log} grep ANNOUNCEMENT Description This command and keyword can be used to troubleshoot what is happening with the queries being received by the Wide Area Bonjour application. You can identify which queries are being responded to, which are being dropped, and for what reason. This command and keyword can be used to troubleshoot what is happening with the announcements being received by the Wide Area Bonjour application. You can identify which announcements are being processed and cached, as well as which are being dropped and for what reason. 14