Large-scale DNS. Hot Topics/An Analysis of Anomalous Queries

Transcription

1 Large-scale DNS Caching Servers Hot Topics/An Analysis of Anomalous Queries Shintaro NAKAGAMI, Tsuyoshi TOYONO Keisuke ISHIBASHI, Haruhiko NISHIDA, and Haruhiko OHSHIMA NTT Communications, OCN NTT Laboratories 1

2 Outline 1.Hot Topics about OCN DNS Caching Servers - Introduction of OCN - Query Trend on OCN DNS Caching Servers - Problems with DNS Caching Servers 2A 2.An Analysis of fanomalous Queries on Large-scale Caching Servers 2

3 Introduction of OCN OCN (AS4713) The largest ISP in JAPAN 7 million customers DNS operation 150 DNS servers -50 name servers / 100 caching servers 2 kinds of DNS application -BIND9 / CNS (CNS has 6 times performance of BIND) 6 billion queries/day (70 thousand queries/sec) 3

4 OCN Cache DNS Structure Packet Capture Point Server FW Server Router IDS L4SW Server Server Auto filtering Load Sharing Almost 100% Service Availability 4

5 Query Trend on OCN DNS Caching Servers The number of queries is increasing rapidly. The annual query increase rate is 150%. The query increase rate is much higher than the customer increase rate. OCNDNScache server query/sec 60,000 50, ery/sec qu 40,000 30,000 20,000 10,000 0 Apr-06 Jun-06 Aug-06 Oct-06 Dec-06 Feb-07 Apr-07 Jun-07 Aug-07 Oct-07 Dec-07 Feb-08 Apr-08 5

6 What types of Query? A>>AAAA>PTR>MX>TXT>>others AAAA TXT A record queries are increasing. 80.0% 70.0% The number of customers and the number of queries 60.0% per one person are 50.0% increasing. MX record queries are decreasing. Repeat MX queries by 40.0% 30.0% cache server qtype ratio A CNAME NAPTR PTR SRV AAAA MX NS SOA TXT spammer, by botnets or by 20.0% worms are decreasing. 10.0% AAAA and TXT record queries increased rapidly this year. 0.0% 2005/4/7 2006/6/ /4/ /4/28 6

7 TXT Record Queries cache server qname ratio (TXT) TXT record is used for reputation check, SPF, DNSBL 2.0% 1.8% and so on. Queries for reputation check 1.4% 1.2% are increasing. 10% 1.0% 0.8% SPF queries from mail servers 0.6% are also increasing. 0.4% 0.2% There were only a few queries 0.0% for DNSBL check until last year. 1.6% 2005/4/7 2006/6/ /4/ /4/28 7

8 Problems with DNS Caching Servers The load of caching servers is higher than that of name servers. Problem queries DDoS attack queries Bogus queries Queries for Short TTL records Birthday attack and Amp attack aren t observed so much. 8

9 DDoS Attack Queries Attacks by worms (2004/04) The number of queries at this time is 6 times more than usual. Forward operation was effective in this attack. Attacks by botnets (2007/10) The number of queries at this time is 2 times more than usual. Auto filtering by IDS worked effectively in this attack. In these case, there were a lot of SERVFAIL queries. SERVFAIL queries cause a heavy load in caching servers. 9

10 Bogus Queries Caching servers receive a lot of Bogus queries. PTR queries for RFC1918 (private IP address) -PTR *** in-addr.arpa. arpa Invalid TLD -*.localhost, *.local These queries are sent to root-servers as well as cache- servers. -> Useless traffic and processing Bogus queries Bogus queries User NXDOMAIN Cache DNS NXDOMAIN Root DNS 10

11 Short TTL Records 3 days - 1 week, 0.7% 1-3 days, 6.6% 2008 OARC DNS Ops Workshop Distribution ratio of all TTL records More than 1 week, 0.0% 6hours-1day day, 25.1% The Distribution ratio of TTL records in OCN caching servers. TTL records for less than 1 hour account for 43.5%. 1-6 hours, 24.0% TTL records for less than 10 minutes account for 14%. TTL records for less than 1 hour There are also 1 second TTL records minutes, If it isn t necessary, long TTL is 43.0% desirable. Less than 1 hour, 43.5% Less than 10 minutes, 33.0% minutes, 8.0% minutes, 16.0% 11

12 Part 2. An Analysis of Anomalous Queries on Large-scale Caching Servers Tsuyoshi TOYONO NTT Lab.

13 Focus on DNS caching servers in/out queries User -> Cache queries (recursive) Cache -> Authoritative (non-recursive) Root Servers Root Servers Root Servers User OS User (resolver) OS User (resolver) OSs (stub resolvers) Cache Servers (ISP) Authoritative Name Servers Authoritative ti Name Servers Authoritative Name Servers From user queries To authoritative server queries 13

14 What are Anomalous queries? (1/2) Invalid queries 1. Nx-Qtype (Non-existent Qtype) Invalid or broken Qtype Qyp (Ex.) Type 0, Type Nx-TLD (Non-existent it ttop Level ldomain) (Ex.).localhost.,.localdomain.,.workgroup. 3. RFC1918 PTR PTR queries for RFC1918 (Ex.) PTR in-addr.arpa

15 What are Anomalous queries? (2/2) They ignore our answers 4. Repeat queries 2008 OARC DNS Ops Workshop Repeat same Qtype, Qname queries from same IP address within very short time (1 sec) 5. Other repeat queries Ignore TTL Repeat same queries that ignored TTL 5-2. Repeat MX Repeat MX queries within very short time (0.1 sec) Characteristic behavior in some worms (Ex.) Netsky 5-3. Repeat Error Error status answers (ServFail, FormErr, Refused) are replayed, but query is repeated 15

16 User queries (to caching servers) Repeat 68.8% Legitimate 15.0% NQt NxQtype 0.1% NxTLD 1.9% RFC % ignorettl 11.7% RepeatMX 0.1% RepeatNxD 1.4% Legitimate queries: only 15% of all queries Repeat and Ignore TTL are 80% of all queries Legitimate NxQtype NxTLD RFC1918 ignorettl RepeatMX RepeatNxD Repeat 16

17 Server answers (to users) Refused NotImp NXDomain 0% 0% other 17% 0% ServFail 5% NoError FormErr FormErr 0% ServFail NXDomain NotImp Refused other NoError 78% Most answers are normal 78% of total answers are No Error 17% of total t answers are NXDomain Few error answers (Server Fail, Format Err, Refused) 17

18 First question We receive 80% anomalous queries Only 15% legitimate queries But do all users behave like that? Analysis of per user queries

19 Number of queries per user per second (CDF) 100% 90% CDF (% %) CDF(%) 80% 70% 60% 50% qps Queries per second (qps) Most users sent a few queries (1 ~ 10 qps) Only 0.07% 07% of all users sent over 100 qps at some point 19

20 Distribution chart of user query rates qps E+08 query count Number of qps count 1. Obeys Zipf s law Most users sent a few queries, a few users sent most of the queries 2. Exceptions of over qps users! 20

21 Percentage of anomalous queries by query rate type rate 100qps 200qps 300qps 400qps 500qps Legitimate 0.09% 0.01% 0% 0% 0% NxQtype 0% 0% 0% 0% 0% NTLD NxTLD 0% 0% 0% 0% 0% RFC % 0% 0% 0% 0% ignorettl 1.63% 0.05% 05% 0.01% 01% 0% 0% RepeatMX 0.01% 0% 0% 0% 0% RepeatNxD 0.64% 0% 0% 0% 0% Repeat 59.69% 59.69% 59.69% 59.69% 59.69% (Percentage of total queries) Most queries from high query rate users are repeat and ignore TTL NO legitimate queries from users sending over 300qps 21

22 Second question A few users send most repeat queries What do they want to know so much? Close analysis of details of repeat queries

23 Analysis of details of repeat queries (1/3) 2008 OARC DNS Ops Workshop We observed 4 characteristic i types in high h query rate users (Type A) NTP servers (yp ) 3.9% of high query rate users, but 70% of high query rate queries I want to know the correct time! Repeated public NTP servers over qps continuously (Ex.) time.stdtime.gov.tw.

24 Analysis of details of repeat queries (Type B) Mail servers (2/3) 76.4% of high query rate users I want to find good SPAM servers! Repeated A and MX record queries including strings such as mail, mx, smtp (Type C) Messenger servers (yp ) g 7.8% of high query rate users Repeated major messenger service servers (Ex.) AOL AIM, MSN, Windows Live, Yahoo What is their purpose? 2008 OARC DNS Ops Workshop

25 Analysis of details of repeat queries (3/3) (Type D) PTR queries 7.8% of high query rate users 2008 OARC DNS Ops Workshop Repeated PTR record for many IP addresses Perhaps due to web log analyzer or related tools Others (Unclassified) Repeated queries for SNS web site domains Repeated queries including strings pic img photo

26 Summary All queries from high query rate user are bogus or unnecessary. We can prevent these anomalous queries easily. Apply query rate limit control per user In this case, 300 qps The load on DNS servers will decrease.

27 Conclusion We should consider the way to exclude bogus queries. We hope for the development of strong BIND for caching servers. 27

28 Fin OARC DNS Ops Workshop

29 Analysis of details of repeat queries PTR, 7.80% Users ratio Unclassified, 4.10% NTP, 3.90% PTR, 2.10% Messenger, 3.80% Unclassified, 9.80% Queries ratio Messenger, 7.80% Mail, 76.40% Mail, 11.80% NTP, 72.50% NTP Mail Messenger PTR Unclassified