MALICIOUS URL DETECTION AND PREVENTION AT BROWSER LEVEL FRAMEWORK

Transcription

1 International Journal of Mechanical Engineering and Technology (IJMET) Volume 8, Issue 12, December 2017, pp , Article ID: IJMET_08_12_054 Available online at ISSN Print: and ISSN Online: IAEME Publication Scopus Indexed MALICIOUS URL DETECTION AND PREVENTION AT BROWSER LEVEL FRAMEWORK M. Sridevi Research Scholar, JNTUH, Hyderabad, T.S, India Dr K.V.N. Sunitha Professor & Principal, BVRIT Engineering for Women, Hyderabad, T.S, India ABSTRACT Nowadays, the web becomes very important in daily life. Along with that are a massive number of threats related to the internet. Google reported that over 50 million website users were warned that the websites they visited trying to steal information or install malicious software. Google s current blacklist has almost 70,000 sites per week. Therefore, web security has become a hot topic both in research and industry; and the most popular and serious threat to web security is a drive-by-download attack. This attack happens when a victim just merely visits a seem-to-be-legitimate webpage (URL) that includes a malicious JavaScript code and automatically redirects the victim to an exploit webpage that installs malware into the victim s computer. According to a study by Google [2], approximately 1.3% of the incoming search queries to Google s search engine returns at least one malicious URL. This indicates that a significant portion of web clients can be the victims of the drive-by download attack. In this paper investigating how to detect and prevent malicious URL at browser level with help of one of the machine learning algorithm online learning. Key words: Heuristic approach, URL, online learning algorithm, supervised learning. Cite this Article: M. Sridevi and Dr K.V.N. Sunitha, Malicious URL Detection and Prevention at Browser Level Framework, International Journal of Mechanical Engineering and Technology 8(12), 2017, pp INTRODUCTION URL is the abbreviation of Uniform Resource Locator, which is the universal address of documents and other resources on the World Wide Web. A URL has two main components: (1) Protocol identifier, it shows what protocol to use, (2)Resource name; it projects the IP address or the domain name where the resource is identified editor@iaeme.com

2 Malicious URL Detection and Prevention at Browser Level Framework The protocol identifier and there source name differed by a colon and two forward slashes. An example is shown in Figure 1.Compromised URLs that are used for cyber-attacks are known as malicious URLs. In fact, it was observed that close to one-third of all websites are potentially malicious [2], demonstrating rampant use of malicious Figure 1 Illustration of a URL The most common method to detect malicious URLs set up by certain antivirus groups is the black-list method. Black-lists are mainly a database of URLs that have been confirmed to be malicious in the earlier. This database is compiled over time as and when it becomes known that a URL is malicious. Such a technique is high-speed due to a simple query overhead and hence is very easy to implement. Additionally, such a method would (intuitively) have a meager false-positive rate (although, it was reported that often blacklisting suffered from non-trivial false-positive rates [3]).However, it is almost impossible to maintain an exhaustive list of malicious URLs, especially since new URLs are generated every day. Attackers use creative techniques to evade blacklists and fool users by changing the URL to "appear" legitimate via obfuscation. Garera et al. [4] identified four types ofobfuscation: Obfuscating the Host with an IP, Obfuscating the Host with another domain, obfuscating the host with large hostnames, and misspelling. All of these try to hide the malicious intentions of the website by masking the malicious URL. Recently, with the increasing popularity of URL shortening services, it has become a new and widespread obfuscation technique (hiding the malicious URL behind a short URL) [5], [6]. Once the URLs appear legitimate, and users visit them, an attack can be launched. This is often done by malicious code embedded into the JavaScript. Often the attackers will also try to obfuscate the code to prevent signature-based Tools for identifying them. Attackers use several other simple techniques to evade blacklists including: fast-flux, in which proxies are automatically generated to host the web-page; algorithmic generation of new URLs; etc. Furthermore, attacker scan frequently launch more than one attack, which modifies the attack-signature, making it undetectable by tools that focus on specific signatures. Blacklisting methods later, have main drawbacks, and it appears almost trivial to bypass them, mainly because blacklists are useless for making predictions on new URLs. To address these issues, we came up with machine learning techniques for Malicious URL Detection Machine Learning approaches, use a set of URLs as training data and based on the statistical properties; learn a prediction function to classify a URL as malicious or benign. This gives them the ability to generalize to new URLs unlike blacklisting methods editor@iaeme.com

3 M. Sridevi and Dr K.V.N. Sunitha 2. DATASET COLLECTION The essential for training machine learning models the presence of training data. In the perspective of malicious URL detection, this would correspond to a set of large number of URLs. Machine learning can broadly be classified into supervised, unsupervised, and semisupervised for the training data After the training data is collected, the next step is to extract informative features such that they sufficiently describe the URL and at the same time, they can be interpreted mathematically by machine learning models. Data extraction from two ways this may include lexical features (i.e URL information) and host based features (WHOIS info,) The first key step is to convert a URL u into a feature vector x, where several types of information can be considered and different techniques can be used. Unlike learning the prediction model, this part cannot be directly computed by a mathematical function (not for most of it). Using domain knowledge and related expertise, a feature representation is constructed by crawling all relevant information about the URL. These range from lexical information (length of URL, the words used in the URL, etc.) to host-based information (WHOIS info, IP address, location, etc.). Once the information is gathered, it is processed to be stored in a feature vector numerical features can be stored in x as is, and identity related information or lexical features are usually stored through a binarization or bag-of-words (BoW) approach. Based on the type of information used, d xt generated from a URL is a d-dimensional vector where d can be less than 100 or can be in the order of millions. Figure 2 Data collection process 3. MALICIOUS URL DETECTION In this section, we present the main principles used by researchers to solve the problem of Malicious URL detection, followed by validating it as a machine learning task Principles of Detecting Malicious URLs Several other methods have been endeavored to tackle the problem of Malicious URL Detection. According to the fundamental principles, these methods can be broadly grouped into two major categories: (i) Blacklisting or Heuristics, and (ii) Machine Learning approaches editor@iaeme.com

4 Malicious URL Detection and Prevention at Browser Level Framework 3.2. Blacklisting or Heuristic Approaches Blacklisting approaches are a conventional and classical technique for detecting malicious URLs, which often maintains a list of URLs that are known to be malicious. Whenever a new URL is visited, a database lookup is performed. If the URL is present in the blacklist, it is considered to be malicious, and then a warning will be generated; else it is assumed to be benign. Blacklisting suffers from the inability to maintain an exhaustive list of all possible malicious URLs, as new URLs can be easily generated daily, thus making it impossible for them to detect new threats [21]. This is particularly of critical concern when the attackers generate new URLs algorithmically, and can thus bypass all blacklists. Despite several problems faced by blacklisting [3], due to their simplicity and efficiency, they continue to be one of the most commonly used techniques by many anti-virus systems today. Heuristic approaches [7] are some extensions of Blacklist based methods, where in the idea is to create a "blacklist of signatures." Common attacks are identified, and based on their behaviors; a signature is assigned to this attack type. Intrusion Detection Systems can scan the web pages for such signatures, and raise a flag if some suspicious behavior is found. These methods have better generalization capabilities than blacklisting, as they can detect threats in new URLs as well. However, such methods can be designed for only a limited number of common threats, and cannot generalize to all types of (novel) attacks. Moreover, using obfuscation techniques, it is not difficult to bypass them. A more specific version of heuristic approaches is through analysis of execution dynamics of the webpage Machine learning approaches Figure 3 A framework for Malicious URL Detection using Machine Learning These approaches try to analyze the information of a URL and its corresponding websites or web pages, by extracting good feature representations of URLs, and training a prediction model on training data of both malicious and benign URLs. There are two-types of features that can be used - static features, and dynamic features. In static analysis, we perform the analysis of a webpage based on information available without executing the URL (i.e., executing JavaScript, or other code) [8] The features extracted include lexical features from the URL string, information about the host, and sometimes even HTML and JavaScript content. Since no execution is required, these methods are safer than the Dynamic approaches. The underlying assumption is that the distribution of these features is different for malicious and benign URLs. Using this distribution information, a prediction model can be built, which can make predictions on new URLs. Due to the relatively safer environment to extracting important information, and the ability to generalize to all types of threats (not just common editor@iaeme.com

5 M. Sridevi and Dr K.V.N. Sunitha ones which have to be defined by a signature), static analysis techniques have been extensively explored by applying machine learning techniques. 4. MACHINE LEARNING ALGORITHMS FOR MALICIOUS URL DETECTION After converting URLs into feature vectors, many of these learning algorithms can be applied to train a predictive model in a relatively straight forward manner. However, to efficiently solve the problem, some efforts have also been explored in devising specific learning algorithms learning algorithms that have been applied for this task, and even suggest suitable machine learning technologies that can be used to solve particular challenges encountered 4.1. Online learning Algorithms Although batch learning algorithms are standard and easy to use, they can suffer from several major limitations when dealing with real-world malicious URL detection tasks due to expensive retraining cost; batch learning algorithms often do not update the model frequently, making them difficult to capture some emerging threats ina timely way. To address these limitations, online learning algorithms have been emerging as a promising direction for resolving the Malicious URL Detection tasks Online Learning Online Learning signifies a family of efficient and accessible learning algorithms that acquire from sequentially data collection where Consider malicious URL detection, given a sequence d of T labeled instances, denoted by D ={f(x1; y1); : : : ; (xt ; yt )}, where x denotes the URL s feature representation, and y t { 1, 1} is the class label. y = +1Denotes a malicious URL, and y t = -1 denotes a benign URL. At each iteration t, the algorithm makes a prediction f (xt) = sign(w. xt) where w is a d-dimensional weight vector initialized to 0 at t = 0. After the prediction, the true class label yt is revealed to the learner, and based on the loss suffered, the learner makes an update of the model to improve predictions in the future. The wideranging framework of an online learning algorithm is outlined in Algorithm. t Online learning algorithms are frequently much more scalable than traditional batch learning algorithms. Both the learning and predicting are computationally very efficient, Formation it mostly suitable for malicious URL detection responsibilities with gradually enormous amounts of training data, where batch learning algorithms may suffer due to their expensive retraining and the high memory and computational constraints. Online learning algorithms are often developed with strong theoretical guarantees such that they can asymptotically learn the prediction models as good as the batch algorithms under mild assumptions editor@iaeme.com

6 Malicious URL Detection and Prevention at Browser Level Framework There is an extensive variety of machine learning algorithms in the collected works that can be straight used in the context of Malicious URL Detection. Due to hypothetically a tremendous size of training data there was a need for scalable algorithms, and that is why Online Learning methods have found greatly success in this domain. 5. CONCLUSIONS In this paper discussed regarding malicious URL detection and Prevention using online learning algorithm at browser level, a supervised learning approach used for train the dataset and which collected from WHOIS, and studied different approaches heuristic and blacklist, finally online learning is effective machine learning algorithm in order to detect and prevent Malicious URL. In this paper we prevented one vulnerability but web applications have multiple vulnerabilities as a Future enhancement to prevent multiple vulnerabilities at a single place using hybrid framework and which integrates features of Naïve base, TF-IDF and unsupervised learning. approach REFERENCES [1] Tran Phuong Thaoet, al, Classification of Landing and Distribution Domains Using Who is Text Mining, 2017 IEEEDOI /Trustcom/BigDataSE/ICESS [2] B. Liang, J. Huang, F. Liu, D. Wang, D. Dong, and Z. Liang, Malicious web pages detection based on abnormal visibility recognition," in E-Business and Information System Security, EBISS 09.International Conference on. IEEE, 2009, pp [3] S. Sinha, M. Bailey, and F. Jahanian, Shades of grey: On the effectiveness of reputationbased "blacklists,"" in Malicious and Unwanted Software, MALWARE rd International Conference on.ieee, 2008, pp [4] S. Garera, N. Provos, M. Chew, and A. D. Rubin, A framework fordetection and measurement of phishing attacks, in Proceedings of the2007 ACM workshop on Recurring malcode. ACM, 2007, pp [5] S. Chhabra, A. Aggarwal, F. Benevenuto, and P. Kumaraguru, Phi.sh/$ social: the phishing landscape through short URLs, in Proceedings of the 8th Annual Collaboration, Electronic messaging, Anti-Abuse and Spam Conference. ACM, 2011, pp [6] Y. Alshboul, R. Nepali, and Y. Wang, Detecting malicious short URLson Twitter," [7] C. Seifert, I. Welch, and P. Komisarczuk, Identification of malicious web pages with static heuristics, in Telecommunication Networks and Applications Conference, ATNAC Australasian. IEEE,2008, pp [8] J. Ma, L. K. Saul, S. Savage, and G. M. Voelker, Beyond blacklists: learning to detect malicious websites from suspicious URLs, in Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2009, pp [9] Karthik. V, Mohan Kumar S and Karthikayini. A Novel Survey on Location Based Node Detection and Identifying the Malicious Activity of Nodes in Sensor Networks. International Journal of Computer Engineering & Technology 8(2), 2017, pp [10] V. Jaiganesh, Dr. P. Sumathi, An Efficient Intrusion Detection Using Relevance Vector Machine, International Journal of Computer Engineering & Technology (IJCET), Volume 4, Issue 1, January- February (2013), pp [11] D. Rajalakshmi and Dr. K. Meena, A Survey of Intrusion Detection with Higher Malicious Misbehavior Detection in MANE T, International Journal of Civil Engineering and Technology, 8(10), 2017, pp [12] Nisma Mobinunnisa and V. Sesha Bhargavi, Detection of Multiple Malicious Nodes in MANETS in a Single Query. International Journal of Computer Engineering & Technology, 8(6), 2017, pp editor@iaeme.com