Improve Security-Events-Center to the Cloud Platform
|
|
|
- Tamsyn Erica Robinson
- 5 years ago
- Views:
Transcription
1 Improve Security-Events-Center to the Cloud Platform Building ICAS with Hadoop and HBase Wei-Yu Chen, Yao-Tsung Wang National Center for High-Performance Computing, Taiwan DATE: 05/08/2009 The Background Story Our Idea and Methods Experiment Results Conclusions Future Works 2
2 Personal Security Install Some Security Software 3 4
3 Internet Security 5 Network IDS Interface 6
4 These Events are MIS s Nightmare!!!!. Difficult to realize the overall accidents 2. Ignoring the crucial information easily!!! 7 The Security Events Center A platform whose purpose is to provide detection and reaction services to security incidents. Main functions Collects all information from both security and non-security products Carries out the unified automatic event evaluation to tell if they are complying with the policy. 8
5 SEC Overview SEC 9 The SEC Component Security Security Operation Events Center System Operation Unit Core Procedure Unit SECDB Soc DB Administrator Interface Event Reaction Sensor Format Transform Unit Sensor 2 Format Transform Unit... Sensor n Format Transform Unit Alert Generator Alert Generator Alert Generator 0
6 Alert Merge Example Destination IP Attack Signature Source IP Destination Port Source Port Packet Protocol Timestamp Host_ Trojan Sip tcp T Host_ Trojan Sip tcp T2 Host_ Trojan Sip tcp T3 Host_2 Trojan Sip tcp T4 Host_3 D.D.O.S Sip udp T5 Host_3 D.D.O.S Sip tcp T5 Host_3 D.D.O.S Sip udp T5 Host_3 D.D.O.S Sip tcp T5 Key Values Host_ Trojan Sip,Sip2 80, ,5002 tcp T,T2,T3 Host_2 Trojan Sip tcp T4 Host_3 D.D.O.S. Sip3,Sip4,Sip5,Sip ,6008 tcp, udp T5 Experiment : Scenario Clean. Probe 2. Break In 3. Escalation 4. Break In 5. DOS 6. DOS Linux 2
7 Experiment Result: Statistic name Alert Correlation Attack signature event name SHELLCODE x86 inc ebx NOOP NETBIOS SMB-DS lsass unicode little endian overflow attempt NETBIOS SMB-DS lsass DsRoler UpgradeDownlevelServer unicode Total event TCP 602 ICMP 395 UDP 96 MISC MS Terminal server request NETBIOS DCERPC ISystemActivator path overflow attempt little endian DDOS Trin00 Master to Daemon default password attempt Correlated event 8 Reduction rate 98.70% NETBIOS SMB-DS IPC$ unicode share access Victim IP Attacker IP What s problem about the SEC?. Enormous Data less Efficient 2. Got Nothing if the database were crash 3. Memory and CPU Exhausted when system is running. 4
8 ICAS, IDS Cloud Analysis System Applying Cloud Computing technique Higher capability Fault tolerance Making alerts algorism to generate manifest report Reducing redundancy Merge relation 5 ICAS Overview ICAS 6
9 Change SEC to ICAS MySQL HBase Core Procedure Map-Reduce Linux Hadoop + Linux Single Machine Multiple Machine 7 System Architecture ICAS Component Overview 8
10 Program Procedure 9 Change SEC to ICAS 20
11 Linux Single Machine Hadoop + Linux Multiple Machine Applications MySQL -> HBase Program language PHP -> JAVA & JSP Data transfer Snort to MySQL -> Log to HDFS 2 Core Procedure Map-Reduce Format Transfer Unit Setup Snort logging to MySQL Setup MySQL client logging to remote MySQL server Core Procedure Unit Fuse redundant data Merge data as event 22
12 Core Procedure Map-Reduce Regular Parser Parsing original snort log and transfer to HDFS (hadoop file system) Analysis Procedure Dispatch job if pool is not empty and insert the result into database Data Mapper <key, value> mapping Data Reducer < key, value valuen> < key2, value valuen> 23 MySQL HBase 24
13 MySQL sec_event HBase 25 MySQL HBase Row Key Time Stamp Column "contents:" Column "anchor:" Column "mime:" com.cnn.www t5 "anchor:cnn si.com" CNN t4 "anchor:my. look.ca" CNN.com t3 <html>... text/html t2 <html>... t <html>... 26
14 MySQL HBase Row Key Time Stamp Column signature:" Column Infor:" Column " SourceIP :" (Destination IP) t sig Infor :Port p,p2.. sip dip t2 sig2 Infor :... (the other info) o,o2 sip2 dip2 Infor : Experiment Machine: CPU : Intel quad-core, Memory : 2g, OS : Linux : Ubuntu 8.04 server Software : version Hadoop : Hbase : 0..3 Java : 6 Alerts Data Sets MIT Lincoln Laboratory, Lincoln Lab Data Sets Computer Security group at UCDavis, tcpdump file 28
15 000 Experimental Result The Consuming Time of Each Number of Data Sets Traditional nodes 2 nodes 4 nodes 6 nodes Analysis Time (sec) Alerts Experimental Result Throughput Data Overall 29 30
16 Pros & Cons Efficient Scalable Economical Reliable Non-realtime Latency Immature 3 Hadoop Development Issues Fully based on correct Hadoop s Version ( Neither backward nor upward compatibility) ICAS can work on Hadoop ICAS has 8 errors and 8 deprecations on Hadoop ICAS has 26 errors and 22 deprecations on hadoop A word-count sample code on hadoop 0.20 can t work for hadoop 0.8 HBase s A version is only correspond to Hadoop s A version (upper or lower not work) Sample codes are hardly to find Deeply in Object-oriented programming 32
17 ICAS supplies a efficient way to analyze and merge huge number of alerts based on cloud platform Until now, there are many components needed to implement 33 Future Works : Overview 34
18 Future Work : Final Result 35 Future Works Including more IDS logs Graphical final result Prepare more large-scale and complete experiment 36
19 Thank You! & Question? DATE:4/4/09 37
An Indian Journal FULL PAPER ABSTRACT KEYWORDS. Trade Science Inc. The study on magnanimous data-storage system based on cloud computing
[Type text] [Type text] [Type text] ISSN : 0974-7435 Volume 10 Issue 11 BioTechnology 2014 An Indian Journal FULL PAPER BTAIJ, 10(11), 2014 [5368-5376] The study on magnanimous data-storage system based
Botnet Detection Using Honeypots. Kalaitzidakis Vasileios
Botnet Detection Using Honeypots Kalaitzidakis Vasileios Athens, June 2009 What Is Botnet A Botnet is a large number of compromised computers, controlled by one or more Command-and-Control Servers, the
HBase: Overview. HBase is a distributed column-oriented data store built on top of HDFS
HBase 1 HBase: Overview HBase is a distributed column-oriented data store built on top of HDFS HBase is an Apache open source project whose goal is to provide storage for the Hadoop Distributed Computing
Venusense UTM Introduction
Venusense UTM Introduction Featuring comprehensive security capabilities, Venusense Unified Threat Management (UTM) products adopt the industry's most advanced multi-core, multi-thread computing architecture,
Network Security. Chapter 0. Attacks and Attack Detection
Network Security Chapter 0 Attacks and Attack Detection 1 Attacks and Attack Detection Have you ever been attacked (in the IT security sense)? What kind of attacks do you know? 2 What can happen? Part
TEL
2003 6 Snort TEL 06-2533131 2605 E-mail ccsu@mail.stut.edu.tw m9090102@email3.stut.edu.tw paper, we use Open Source like Snort[10] to construct the Intrusion Detection System (IDS). Snort system will produce
Incorporating Network Flows in Intrusion Incident Handling and Analysis
Regional Visualization and Analytics Center Incorporating Network Flows in Intrusion Incident Handling and Analysis John Gerth Stanford University gerth@stanford.edu FloCon 2008 1 EE/CS Network Infrastructure
Data Informatics. Seon Ho Kim, Ph.D.
Data Informatics Seon Ho Kim, Ph.D. seonkim@usc.edu HBase HBase is.. A distributed data store that can scale horizontally to 1,000s of commodity servers and petabytes of indexed storage. Designed to operate
Configuring attack detection and prevention 1
Contents Configuring attack detection and prevention 1 Overview 1 Attacks that the device can prevent 1 Single-packet attacks 1 Scanning attacks 2 Flood attacks 3 TCP fragment attack 4 Login DoS attack
EXTRACT DATA IN LARGE DATABASE WITH HADOOP
International Journal of Advances in Engineering & Scientific Research (IJAESR) ISSN: 2349 3607 (Online), ISSN: 2349 4824 (Print) Download Full paper from : http://www.arseam.com/content/volume-1-issue-7-nov-2014-0
Column Stores and HBase. Rui LIU, Maksim Hrytsenia
Column Stores and HBase Rui LIU, Maksim Hrytsenia December 2017 Contents 1 Hadoop 2 1.1 Creation................................ 2 2 HBase 3 2.1 Column Store Database....................... 3 2.2 HBase
High Performance Computing on MapReduce Programming Framework
International Journal of Private Cloud Computing Environment and Management Vol. 2, No. 1, (2015), pp. 27-32 http://dx.doi.org/10.21742/ijpccem.2015.2.1.04 High Performance Computing on MapReduce Programming
McAfee Network Security Platform 9.2
McAfee Network Security Platform 9.2 (9.2.7.22-9.2.7.20 Manager-Virtual IPS Release Notes) Contents About this release New features Enhancements Resolved issues Installation instructions Known issues Product
(some more) DCE-RPC Tips & Tricks. Renaud Bidou Security Consultant Radware EMEA
(some more) DCE-RPC Tips & Tricks Renaud Bidou Security Consultant Radware EMEA renaudb@radware.com 1 Any security device can be bypassed Let s prove it 2 Introduction 3 Objective Use old exploit oc192
Network Intrusion Analysis (Hands on)
Network Intrusion Analysis (Hands on) TCP/IP protocol suite is the core of the Internet and it is vital to understand how it works together, its strengths and weaknesses and how it can be used to detect
Intrusion Detection System (IDS) IT443 Network Security Administration Slides courtesy of Bo Sheng
Intrusion Detection System (IDS) IT443 Network Security Administration Slides courtesy of Bo Sheng 1 Internet Security Mechanisms Prevent: Firewall, IPsec, SSL Detect: Intrusion Detection Survive/ Response:
Configuring attack detection and prevention 1
Contents Configuring attack detection and prevention 1 Overview 1 Attacks that the device can prevent 1 Single-packet attacks 1 Scanning attacks 2 Flood attacks 3 TCP fragment attack 4 Login DoS attack
Chapter 5. The MapReduce Programming Model and Implementation
Chapter 5. The MapReduce Programming Model and Implementation - Traditional computing: data-to-computing (send data to computing) * Data stored in separate repository * Data brought into system for computing
Fujitsu/Fujitsu Labs Technologies for Big Data in Cloud and Business Opportunities
Fujitsu/Fujitsu Labs Technologies for Big Data in Cloud and Business Opportunities Satoshi Tsuchiya Cloud Computing Research Center Fujitsu Laboratories Ltd. January, 2012 Overview: Fujitsu s Cloud and
FROM LEGACY, TO BATCH, TO NEAR REAL-TIME. Marc Sturlese, Dani Solà
FROM LEGACY, TO BATCH, TO NEAR REAL-TIME Marc Sturlese, Dani Solà WHO ARE WE? Marc Sturlese - @sturlese Backend engineer, focused on R&D Interests: search, scalability Dani Solà - @dani_sola Backend engineer
Data Clustering on the Parallel Hadoop MapReduce Model. Dimitrios Verraros
Data Clustering on the Parallel Hadoop MapReduce Model Dimitrios Verraros Overview The purpose of this thesis is to implement and benchmark the performance of a parallel K- means clustering algorithm on
Hackveda Training - Ethical Hacking, Networking & Security
Hackveda Training - Ethical Hacking, Networking & Security Day1: Hacking windows 7 / 8 system and security Part1 a.) Windows Login Password Bypass manually without CD / DVD b.) Windows Login Password Bypass
Chapter 7. Denial of Service Attacks
Chapter 7 Denial of Service Attacks DoS attack: An action that prevents or impairs the authorized use of networks, systems, or applications by exhausting resources such as central processing units (CPU),
Alert Correlation System with Automatic Extraction of Attack Strategies by Using Dynamic Feature Weights
Alert Correlation System with Automatic Extraction of Attack Strategies by Using Dynamic Feature Weights Chih-Hung Wang*, Ye-Chen Chiou Department of Computer Science and Information Engineering, National
Different attack manifestations Network packets OS calls Audit records Application logs Different types of intrusion detection Host vs network IT
Different attack manifestations Network packets OS calls Audit records Application logs Different types of intrusion detection Host vs network IT environment (e.g., Windows vs Linux) Levels of abstraction
Advanced Network Troubleshooting Using Wireshark (Hands-on)
Advanced Network Troubleshooting Using Wireshark (Hands-on) Description This course is a continuation of the "Basic Network Troubleshooting Using Wireshark" course, and comes to provide the participants
Anomaly Detection in Communication Networks
Anomaly Detection in Communication Networks Prof. D. J. Parish High Speed networks Group Department of Electronic and Electrical Engineering D.J.Parish@lboro.ac.uk Loughborough University Overview u u
Intrusion Detection. October 19, 2018
Intrusion Detection October 19, 2018 Administrative submittal instructions answer the lab assignment s questions in written report form, as a text, pdf, or Word document file (no obscure formats please)
Table of Contents. 1 Intrusion Detection Statistics 1-1 Overview 1-1 Displaying Intrusion Detection Statistics 1-1
Table of Contents 1 Intrusion Detection Statistics 1-1 Overview 1-1 Displaying Intrusion Detection Statistics 1-1 i 1 Intrusion Detection Statistics Overview Intrusion detection is an important network
WormTerminator: : An Effective Containment of Unknown and Polymorphic Fast Spreading Worms
WormTerminator: : An Effective Containment of Unknown and Polymorphic Fast Spreading Worms Songqing Chen, Xinyuan Wang, Lei Liu George Mason University, VA Xinwen Zhang Samsung Computer Science Lab, CA
Approaches to Event Prediction in Complex Environments
Calhoun: The NPS Institutional Archive Faculty and Researcher Publications Faculty and Researcher Publications 2012-08-02 Approaches to Event Prediction in Complex Environments Tan, Terence Monterey, California:
Introduction to Hadoop and MapReduce
Introduction to Hadoop and MapReduce Antonino Virgillito THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION Large-scale Computation Traditional solutions for computing large
ECE 697J Advanced Topics in Computer Networks
ECE 697J Advanced Topics in Computer Networks Network Measurement 12/02/03 Tilman Wolf 1 Overview Lab 3 requires performance measurement Throughput Collecting of packet headers Network Measurement Active
Networks and distributed computing
Networks and distributed computing Abstractions provided for networks network card has fixed MAC address -> deliver message to computer on LAN -> machine-to-machine communication -> unordered messages
Voldemort. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Overview Design Evaluation
Voldemort Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi Leader Election 1/29 Outline 1 2 3 Smruti R. Sarangi Leader Election 2/29 Data
Certified Snort Professional VS-1148
VS-1148 Certified Snort Professional Certification Code VS-1148 Vskills certification for Snort Professional assesses the candidate as per the company s need for network security and assessment. The certification
Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics
Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics Cy Erbay Senior Director Striim Executive Summary Striim is Uniquely Qualified to Solve the Challenges of Real-Time
MapReduce & BigTable
CPSC 426/526 MapReduce & BigTable Ennan Zhai Computer Science Department Yale University Lecture Roadmap Cloud Computing Overview Challenges in the Clouds Distributed File Systems: GFS Data Process & Analysis:
CS November 2018
Bigtable Highly available distributed storage Distributed Systems 19. Bigtable Built with semi-structured data in mind URLs: content, metadata, links, anchors, page rank User data: preferences, account
NIDS: Snort. Group 8. Niccolò Bisagno, Francesco Fiorenza, Giulio Carlo Gialanella, Riccardo Isoli
NIDS: Snort Group 8 Niccolò Bisagno, Francesco Fiorenza, Giulio Carlo Gialanella, Riccardo Isoli 1 Summary NIDS Snort Syn Flood Attack Exploit Kit Detection: Bleeding Life Packet Level Evasion Snort as
Distributed Face Recognition Using Hadoop
Distributed Face Recognition Using Hadoop A. Thorat, V. Malhotra, S. Narvekar and A. Joshi Dept. of Computer Engineering and IT College of Engineering, Pune {abhishekthorat02@gmail.com, vinayak.malhotra20@gmail.com,
Processing Technology of Massive Human Health Data Based on Hadoop
6th International Conference on Machinery, Materials, Environment, Biotechnology and Computer (MMEBC 2016) Processing Technology of Massive Human Health Data Based on Hadoop Miao Liu1, a, Junsheng Yu1,
CS November 2017
Bigtable Highly available distributed storage Distributed Systems 18. Bigtable Built with semi-structured data in mind URLs: content, metadata, links, anchors, page rank User data: preferences, account
Computer Forensics: Investigating Network Intrusions and Cybercrime, 2nd Edition. Chapter 2 Investigating Network Traffic
Computer Forensics: Investigating Network Intrusions and Cybercrime, 2nd Edition Chapter 2 Investigating Network Traffic Objectives After completing this chapter, you should be able to: Understand network
Improving Hadoop MapReduce Performance on Supercomputers with JVM Reuse
Thanh-Chung Dao 1 Improving Hadoop MapReduce Performance on Supercomputers with JVM Reuse Thanh-Chung Dao and Shigeru Chiba The University of Tokyo Thanh-Chung Dao 2 Supercomputers Expensive clusters Multi-core
Intrusion Detection Systems (IDS)
Intrusion Detection Systems (IDS) Presented by Erland Jonsson Department of Computer Science and Engineering Intruders & Attacks Cyber criminals Activists State-sponsored organizations Advanced Persistent
Efficient Load Balancing and Disk Failure Avoidance Approach Using Restful Web Services
Efficient Load Balancing and Disk Failure Avoidance Approach Using Restful Web Services Neha Shiraz, Dr. Parikshit N. Mahalle Persuing M.E, Department of Computer Engineering, Smt. Kashibai Navale College
Managing Latency in IPS Networks
Revision C McAfee Network Security Platform (Managing Latency in IPS Networks) Managing Latency in IPS Networks McAfee Network Security Platform provides you with a set of pre-defined recommended settings
System Design for a Million TPS
System Design for a Million TPS Hüsnü Sensoy Global Maksimum Data & Information Technologies Global Maksimum Data & Information Technologies Focused just on large scale data and information problems. Complex
Low-Cost Traffic Analysis of Tor
Low-Cost Traffic Analysis of Tor Steven J. Murdoch, George Danezis University of Cambridge, Computer Laboratory Review of Tor Support anonymous transport of TCP streams over the Internet Support anonymous
Defending against Flooding-Based Distributed Denial-of-Service Attacks: A Tutorial
Defending against Flooding-Based Distributed Denial-of-Service Attacks: A Tutorial Paper by Rocky K C Chang, The Hong Kong Polytechnic University Published in the October 2002 issue of IEEE Communications
Intrusion Detection. What is Intrusion Detection
Intrusion Detection 1 What is Intrusion Detection We are referering to the act of detecting an unauthorized intrusion by a computer on a Network. Attemp to compromise or otherwise do harm, to other Network
Snort Rules Classification and Interpretation
Snort Rules Classification and Interpretation Pop2 Rules: Class Type Attempted Admin(SID: 1934, 284,285) GEN:SID 1:1934 Message POP2 FOLD overflow attempt Summary This event is generated when an attempt
Huge Data Analysis and Processing Platform based on Hadoop Yuanbin LI1, a, Rong CHEN2
2nd International Conference on Materials Science, Machinery and Energy Engineering (MSMEE 2017) Huge Data Analysis and Processing Platform based on Hadoop Yuanbin LI1, a, Rong CHEN2 1 Information Engineering
Towards Real-Time, Many Task Applications on Large Distributed Systems
Towards Real-Time, Many Task Applications on Large Distributed Systems - focusing on the implementation of RT-BOINC Sangho Yi (sangho.yi@inria.fr) Content Motivation and Background RT-BOINC in a nutshell
Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical
Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or
A Robust Cloud-based Service Architecture for Multimedia Streaming Using Hadoop
A Robust Cloud-based Service Architecture for Multimedia Streaming Using Hadoop Myoungjin Kim 1, Seungho Han 1, Jongjin Jung 3, Hanku Lee 1,2,*, Okkyung Choi 2 1 Department of Internet and Multimedia Engineering,
Decision analysis of the weather log by Hadoop
Advances in Engineering Research (AER), volume 116 International Conference on Communication and Electronic Information Engineering (CEIE 2016) Decision analysis of the weather log by Hadoop Hao Wu Department
Performance and Scalability with Griddable.io
Performance and Scalability with Griddable.io Executive summary Griddable.io is an industry-leading timeline-consistent synchronized data integration grid across a range of source and target data systems.
Indexing Strategies of MapReduce for Information Retrieval in Big Data
International Journal of Advances in Computer Science and Technology (IJACST), Vol.5, No.3, Pages : 01-06 (2016) Indexing Strategies of MapReduce for Information Retrieval in Big Data Mazen Farid, Rohaya
Big Data XML Parsing in Pentaho Data Integration (PDI)
Big Data XML Parsing in Pentaho Data Integration (PDI) Change log (if you want to use it): Date Version Author Changes Contents Overview... 1 Before You Begin... 1 Terms You Should Know... 1 Selecting
Research on Load Balancing and Database Replication based on Linux
Joint International Information Technology, Mechanical and Electronic Engineering Conference (JIMEC 2016) Research on Load Balancing and Database Replication based on Linux Ou Li*, Yan Chen, Taoying Li
CE Advanced Network Security Botnets
CE 817 - Advanced Network Security Botnets Lecture 11 Mehdi Kharrazi Department of Computer Engineering Sharif University of Technology Acknowledgments: Some of the slides are fully or partially obtained
TCP Tuning for the Web
TCP Tuning for the Web Jason Cook - @macros - jason@fastly.com Me Co-founder and Operations at Fastly Former Operations Engineer at Wikia Lots of Sysadmin and Linux consulting The Goal Make the best use
Multi-phase IRC Botnet & Botnet Behavior Detection Model
Software Verification and Validation Multi-phase IRC Botnet & Botnet Behavior Detection Model Aymen AlAwadi aymen@tmit.bme.hu Budapest university of technology and economics Department of Telecommunications
CS395/495 Computer Security Project #2
CS395/495 Computer Security Project #2 Important Dates Out: 1/19/2005 Due: 2/15/2005 11:59pm Winter 2005 Project Overview Intrusion Detection System (IDS) is a common tool to detect the malicious activity
HP 5120 SI Switch Series
HP 5120 SI Switch Series Network Management and Monitoring Configuration Guide Part number: 5998-1813 Software version: Release 1505 Document version: 6W102-20121111 Legal and notice information Copyright
Improving the Database Logging Performance of the Snort Network Intrusion Detection Sensor
-0- Improving the Database Logging Performance of the Snort Network Intrusion Detection Sensor Lambert Schaelicke, Matthew R. Geiger, Curt J. Freeland Department of Computer Science and Engineering University
NIC TEAMING IEEE 802.3ad
WHITE PAPER NIC TEAMING IEEE 802.3ad NIC Teaming IEEE 802.3ad Summary This tech note describes the NIC (Network Interface Card) teaming capabilities of VMware ESX Server 2 including its benefits, performance
Top 25 Hadoop Admin Interview Questions and Answers
Top 25 Hadoop Admin Interview Questions and Answers 1) What daemons are needed to run a Hadoop cluster? DataNode, NameNode, TaskTracker, and JobTracker are required to run Hadoop cluster. 2) Which OS are
Linux Systems Security. Logging and Network Monitoring NETS1028 Fall 2016
Linux Systems Security Logging and Network Monitoring NETS1028 Fall 2016 Monitoring Monitoring can take many forms, from passive periodic inspection to realtime intrusion detection For this unit, we will
April Final Quiz COSC MapReduce Programming a) Explain briefly the main ideas and components of the MapReduce programming model.
1. MapReduce Programming a) Explain briefly the main ideas and components of the MapReduce programming model. MapReduce is a framework for processing big data which processes data in two phases, a Map
Distributed Systems 16. Distributed File Systems II
Distributed Systems 16. Distributed File Systems II Paul Krzyzanowski pxk@cs.rutgers.edu 1 Review NFS RPC-based access AFS Long-term caching CODA Read/write replication & disconnected operation DFS AFS
CIT 480: Securing Computer Systems
CIT 480: Securing Computer Systems Scanning CIT 480: Securing Computer Systems Slide #1 Topics 1. Port Scanning 2. Stealth Scanning 3. Version Identification 4. OS Fingerprinting CIT 480: Securing Computer
Research and Realization of AP Clustering Algorithm Based on Cloud Computing Yue Qiang1, a *, Hu Zhongyu2, b, Lei Xinhua1, c, Li Xiaoming3, d
4th International Conference on Machinery, Materials and Computing Technology (ICMMCT 2016) Research and Realization of AP Clustering Algorithm Based on Cloud Computing Yue Qiang1, a *, Hu Zhongyu2, b,
TITLE: PRE-REQUISITE THEORY. 1. Introduction to Hadoop. 2. Cluster. Implement sort algorithm and run it using HADOOP
TITLE: Implement sort algorithm and run it using HADOOP PRE-REQUISITE Preliminary knowledge of clusters and overview of Hadoop and its basic functionality. THEORY 1. Introduction to Hadoop The Apache Hadoop
Configuring Anomaly Detection
CHAPTER 9 Caution Anomaly detection assumes it gets traffic from both directions. If the sensor is configured to see only one direction of traffic, you should turn off anomaly detection. Otherwise, when
Network Vulnerability Scan
Network Vulnerability Scan Aug 28, 2018 Ethnio, Inc. 6121 W SUNSET BLVD LOS angeles, CA 90028 Tel (888) 879-7439 ETHN.io Summary Ethnio conducts ongoing network vulnerability scans using both Qualys and
Cycle Sharing Systems
Cycle Sharing Systems Jagadeesh Dyaberi Dependable Computing Systems Lab Purdue University 10/31/2005 1 Introduction Design of Program Security Communication Architecture Implementation Conclusion Outline
Facilitating Consistency Check between Specification & Implementation with MapReduce Framework
Facilitating Consistency Check between Specification & Implementation with MapReduce Framework Shigeru KUSAKABE, Yoichi OMORI, Keijiro ARAKI Kyushu University, Japan 2 Our expectation Light-weight formal
Martin Brunner, Jonas Pfoh Sept. 6, Sept. 24, 2010
Martin Brunner, Jonas Pfoh martin.brunner@sit.fraunhofer.de;pfoh@sec.in.tum.de IT Security Lab Technische Universität München Munich, Germany Sept. 6, 2010 - Sept. 24, 2010 Outline 1 2 3 4 5 6 Welcome
Asura: A huge PCAP file analyzer for anomaly packets detection using massive multithreading
Asura: A huge PCAP file analyzer for anomaly packets detection using massive multithreading DEF CON 26, Aug 12 2018 Ruo Ando Center for Cybersecurity Research and Development National Institute of Informatics
Configuring Anomaly Detection
CHAPTER 9 This chapter describes anomaly detection and its features and how to configure them. It contains the following topics: Understanding Security Policies, page 9-2 Understanding Anomaly Detection,
Security Operation Center Concepts and Implementation
Security Operation Center Concepts and Implementation renaud.bidou@intexxia.com > SOC Modules > Global Architecture > Collection & Storage > Correlation > SOC Modules R Box R Box reaction and reporting
CSE 565 Computer Security Fall 2018
CSE 565 Computer Security Fall 2018 Lecture 19: Intrusion Detection Department of Computer Science and Engineering University at Buffalo 1 Lecture Outline Intruders Intrusion detection host-based network-based
Configuring IP Logging
CHAPTER 8 This chapter describes how to configure IP logging on the sensor. It contains the following sections: Understanding IP Logging, page 8-1 Configuring Automatic IP Logging, page 8-2 Configuring
Lambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL. May 2015
Lambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL May 2015 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document
OUTLINE PERFORMANCE BENCHMARKING 7/23/18 SUB BENCHMARKING THE SECURITY OF SOFTWARE SYSTEMS OR TO BENCHMARK OR NOT TO BENCHMARK
BENCHMARKING THE SECURITY OF SOFTWARE SYSTEMS OR TO BENCHMARK OR NOT TO BENCHMARK mvieira@dei.uc.pt Department of Informatics Engineering University of Coimbra - Portugal QRS 2018 Lisbon, Portugal July
A Scalable and Reliable Message Transport Service for the ATLAS Trigger and Data Acquisition System
A Scalable and Reliable Message Transport Service for the ATLAS Trigger and Data Acquisition System Andrei Kazarov, CERN / Petersburg NPI, NRC Kurchatov Institute 19th IEEE Real Time Conference 2014, Nara
Shedding Light on the Configuration of Dark Addresses
Shedding Light on the Configuration of Dark Addresses Sushant Sinha, Michael Bailey, and Farnam Jahanian Department of Electrical Engineering and Computer Science University of Michigan Ann Arbor, Michigan,
SentinelOne Technical Brief
SentinelOne Technical Brief SentinelOne unifies prevention, detection and response in a fundamentally new approach to endpoint protection, driven by behavior-based threat detection and intelligent automation.
Enhanced Hadoop with Search and MapReduce Concurrency Optimization
Volume 114 No. 12 2017, 323-331 ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu ijpam.eu Enhanced Hadoop with Search and MapReduce Concurrency Optimization
CISC 7610 Lecture 5 Distributed multimedia databases. Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL
CISC 7610 Lecture 5 Distributed multimedia databases Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL Motivation YouTube receives 400 hours of video per minute That is 200M hours
CIS 601 Graduate Seminar. Dr. Sunnie S. Chung Dhruv Patel ( ) Kalpesh Sharma ( )
Guide: CIS 601 Graduate Seminar Presented By: Dr. Sunnie S. Chung Dhruv Patel (2652790) Kalpesh Sharma (2660576) Introduction Background Parallel Data Warehouse (PDW) Hive MongoDB Client-side Shared SQL
Hadoop, Yarn and Beyond
Hadoop, Yarn and Beyond 1 B. R A M A M U R T H Y Overview We learned about Hadoop1.x or the core. Just like Java evolved, Java core, Java 1.X, Java 2.. So on, software and systems evolve, naturally.. Lets
Pre processors. Detection Engine
Packet Decoder Pre processors Detection Engine Logging and Alerting System Output Modules Filesystem Syslog Database XML Firewall config You should know how the rules are constructed in order to fully
CS 6343: CLOUD COMPUTING Term Project
CS 6343: CLOUD COMPUTING Term Project Project Goal Explore existing Cloud storage systems Implement some components in Cloud storage systems to get a better understanding on the implementation issues in
DDoS Testing with XM-2G. Step by Step Guide
DDoS Testing with XM-G Step by Step Guide DDoS DEFINED Distributed Denial of Service (DDoS) Multiple compromised systems usually infected with a Trojan are used to target a single system causing a Denial
Honeypot Hacker Tracking and Computer Forensics
Honeypot Hacker Tracking and Computer Forensics Manfred Hung manfred.hung@pisa.org.hk Agenda Honeypot History Value of Honeypot Honeypot Technology Common Honypot products/solutions Honeypot deployment
Remote Procedure Call. Tom Anderson
Remote Procedure Call Tom Anderson Why Are Distributed Systems Hard? Asynchrony Different nodes run at different speeds Messages can be unpredictably, arbitrarily delayed Failures (partial and ambiguous)