Statistical Techniques for Network Security by Yun Wang


1358b14a40e3832-261x361.jpeg Author Yun Wang
Isbn 9781599047089
File size 9MB
Year 2008
Pages 476
Language English
File format PDF
Category security



 

 Statistical Techniques for Network Security: Modern Statistically-Based Intrusion Detection and Protection Yun Wang Center for Outcomes Research and Evaluation, Yale University and Yale New Haven Health, USA Information Science reference Hershey • New York ii Director of Editorial Content: Director of Production: Managing Editor: Assistant Managing Editor: Typesetter: Cover Design: Printed at: Kristin Klinger Jennifer Neidig Jamie Snavely Carole Coulson Carole Coulson Lisa Tosheff Yurchak Printing Inc. Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue, Suite 200 Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com/reference and in the United Kingdom by Information Science Reference (an imprint of IGI Global) 3 Henrietta Street Covent Garden London WC2E 8LU Tel: 44 20 7240 0856 Fax: 44 20 7379 0609 Web site: http://www.eurospanbookstore.com Copyright © 2009 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Wang, Yun. Statistical techniques for network security : modern statistically-based intrusion detection and protection / by Yun Wang. p. cm. Includes bibliographical references and index. Summary: “This book aims to provide a guide for applying modern statistical techniques for intrusion detection and prevention, and serve as a reference for individuals such as network administrators, information and network security specialists, IT professionals, IT-related risk analysis and management professionals, and students and researchers who are interested in the use of statistical techniques for network security”--Provided by publisher. ISBN 978-1-59904-708-9 (hbk.) -- ISBN 978-1-59904-710-2 (ebook) 1. Computer networks--Security measures 2. Computer security. I. Title. TK5105.59.W36 2008 005.8--dc22 2008023192 British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is original material. The views expressed in this publication are those of the authors, but not necessarily of the publisher. If a library purchased a print copy of this publication, please go to http://www.igi-global.com/agreement for information on activating the library's complimentary electronic access to this publication. To My Parents and My Family iv Table of Contents Preface.............................................................................................................................. viii Acknowledgment............................................................................................................... xi Section I: Foundations Chapter I Statistical Opportunities, Roles, and Challenges in Network Security..........................1 Introduction...........................................................................................................................1 Overview...............................................................................................................................2 Statistical Approaches in Practice......................................................................................14 Fundamental Statistical Roles and Challenges in Network Security..................................22 Summary.............................................................................................................................27 References...........................................................................................................................27 Endnotes..............................................................................................................................34 Chapter II Statistical Analysis Software............................................................................................35 Introduction.........................................................................................................................35 The SAS System...................................................................................................................37 STATA . ................................................................................................................................45 R..........................................................................................................................................48 Other Packages...................................................................................................................54 Summary.............................................................................................................................58 References...........................................................................................................................59 Endnotes..............................................................................................................................59  Chapter III Network Traffic and Data.................................................................................................60 Introduction.........................................................................................................................60 System-Specific Traffic........................................................................................................60 User-Specific Data..............................................................................................................65 Publicly Available Data......................................................................................................70 Summary.............................................................................................................................91 References...........................................................................................................................92 Endnotes..............................................................................................................................94 Appendix.............................................................................................................................95 Chapter IV Network Data Characteristics.......................................................................................104 Introduction.......................................................................................................................104 Random Variables ............................................................................................................105 Variable Distributions.......................................................................................................109 Network Data Modules.....................................................................................................116 Summary...........................................................................................................................121 References.........................................................................................................................121 Section II: Data Mining and Modeling Chapter V Exploring Network Data................................................................................................124 Introduction.......................................................................................................................124 Descriptive Analysis..........................................................................................................125 Visualizing Analysis..........................................................................................................134 Data Transformation.........................................................................................................144 Summary...........................................................................................................................155 References.........................................................................................................................156 Appendix...........................................................................................................................157 Chapter VI Data Reduction................................................................................................................172 Introduction.......................................................................................................................172 Data Structure Detection..................................................................................................173 Sampling Network Traffic.................................................................................................188 Sample Size.......................................................................................................................199 Summary...........................................................................................................................206 References.........................................................................................................................206 Appendix...........................................................................................................................210 Chapter VII Models Network Data for Association and Prediction................................................220 Introduction.......................................................................................................................220 Bivariate Analysis.............................................................................................................221 Linear Regression Modeling.............................................................................................232 Robustness Association.....................................................................................................249 Summary...........................................................................................................................257 References.........................................................................................................................257 Appendix...........................................................................................................................260 Chapter VIII Measuring User Behavior...............................................................................................261 Introduction.......................................................................................................................261 User Behavior Pattern......................................................................................................262 Scoring Methods...............................................................................................................276 Profiling Models................................................................................................................286 Summary...........................................................................................................................297 References.........................................................................................................................297 Appendix...........................................................................................................................301 Section III: Classifications, Profiles, and Making Better Decisions Chapter IX Classification Based on Supervised Learning............................................................. 305 Introduction.......................................................................................................................305 Generalized Linear Methods ...........................................................................................306 Nonparametric Methods...................................................................................................316 Other Linear and Nonlinear Methods...............................................................................333 Summary...........................................................................................................................342 References.........................................................................................................................343 Endnote.............................................................................................................................347 Chapter X Classification Based on Unsupervised Learning..........................................................348 Introduction.......................................................................................................................348 Probability Models............................................................................................................349 Similarity Models..............................................................................................................365 Multidimensional Models.................................................................................................379 Summary...........................................................................................................................390 References.........................................................................................................................392 Appendix...........................................................................................................................395 Chapter XI Decision Analysis in Network Security.........................................................................396 Introduction.......................................................................................................................396 Analysis of Uncertainty.....................................................................................................398 Statistical Control Chart...................................................................................................411 Ranking.............................................................................................................................415 Summary...........................................................................................................................422 References.........................................................................................................................423 Appendix...........................................................................................................................425 Chapter XII Evaluation . .....................................................................................................................427 Introduction.......................................................................................................................427 Data Reliability, Validity, and Quality..............................................................................428 Goodness of Classification................................................................................................435 Assess Model Performance...............................................................................................447 Summary...........................................................................................................................455 References.........................................................................................................................456 About the Author............................................................................................................458 Index.................................................................................................................................459 viii Preface This book describes modern statistical techniques for intrusion detection and prevention from an applied perspective. The use of statistical techniques in network security has attracted a lot of attention by researchers and professionals from the information system, computer science, and statistics fields over the last three decades. The idea behind intrusion detection and prevention seems simple: use legitimate user behavior patterns to identify anomalous user behavior patterns. Unfortunately, intrusion detection and prevention are difficult tasks to implement, and each has its own set of unique challenges. This discipline is still in development and there are many complicated topics that future research still needs to address. Any IP addresses mentioned or used in the book are synthetic and they do not reflect IP addresses in the real world This book aims to provide a guide for applying modern statistical techniques for intrusion detection and prevention, and serve as a reference for individuals such as network administrators, information and network security specialists, IT professionals, IT-related risk analysis and management professionals, and students and researchers who are interested in the use of statistical techniques for network security. Since the topics coved in this book are interdisciplinary (network security, risk management, and statistics), how to present materials from these different fields effectively and efficiently has proven to be a great challenge. As a result, this book attempts to focus on the application perspective with minimal statistical theoretical statements. Each section generally begins with a limited number of necessary statistical concepts that are related to the topic being presented, and is then followed by examples to help readers better understand specific topics. References are provided at the end of each chapter to help readers gain a more detailed and topic-specific understanding of the materials. There are over 80 examples in this book, nearly all of which are accompanied by corresponding programming codes. These examples can benefit readers by motivating and enhancing the understanding of the discussed concepts and topics. Although there is no prerequisite for readers to have taken an elementary statistics course before using this book, familiarity with basic statistical concepts and statistical analytic tools will be helpful in better utilizing the materials introduced in this book. The book contains 12 chapters and appendices. The chapters are divided into three sections: Foundations (Chapter I through Chapter IV), Data Mining and Modeling (Chapter V through Chapter VIII), and Classifications, Profiles, and Making Better Decisions (Chapter IX through Chapter XII). Topics are introduced hierarchically. Chapter I starts with an introduction to the history of network security in intrusion detection and prevention, is followed by an overview of the statistical approaches in practice, and ends with discussions of fundamental statistical roles and challenges in the network security field. Chapter II provides ix a quick review of statistical analytic packages, mainly focusing on SAS, Stata, and R, but also discussing S-Plus, WinBugs, and MATLAB. Essential features and attributes of each of these software applications are reviewed and discussed with examples. Chapter III covers network traffic and data (both system-specific data and user-specific data). This chapter also introduces some popular and publicly available datasets: (1) the Defense Advanced Research Projects Agency (DARPA) Intrusion Detection Evaluation Offline Data developed by the Lincoln Laboratory at the Massachusetts Institute of Technology (MIT), (2) the Third International Knowledge Discovery and Data Mining Tools Competition’s (KDD-Cup) 1999 data that was created based on the MIT-DARPA data, (3) the spam-email data created by Mark Hopkins, Erik Reeber, George Forman, and Jaap Suermondt at the Hewlett-Packard Labs, (4) the web visit sequence data donated by Jack S. Breese, David Heckerman, and Carl M. Kadie at Microsoft Research, and (5) the masquerading user data created by Matt Schonlau from RAND. All of these datasets were available to be downloaded off the web for free during the time this book was written. The programming codes used to process these datasets are also included (except for the masquerading user data). Despite the fact that the MIT-DARPA data has contributed tremendously to research progress in the field of intrusion detection during the past decade, it has been criticized and considered by many researchers. More specifically, critiques focus on the data being outdated and unable to accommodate the latest trend in attacks. However, this data is still the largest publicly available data for researchers today, and is still the most widely used public benchmark for testing intrusion detection systems. One of the important values of this data is to be a proxy for developing, testing and evaluating detecting algorithms, rather than to be a solid data source for a real time system. If a detection and prevention system has a high performance based on the MIT-DARPA data, this system is more likely to have a good performance in real time data, which is why the MIT-DARPA data was chosen, used and adapted within many examples in this book. Chapter IV examines the characteristics of network traffic and data. Limited and essential concepts, including variables, distributions and data types, are reviewed to provide readers with a statistical background. Chapter V reviews the methods for exploring network data. It covers both descriptive and visualizing analyses, which aim to detect data structures and data attributes. Approaches for normalized, centralized and standardized data are also provided in this chapter. Materials introduced in this chapter are important for gaining necessary techniques and tools to understand the unique characteristics of network traffic data. Chapter VI covers the topic of data reduction. Factor analysis and statistical sampling approaches are introduced for eliminating unnecessary and redundant variables and reducing the size of network traffic data. Chapters VII and VIII cover the topics of modeling network traffic for association and prediction and measuring and profiling user behavior. Chapter VII introduces various approaches to modeling for association and prediction, including bivariate analysis, linear regression, and the time-to-event modeling, which is important when tracking site or system behavior patterns change over time. Chapter VII also examines several approaches for selecting robust predictors, such as bootstrapping simulation and stepwise procedures. Chapter VIII reviews the characteristics and attributes of user behavior patterns and presents score modeling approaches to measure user and system behavior. Chapter VIII also introduces the methods for profiling user behavior, including use of the item response modeling and hierarchical generalized linear modeling techniques for the network security area. Classification is the key task in network security. Profiling user or system behavior is only meaningful when a robust classifier exists. In general, classification in network  security aims to achieve three goals: (1) determine what the classes or categories of the entire network traffic should be (e.g., normal connections, attacks, anomaly connections), (2) develop an algorithm to classify data into these classes, and (3) validate this algorithm. Chapters IX through XII are designed to address these three goals. Chapters IX and X focus more specifically on the first and second goals, and cover various modern supervised and unsupervised statistical learning topics with examples and programming codes for most of the topics discussed. Chapter IX examines both parametric- and nonparametric-based classification techniques, including logistic, Poisson, probit regressions, linear discriminant analysis, k-nearest neighbor, Naïve Bayesian approach, regression trees, and support vector machines. Chapter X discusses unsupervised learning techniques for network traffic classification. Topics cover probability models, measure of similarity, and multidimensional analyses. Various techniques, such as latent class model, hidden Markov model, k-means clustering, principal component analysis, multidimensional scaling, and self-organizing maps are discussed. Finally, Chapters XI and XII provide discussions on decision analysis and evaluation of classification results. Statistical simulation techniques, along with the interval estimates, are introduced to address uncertainty in network security at the beginning of Chapter XI. Statistical control charts and ranking methods are discussed to support the ability to make better network security-related decisions. Chapter XII covers various methods and techniques used to assess the reliability, validity and quality in network traffic data, as well as the procedures used for evaluate the goodness of classification and model performance. Covered topics include sensitivity, specificity, receiver operating characteristic curve, misclassification, goodness-of-fit, predictive ability, and residual analysis. The appendices include supplementary materials, such as large tables, long programming codes, programming macros, and analysis results that are useful, but not necessary to present in formal chapters or sections of this book. xi Acknowledgment This book has benefited greatly from the assistance of many people. First, I would like to express my sincere appreciation to Dr. Harlan Krumholz (Yale University) for the help, understanding, interest, advice, and support that he has given me during the 15 years that I have been working with him. Without him, this project would not have been possible. I am extremely grateful to Dr. Sharon-Lise Normand (Harvard University), who has provided invaluable help, advice, and guidance in both advanced statistical theory and applied statistics for many years. I would like to thank my colleagues and friends, including Drs. Martha Radford (New York University), JoAnne Foody (Harvard University), Jonathan Fine (Norwalk Health), Jennifer Mattera, Judy Lichtman (Yale University), and Thomas Meehan (Qualidigm) for many useful and insightful discussions. Although many of these discussions were restricted to the classification and profiling tasks in the healthcare area, they provided the roots for me to expand basic principles into the network security area. Special appreciation goes to Jane Yoon (Harvard University), who provided many helpful suggestions and a thorough, line-by-line critique of the manuscript. I would also like to acknowledge Emi Watanabe (Yale University) for her great work in producing many of the figures in this book. Assistance was also provided by Tierney Giannotti and Lynda Grayson (both at Qualidigm), and Mian Wang (Massachusetts Institute of Technology). I am grateful to Dr. Inyoung Kim (Virginia Tech) for an early discussion of the content of this book, Nate Melby (Trane Inc.), my editor at IGI Global, and the anonymous reviewers for their reading and detailed comments, which have substantially improved the presentation of the research provided in this book. Finally, I am deeply appreciative to Marcia Petrillo, CEO of Qualidigm, who provided me the opportunity to work with her 15 years ago after I had graduated. Without her influences, I would not have been able to conduct this project. I have worked on this book for many nights and weekends over the past years. I would like to thank my family members for their understanding, assistance and patience, and thank Lhong Dxj for her encouragements. It was their support, love, and help that allowed me to complete this project. I have tried to eliminate any and all foreseeable errors in this book, but I am sure that there may still be some errors, ambiguities or oversights that have not been revised, for which I take full responsibility. I would greatly appreciate any feedback and suggestions. They can be emailed to me at [email protected] xii The content of this publication does not necessarily reflect the views or policies of Yale University or Yale New Haven Health; nor does mention of trade names, commercial products, or organizations imply endorsement by Yale University or Yale New Haven Health. The author assumes full responsibility for the accuracy and completeness of the ideas represented. Yun Wang New Haven, CT April 2008 xiv Section I Foundations Statistical Opportunities, Roles, and Challenges in Network Security  Chapter I Statistical Opportunities, Roles, and Challenges in Network Security To me, a personal computer should be small, reliable, convenient to use and inexpensive. -The Apple-II, Stephen Wozniak Introduction In this chapter, we will provide a brief overview of network security and introduce essential concepts of intrusion detection and prevention and review their basic principles and guidelines. Then, we will discuss statistical approaches in practice as well as statistical opportunities, roles, and challenges in network security. Network security has become a very popular topic. A simple Google search based on the keyword “network security” showed 2.2 million items on February 29, 2008. Network security aims to protect the entire infrastructure of a computer network and its corresponding services from unauthorized access. The two key elements of network security are risk assessment and risk management. There are several fundamental components in network security: (1) security-specific infrastructures, such as hardware- and software-based firewalls and physical security approaches, (2) security polices, which include security protocols, users’ authentications, authorizations, access controls, information integrity and confidentiality, (3) detection of malicious programs, including anti-viruses, worms, or Trojan horses, and spyware or malware, and (4) intrusion detection and prevention, which encompasses network traffic surveillance and analyzing and profiling user behavior. Since the topic of network security links a great number of research areas and disciplines, we will focus on the component of intrusion detection and prevention in this book. Readers Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.  Wang who are interested in other components or want to gain more detailed information on the entire topic may refer to Smedinghoff (1996), Curtin (1997), Garfinkel and Spafford (1997), McClure, Scambray, and Kurtz, (1999), Strebe and Perkins (2000), Bishop (2003), Maiwald (2003), Stallings (2003), Lazarevic, Ertoz, Kumar, Ozgur, & Srivastava, (2003), Bragg, Rhodes-Ousley, Strassberg (2004), McNab (2007), and Dasarathy (2008). For wireless network security, Vacca (2006) provides an essential step-by-step guide that explains the wireless-specific security challenges and tasks, and for mobile phone related intrusion detection refer to Isohara, Takemori & Sasase (2008). Finally, for an overall introduction on network security, including key tools and technologies used to secure network access, refer to Network Security Principles and Practices by Malik (2003) and Network Security Fundamentals by Laet & Schauwers (2005). The use of statistical techniques in network security for intrusion detection and prevention has attracted great attention by researchers from both statistical and computer science fields. The idea behind intrusion detection and prevention is to use normal (anomaly-free) patterns of legitimate user behavior to identify and distinguish the behavior patterns of anomalous users (Anderson, 1972; Anderson, 1980; Stallings, 2003), and although this idea seems simple, intrusion detection and prevention are difficult tasks to implement and each have their own set of unique challenges. This discipline is in development and many difficult topics for research need to be addressed (INFOSEC Research Council, 1999; McHugh, 2000b; Taylor & Alves-Foss, 2002). Ideally, a perfect detection system needs four essential characteristics: (1) the ability to detect a wide variety of intrusions, (2) the ability to detect intrusions in a timely fashion, (3) the ability to present the analysis in a simple format, and (4) the ability to perform these tasks accurately (Bishop, 2003). Although statistical methods have been adapted to achieve these goals over the past decades (Anderson, 1980; Vaccaro & Liepins, 1989; Lunt & Jagannathan, 1988; Smaha, 1988; Teng, Chen & Lu, 1990; Anderson, Frivold & Valdes, 1995; Forrest, Hofmeyr, Somayaji & Longstaff, 1996; Qu, Vetter & Jou, 1997; Neumann & Prras, 1999; Masum, Ye, Chen & Noh, 2000; Valdes & Skinner, 2000; Barbard, Wu & Jajodia, 2001; Jha, Tan & Maxion, 2001; Taylor & Alves-Foss, 2001; Zhang, Li, Manikopoulos, Jorgenson & Ucles, 2001; Ye, Emran, Chen & Vilbert, 2002; Shyu, Chen, Sarinnapakorn, & Chang, 2003; Zhou & Lang, 2003; Qin & Hwang, 2004; Leung & Leckie, 2005; Wang 2005; Wang & Cannady 2005, Wang & Seidman, 2006; Wang & Normand, 2006; Gharibian & Ghorbani, 2007; Khan, Awad & Thuraisingham, 2007; Wang 2007; Herrero, al. et, 2007; Nayyar & Ghorbani, 2008), the gap between the performance of what we expect and what is currently available in both intrusion detection and intrusion prevention systems is still remarkable. With rapid advancements being made in computer and network technology, as well as increasing information and national security threats, the demand for reducing this gap has increased significantly; regardless, there are great challenges and technical difficulties in overcoming such a gap. In the following sections, we will briefly review previous studies and discuss some basic challenges. More historical information and trends on this topic also can be found from McHugh (2000b) and Patcha & Park (2007). Overview Although the idea behind intrusion detection is simple—using normal patterns of legitimate user behavior to identify and distinguish the behavior of an anomalous user, intrusion detecCopyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited. Statistical Opportunities, Roles, and Challenges in Network Security  tion is a difficult task to implement. In the below sections, we will briefly review the history of network security, outline some basic concepts of intrusion detection and prevention and present some principles and guidelines for developing a robust security system. Brief History of Network Security Since James P. Anderson (1972) outlined the increasing awareness of computer security problems and presented a project plan to address computer security challenges in 1972 for the United States Air Force (USAF), interest in intrusion detection research has been growing. For more than three decades, this issue has evolved from its embryonic stage, through early development, to today’s modern era as briefly summarized below. Embryonic Stage (1970-1979) During this period, almost all research was performed under United States Government contracts and limited to providing security solutions for sharing classified information on the same network without compromising security. The report of the Defense Science Board Task Force (DSBTF) on Computer Security written by Willis H. Ware (1970), a researcher from the RAND Corporation, raised for the first time the broader issue of using computers to store and process classified information in a multi-user, multi-access, multi-job environment regardless of the configuration. Later, the USAF published two reports: Computer Security Technology Planning Study Volumes I and II – authored by James P. Anderson (1972) that presented a detailed plan for USAF computer security requirements and outlined the USAF’s increasing awareness of computer security problems. The Volume II report described an advanced development and engineering program with a multi-level secure computing capability to address computer security problems, and also addressed the related development of communication security products and the interim solution to computer security challenges during the early 1970s. These reports established an important milestone in the computer security field during its early development. Anderson has been an independent contractor since the 1960s, and his work has centered on research and development activities for United States Government agencies that needed to make computer security a reality. He contributed to the Trusted Computer System Evaluation Criteria (TCSEC) and the draft of the Network Interpretation of the TCSEC (Department of Defense, 1985). Anderson received the National Computer System Security Award given jointly by the National Computer Security Center and NIST in 1990. Other important studies conducted during the 1970s included the USAF report, Multics Security Evaluation: Vulnerability Analysis by Karger and Schell (1974), the U.S. Department of Commerce’s report, Operating System Structures to Support Security and Reliable Software by Linden (1976) and the Stanford Research Institute’s study, Provably Secure Operating System, conducted by Neumann and his colleagues (1975). Nevertheless, after more than 30 years, not only does the initial security challenge of how to share classified information on the same network without compromising security still remain, but new challenges, such as wireless network security, Internet security, are also emergening. During this period, one remarkable milestone in the discipline of computer science is the birth of the Apple1-I and Apple-II computer designed by Stephen Wozniak, which started the Macintosh 2 and Personal Computer era. The Apple-I was designed late in 1975 and later sold nationwide through retail computer stores with a price below $700. It was the first microprocessor system product on the market to completely integrate the display generation circuitry, microprocessor, memory and power supply on the same board, which Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.  Wang allowed the users to run the Apple BASIC interpreter with no additional electronics other than a keyboard and video monitor (Wozniak, 1977). The Apple II was released on April 1977 with 48K RAM, 4k ROM, 1.0 MHz CPU, and a generic cassette drive with a price of US $2638. Figure 1.1 shows a revised mode of Apple II, the Apple II plus released on 1979 with an external 143K floppy drive. Nevertheless, while researchers and consumers were intoxicated with the capabilities demonstrated by Apple computers, a new era of computer and network security, the anticomputer’s virus or simple anti-virus had been initializing. Five years later, a 15-year-old high school freshman, Richard Skrenta, wrote the world’s first computer virus, Elk Cloner, on an Apple II computer. The virus started as a teenage prank ended up infecting floppy disks where the machines operating system resided. Early Period (1980-1989) James P. Anderson (1980) conducted the USAF computer security plan and published the results in 1980. In the 142-page report entitled Computer Security Threat Monitoring and Surveillance, he described the fact that network audit data collected daily from all computers can play an important role in the security program for a network system and that these data should be used to monitor threats. However, at that time, the importance of such data had not been realized, and all available security efforts were focused on denying access to sensitive data from an unauthorized source. The conclusions of the report did not suggest the elimination of any existing security audit data collection and distribution; rather, it recommended using the data for detecting intrusion, which was an initial effort to provide an opportunity for applying statistical science in network security. This report is considered the most important article that initiated the original intrusion detection concept and established the notion of intrusion detection. In 1983, Dorothy Denning, a researcher from SRI International, led a project that focused on analyzing audit trail data from government mainframe computers and created profiles of user activities. Between 1984 and 1986, Denning (1984) and her team researched and developed the first model for intrusion detection known as the Intrusion Detection Expert System (IDES), which was initially a rule-based expert system trained to detect known misuse activity. In 1987, Denning published the decisive work entitled An Intrusion-Detection Model, which revealed the necessary information for commercial intrusion detection system development. In this paper, Denning proposed the first comprehensive model of intrusion detection systems based on the hypothesis that a computer system’s vulnerabilities involve abnormal use of the system. Therefore, security violations could be Figure 1.1 Apple II plus (image from http://www.oldcomputers.net) Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited. Statistical Opportunities, Roles, and Challenges in Network Security  detected from abnormal patterns of system usage. The article demonstrated that to detect abnormal patterns, one could construct a model based on six main components: subjects, objects, audit records, profiles, anomaly records, and activity rules. This paper was another important milestone in the history of intrusion detection. Although the idea was proposed in 1987, it remains true today, and the six components have become individual research topics for modern intrusion detection. The birth of TCSEC is a remarkable event during this period. TCSEC, as defined in the Department of Defense (DoD) TCSEC 5200.28-STD, December 1985, is a DoD’s standard that establishes basic requirements for assessing the effectiveness of computer security controls built into a computer system. This standard finally led to the birth of Air Force Systems Security Instructions 5024 (AFSSI 5024) on September 1997. Another remarkable milestone in this period is the birth of the World Wide Web (WWW) in 1989, which was delivered by Tim Berners-Lee from the United Kingdom and Robert Cailliau from Belgium who was working in Switzerland. The Web is a collection of interconnected documents and other resources through the Internet that is subsequently, a collection of interconnected computer networks. Since the creation of the Web, network security has encountered a series of new generational challenges including anti-attacks from the Web. During the late 1980s, there were other significant advanced developments. In 1988, the Haystack system was developed by S. Smaha (1988), a researcher at Lawrence Livermore Laboratories at the University of California at Davis, to assist USAF security officers in detecting misuse of the mainframes at USAF bases. This project created an intrusion detection system that analyzed audit data by comparing them with defined patterns. In 1989, members from the original Haystack project formed a commercial company called Haystack Labs and released the newest version of the technology, Stalker, which was a host-based, pattern-matching detection system with robust search capabilities to either manually or automatically query the audit data. Also in 1988, the Morris worm caused the Internet to be unavailable for several days. This incident highlighted the need for computer security. Since 1980, the Institute of Electrical and Electronics Engineers (IEEE) Symposium on Security and Privacy has been the premier forum for the presentation of developments in computer security and electronic privacy. In 1998 the Journal of Cryptology established and issued its first volume. As an official journal of the International Association for Cryptologic Research, the journal provides a forum for publication of original results in all areas of modern information security and brings together researchers and practitioners in the field. Development and Commercial Period (1990-1999) The theme of network security on intrusion detection research entered a new phase in the 1990’s. Unlike the previous decade in which the U.S. Government founded most studies and the scope of work was limited directly to governmental priorities, two significant changes occurred. First, the increasing threat and need for network security had attracted the attention of researchers from many academic institutions, and new ideas and studies were introduced and conducted at academic levels. 1990 was another important milestone year in the history of intrusion detection; a year in which Heberlein (1990) and his colleagues from the University of California at Davis published their article entitled A Network Security Monitor. This article introduced a new concept to the intrusion detection research area; instead of examining audit trail data on a host computer system, network traffic could be used for monitoring suspicious behavior. As a result, two intrusion detection system’s research branches were formed: Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.

Author Yun Wang Isbn 9781599047089 File size 9MB Year 2008 Pages 476 Language English File format PDF Category Security Book Description: FacebookTwitterGoogle+TumblrDiggMySpaceShare Intrusion detection and protection is a key component in the framework of the computer and network security area. Although various classification algorithms and approaches have been developed and proposed over the last decade, the statistically-based method remains the most common approach to anomaly intrusion detection. Statistical Techniques for Network Security: Modern Statistically-Based Intrusion Detection and Protection bridges between applied statistical modeling techniques and network security to provide statistical modeling and simulating approaches to address the needs for intrusion detection and protection. Covering in-depth topics such as network traffic data, anomaly intrusion detection, and prediction events, this authoritative source collects must-read research for network administrators, information and network security professionals, statistics and computer science learners, and researchers in related fields.       Download (9MB) Dynamic Networks and Cyber-Security Hadoop Security: Protecting Your Big Data Platform Data Analysis For Network Cyber-Security Fraud Analytics Using Descriptive, Predictive, and Social Network Techniques: A Guide to Data Science for Fraud Detection The Practice of Network Security Monitoring Load more posts

Leave a Reply

Your email address will not be published. Required fields are marked *