What is an intrusion detection system?

Before today's article begins , Please watch this video first , It's easy to get into a door ：

https://cloud.tencent.com/developer/video/23621

Intrusion detection overview

A malicious attack on an organization's information base by electronic means is called information warfare (information warfare) . The purpose of the attack may interfere with the normal activities of the organization , Even attempt to cause serious damage to the organization's information base . All kinds of countermeasures against information warfare can be classified into three categories ： Protect 、 testing 、 Respond to .

** Protect ** ( Intrusion prevention ) Refers to the protection hardware 、 Software 、 Data protection against various attacks . At present, various network security facilities such as firewall and VPN, Various encryption technologies , Authentication technology , Vulnerable scanning and so on are within the scope of protection , They are the first line of defense for computer systems .

** testing ** ( Intrusion detection ) Study how to detect network attacks efficiently and correctly . Intrusion prevention alone is not enough to protect the security of computers , There are inevitable defects in any system and protocol , It may be the protocol itself or the implementation of the protocol , There are also social relations issues beyond technology , Can threaten information security . So even with these protective measures , The intruder may still exploit the corresponding flaw to break into the system , This means that intrusion detection can not be replaced by other security measures .

** Respond to ** ( Intrusion response ) It is the processing work after intrusion detection , It mainly includes loss assessment , Eradicate the back door left by the invaders , Data recovery , Collect evidence left by intruders, etc . These three security measures constitute a complete information warfare defense system .

Intrusion detection （Intrusion Detection ,ID） Is one of the topics discussed in this chapter , It monitors certain information of the computer system , To analyze , Detecting intrusion , And react . The system information detected by the intrusion detection system includes system records , The network traffic , Application logs, etc . intrusion （ Intrusion ） It is defined as unauthorized computer users and improper use (misuse) A legitimate user of a computer ( Internal threats ), Endangering or attempting to endanger the integrity of resources 、 confidentiality 、 Usability behavior . intrusion detection system （Intrusion DetectionSystem,IDS） It is the hardware and software to realize the intrusion detection function . Intrusion detection is based on the assumption that , namely ： Intrusion behavior is significantly different from normal behavior , So it can be detected . The research of intrusion detection began in 20 century 80 years , Get into 90 In the s, intrusion detection has become a hot spot in research and application , There have been many research prototypes and commercial products .

Intrusion detection system is the complement of intrusion prevention system in function , It is not a substitute for intrusion prevention system . contrary , It works with these systems , Detect attacks that have evaded the control of these systems . Intrusion detection system is a kind of computer system security 、 The second line of defense for network security .

An ideal intrusion detection system has the following characteristics ：

• Capable of continuous operation with minimal human intervention .
• Be able to recover and reset from system crash .
• Can resist attack .IDS You must be able to monitor yourself and detect whether you have been changed by an attacker .
• The runtime takes up the least overhead of the system .
• It can be configured according to the security policy of the monitored system .
• Be able to adapt to changes in system and user behavior during use .

As the size of the monitored system and the chance of being attacked increase , We believe that the following characteristics are equally important ：

• It can detect the network with a certain scale .
• Guarantee IDS When a part is broken , The impact on the rest should be as small as possible .
• Allow dynamic reconfiguration , That is, it must have the ability to be configured again without restarting .
• Provide a very low false positive rate .
• Provide interoperability , Running in different environments IDS Components can interact .
• Provide a convenient user interface , Make it easy for managers to configure and monitor the system .
• It can detect intrusion in a real-time or near real-time manner .

Current intrusion detection systems （ Including research prototypes and commercial IDS） The number of has exceeded onehundred , They have only part of the above characteristics .

Intrusion detection system architecture

CIDF (Common Intrusion Detection Framework) Defines the universal IDS System structure , It divides the intrusion detection system into four functional modules ：

** Event generator **（Event generater, E-box） Collect intrusion detection events , And provide IDS Treatment of other parts , yes IDS Information sources for . Events cover a wide range , It can be either network activity or system call sequence and other system information . The quality of the event 、 Quantity and type are right IDS The impact on performance is significant .

** Event analyzer **（Analysis engine, A-box ） Analyze the input events and detect intrusion . many IDS All of the researches on this topic focus on how to improve the capability of event analyzer , Including improving the accuracy of identifying known intrusions and improving the probability of detecting unknown intrusions .

** Event database **（Event database, D-box ）E-boxes and A-boxes Generating a lot of data , The data must be stored properly , For future use . D-box Its function is to store and manage this data , be used for IDS Training and evidence preservation .

** Event responder **（Response unit, C-box） Respond to intrusions , This includes warning Administrators , Disconnect the intrusion connection , Eradicate the back door left by intruders and data recovery .

CIDF Summed up IDS The function of , And a reasonable division . This model can be used to describe the existing IDS The system structure of . Yes IDS It provides valuable guidance for the design and implementation of .

Intrusion detection system classification

In order to classify accurately , The first thing to do is to identify the IDS features .IDS It's a complex system , If only one feature is used for classification , The result will be rough . Therefore, this chapter compares... According to various characteristics IDS It is classified from different angles . The event analyzer is IDS The core of , Therefore, the detection methods are classified first . Secondly, it classifies from the perspective of event generator , Take the type of event or the method of event collection as the classification standard .

Classification of test methods

Intrusion detection methods can be broadly divided into two categories ： Abuse detection (misuse detection) 、 Anomaly detection (anomaly detection) . stay IDS in , Any one of these events can fall into three categories :

• Known intrusion
• Known normal state
• Unable to determine the status

The third event may be an unknown intrusion , It may also be normal , However, under the existing system and technology, it is impossible to determine . The current detection methods are to identify the known intrusion and the known normal state , Abuse detection identifies known intrusions , However, unknown intrusions in undeterminable States will be missed (false negative) , Anomaly detection will know the intrusion according to the known normal state 、 All undecidable states are treated as exceptions , Therefore, false positives will occur (falsepositive) .

1. Abuse detection

Based on knowledge of known intrusions , Detect intrusion in input events . This method does not care about normal behavior , Study only known intrusions , It can detect known intrusions more accurately , But the ability to detect unknown intrusion is limited . Most businesses today IDS All use this method . The technologies used in abuse detection include ：

2. expert system

Using expert system technology , Use rules to represent intrusions . It's usually used forward-chaining 、production-based And other expert system tools . for example DARPA Of Emerald project , take P-BEST The toolkit is applied to intrusion detection .

3. State transition model

The intrusion is represented as a series of system state transitions , Detect intrusions by monitoring changes in system or network status . A typical system is NetSTAT.

4. Protocol analysis and string matching

Match the known attack pattern with the input event to determine the occurrence of the intrusion , This method has high speed 、 Good expansibility , But it is prone to false positives . Typical systems include shadow、Bro and Snort etc. .

5. Anomaly detection

Contrary to abuse detection , Anomaly detection studies the normal state of the system , By monitoring user behavior patterns 、 Host system call characteristics 、 Network connection status, etc , Establish the normal model of the system . In operation , Compare the current system behavior with the normal model , Determine the nature of the event according to the degree of deviation from the normal . This method is likely to detect unknown intrusion and variant attack , However, the existing systems usually have a large number of false positives . The detection of unknown intrusion is IDS The most challenging question in , It is more difficult than improper behavior detection . Anomaly detection usually uses statistical methods and machine learning methods .

6. Statistical methods

Using statistical analysis method to establish the normal model of the system . Statistical data sources include ： User keystroke characteristics 、telnet The average length of the conversation, etc . The attribute of the event is judged by monitoring the deviation between the input value and the expected value ,Emerald and cmds All include this way .

7. Machine learning methods

Methods and tools in the field of machine learning, such as neural networks 、 data mining 、 Genetic algorithm (ga) 、 Application of Bayesian network and artificial immune system in anomaly detection . This method is also used to identify anomalies by establishing a normal model . Each method has different application scope and characteristics . One of the hotspots of current research is noise data learning .

8. Mixed detection

The above two kinds of detection methods have their own advantages , Abuse detection can detect known attacks accurately and efficiently ; Anomaly detection can identify unknown attacks . At present, any kind of system can not complete all the intrusion detection tasks well . blend IDS It contains both pattern recognition and anomaly recognition systems , And according to the characteristics of the two methods, the division of labor , It can accurately identify known attacks , Some unknown attacks can be found , It can reduce false positives and false negatives .Emerlad It is a typical hybrid system .

System structure classification

from IDS The monitored events can be classified into network-based events IDS（Network-based IDS,NIDS） And host based IDS. according to IDS The response mode of can be divided into real-time IDS And non real time IDS, It is divided into distributed events according to the way of collecting events IDS And centralized IDS.

Host based IDS Data sources include ： System call sequence , Storage system activity records , System logs, etc . Due to host based IDS Have a full grasp of the host information and have strong control over the host , Therefore, compared with network intrusion detection system , Its detection accuracy is higher , Lower false positive rate . Simultaneous host based IDS More difficult to cheat , The response to the attack is also more effective ： The intrusion connection can be disconnected , Kill process . Its disadvantage is that only one host can be protected , And a certain burden on the host , And poor portability .

Web based IDS Detect intrusion activity by monitoring network traffic , It is called network intrusion detection system for short (NIDS).NIDS It can protect the whole network , Its advantages are simplicity and portability . NIDS The use of will not have a significant impact on the existing network system , And can be applied to various network environments . Network intrusion detection system only deals with network data , The semantic grasp of data is insufficient , Vulnerable to attack and deception . Adapting to high-speed network and improving scalability are NIDS Problems to be solved .

Two of the above IDS Can not complete effective intrusion detection alone , The combination of the two can achieve the effect of learning from each other . At present, some businesses IDS This scheme has been adopted , Such as RealSecurity,Axnet.

IDS The application scale of is also an important parameter . Existing business IDS The scope of application is LAN . With the development of network intrusion , Attacks have evolved into coordinated attacks from different hosts . Detection of such attacks is an existing IDS Not competent , Need to rely on multi-point distributed network intrusion detection system , Detect through joint defense .

3、 ... and 、 A typical intrusion detection system

IDS From the last century 80 The years have begun , The first business IDS Also in the 1991 The year was born . At present, all kinds of IDS The number of research projects and commercial products is extremely large , The following is an introduction to the representative intrusion detection system , It's divided into business IDS、IDS Research projects and free software .

1. Open source IDS project ：Snort

Snort It is a network intrusion detection system based on abuse detection that runs on a single computer . Snort adopt libpcap Get network package , And analyze the protocol . It defines a simple and flexible network intrusion description language , Describe the network intrusion ( Intrusion signature or intrusion signal ).Snort Match and search network data according to intrusion description , Capable of detecting various network attacks and reconnaissance , Include : Buffer overflow attack , Port scanning , CGI attack ,SMB Reconnaissance, etc . It also provides a variety of attack response methods . For the latest attack methods , Use Snort Intrusion description language can write the description of new attacks quickly and easily , So that Snort Can detect this attack . stay Internet Publishing has been established on Snort Intrusion mode database site . Snort Is a very dynamic free software , Under the development of volunteers all over the world , Technology and functions are constantly improving .

2. Commercial products

Mainstream business in the international market IDS Most of the products are web-based , Systems using abuse detection methods . There are mainly ：

1、RealSecure

RealSecure from Internet Security Systems（ISS） Development , There are three system components ：

• Network intrusion detection agent,
• Host intrusion detection agent
• Administrative console .

RealSecure It belongs to distributed structure , Each network monitor runs on a dedicated workstation , Monitor different network segments . RealSecure The intrusion detection method belongs to abuse detection , It can detect almost all mainstream attacks , The seamless integration of host based detection and network-based detection is realized . For different applications such as Exchange、MS SQL、LDAP、Oracle and Sybase etc. ,RealSecure Special system agents are provided for intrusion detection . The whole system is configured by a management program and interacts with users , Information such as safety reports can be provided .RealSecure The disadvantage of is that you can't do package reassembly , This makes it vulnerable to deception .ISS In seven IDS Got the highest rating in the evaluation of .

2、NFR

Network Flight Recorder? (NFR) It is a network intrusion detection system based on abuse detection . It is available in two versions ： Business Edition , Research edition （ Provide source code ）, The release of the research version has been stopped . NFR Use the modified Libpcap Carry out network packet capturing , And has a perfect package analysis script language N-code, It can be used to write detection and processing programs for various attacks . NFR Is the world's first TCP Package reorganization function IDS product , This makes NFR Able to resist Ptacek and Newsham Proposed avoidance IDS Methods .

3. The research system mainly includes ：

1、EMERALD

EMERALD (Event Monitoring Enabling Responses to Anomalous LiveDisturbances) yes DARPA A sub project of intrusion detection research , It integrates the abuse detection module and the anomaly detection and analysis module for collaborative analysis . Its development fully applies the idea of software engineering , Each module of the system is independent 、 Reusability . The design objectives of the system include the rapid integration of novel analytical techniques , It can quickly adapt to the dynamic changes of the existing network environment configuration .

EMERALD The expert system is applied P-BEST Implement abuse detection , Application of statistical analysis technology to achieve anomaly detection . The system structure is a distributed structure that can be easily extended , The monitors can communicate with each other , Form a hierarchical structure . The monitor has cross network communication capability , Sharable analysis methods , Cooperate to detect distributed network attacks , Suitable for large-scale wide area network intrusion detection .EMERALD Still developing , The technologies and methods it uses represent IDS The direction of Technological Development .

2、JAM

JAM yes DARPA A research project of , The distributed multi-agent structure is adopted . Every agent Distributed learning on different data sources , And share knowledge .JAM Data mining technology and meta-learning Technology for anomaly detection . at present JAM From experimental system to application , The technology will be applied to NFR In the new generation of products .