Toivo Talikka

Total Data Pty Ltd

Computer system installation, support and IT management consultancy

Forestville NSW Australia        tel 0410 532 923       email


Intrusion Detection Systems

0        SUMMARY

[This research paper was produced as part of requirements of the postgraduate course "Management Issues in Telecommunications" at the Faculty of Business, University of Southern Queensland on 06/11/2001.]

This paper describes the concept of intrusion detection in local area networks, connected to the Internet.   An overview is presented of the development of intrusion detection from host based systems to network based systems.   The paper discusses in detail intrusion detection techniques and vulnerabilities in intrusion detection systems (IDS) and touches also intrusion detection and response (IDR) systems.  A distinction is made between host-based and network-based intrusion systems.   The network security standards, general connectivity and compatibility requirements of IDS are discussed, including the impact made by open source systems like FreeBSD and Linux.

Because of the increasing occurrences of network intrusions, IDS technologies have developed further in the last few years.  There have been a number of developments in the networking area, e.g. move to switched networks, and emergence of new security protocols, like IPsec.  Some of these technologies have made the protection of network traffic easier, but the use of encryption has also made it more difficult to detect patterns in the IP packets. Enhancements are being made into network protocols, e.g. ICMP Traceback message, and functionality is being added to network equipment, to turn network devices into 'active network' so that the intruders can be located and the intrusion contained.

Currently SME companies do not generally have the expertise to install and run their own IDS system.  An outsourced, fully managed solution may be applicable where a firewall system is not adequate.  In the future networks are expected to develop into the same direction as present day 'active networks', where protocols and the infrastructure offer more protection from intrusions.








5.1 Host Based IDS

5.2 Network Based IDD


6.1 Development Cycle

6.2 Open Source Environment


7.1 Ptacek And Newsham

7.2 Protecting NIDS


8.1 Standard Interfaces

8.2 Performance and Security

8.3 Active Networks

8.4 Internet in Space







The purpose of the paper is to provide background information in the selection and evaluation of Intrusion Detection Systems (IDS) and find areas for further research into the techniques and management of Intrusion Detection and Response (IDR) systems.

The research has involved recent academic reports and treatises on intrusion detection techniques and also information about circumventing intrusion detection systems, available from the security team communities. The paper has aimed at a balance between the two sources, academia and the security and hacking communities. 

The technology focus of the report is on the Network IDS (NIDS) products and the known vulnerabilities of the TCP/IP network protocols. The report summarises methods of detecting network intrusions and trends in the development of Intrusion Detection Systems in the last ten years.  It also attempts to map the potential of secure networking against the availability of penetration tools and malicious source code from the Internet.  The paper concentrates on the Internet connectivity and TCP/IP networks, even though e.g. IPX/SPX networks running Novell Netware are an important part of LAN solutions. The management of network protection systems is outside the scope of this paper.

The major findings of this report are in the current status and development directions of intrusion detection systems.  The published IDS test results quote good detection rates for old, known intrusion patterns but in the same tests done only three years ago the detection rates for new types of intrusions are still negligible.  Even though the theory behind active response capability has developed, the reliability of intrusion detection and response is still largely dependent on network design and the skills of the network administrator and the security officers.


There is a need for Intrusion Detection System in current local area networks to report security incidents and protect the network against intrusions from the Internet.    The rates of intrusions and security incidents have increased dramatically in the last few years.  CERT/CC, the Coordination Centre of the Computer Emergency Response Team at Carnegie-Mellon University, has recorded a doubling of both software vulnerabilities and reported security incidents every year since 1999.   In the first nine months of this year, 1820 software vulnerabilities were reported to CERT.  During the same period, 34754 incidents were reported (CERT  2001).

One of the challenges of a network or system administrator, involved in monitoring and maintaining the security of the network, is that the product descriptions by vendors and the expertise of their sales personnel and support staff are selectively biased towards the product they represent.  Any 'white papers' and product comparisons provided as part of the pre-sales campaign are oriented towards supporting one product.  It is difficult to get objective comparisons between different systems because of dependence on information provided by the vendors and the complexity of multi-vendor network environments. Kvarnström (1999) believed the IDS of the day were not mature enough for large scale enterprise deployment.

Early intrusion attacks and, consequently, intrusion detection expertise were developed in Unix environment.  The Lawrence Livermore National Laboratories in the U.S. published a guide for Unix system administrators describing techniques and actions in case their systems were targeted in an intrusion attack or used to attack other host systems (Pienarczyk et al. 1994).   Unix has traditionally been used in the universities, because of the free licenses from AT&T. A number of research projects in the last ten to fifteen years have concentrated on Unix based systems and related operating system research, as an example Kumar (1995).  

Research into intrusion detection up to four years ago has been summarised by Axelsson (1998).   There has been a marked shift from host-based intrusion detection systems to network-based systems but the difficulty of tracking transactions in a fast network with encrypted traffic has lead into a hybrid system of IDS.   There is a need for both policy based and anomaly based detection.  Axelsson prefers non-realtime detection and accurate reporting of events that happened, rather than the IDS issuing an immediate but vague warning (1998, pp.12-13).

In his later research Axelsson (2000, pp. 8-9) classifies ID systems into three categories, based on the type of intrusion the system most easily detects:

Axelsson mentions a trend towards increasing security and interoperability of the IDS, and a trend towards distributed intrusion detection in larger environments.  The number of intrusion detection systems capable of active response is increasing.  Axelsson points out that '[…] security concerns about the intrusion detection system itself raise the possibility of the attacker exploiting an active response to effect a denial of service attack on the system being monitored, or an innocent third party' (Axelsson 2000, p. 13).

Allen, Christie and others (2000) at the Software Engineering Institute of Carnegie-Mellon University produced a comprehensive report about the state of intrusion detection systems, which would have deserved more coverage in this paper than it has.  This paper has concentrated on finding the underlying vulnerabilities in the network technology and trends in the IDS architecture.


Practical tests with a number of IDS done by MIT Lincoln Laboratories with the US Air Force and DARPA (Defense Advanced Projects Agency, part of the U.S. Department of Defense, in 1998 involved simulated attacks over the Internet against remote systems:

It is difficult to prevent such attacks by the use of security policies, firewalls, or other mechanisms because system and application software always contains unknown weaknesses or bugs.  In addition, attackers continually exploit complex, often unforeseen, interactions between software components and/or network protocols.

                                                                        (Lippman, Fried et al. 2000, p.12)

In the DARPA tests in 1998, the off-line mode of the test involved detection of attacks from recorded network traffic and host audit logs.  In the on-line test the IDS was required to detect the intrusion during normal operation of the network.   The best performance was recorded from old, known attack types against Unix hosts, where the rate of detection was 63% to 93%. 

The evaluation focused on Unix hosts and attacks from remote hosts, as detected by eight research systems.  The test bed simulated an active network with hundreds of users and thousands of hosts. The actual test was 'an off-line blind evaluation' where each IDS provider were first given training data from the test bed to allow the IDS to be configured and trained for high detection rates and low numbers of false alarms.    Improved performance of the research systems was attributed to the availability of test data and the fact that the attack signatures were similar in the training and test data (Lippman, Fried et al. 2000, p. 12-4, 21, 25).  At the same time, 10 false alarms were produced per day.  

The 1999 evaluation included Microsoft Windows NT hosts and inside attacks.  The results were but 'less than 10' false alarms were produced per day (Appendix 2).  The paper reporting the test results recommends a combination of host-based and network-based intrusion detection system as the best alternative.

The state of the art in IDS, as described by critical research, is still not robust:

After more than two decades of research and development efforts, the leading IDSs still have marginal detection rates and high false alarm rates, especially in the face of stealthy or novel intrusions […].

                                                Lee et al. 2000, p. 2

Allen, Christie et al. (1999, p. vii) are concerned that Intrusion Detection tools are deployed before they are fully understood:  'Over-reliance on ID technologies can create a false sense of confidence about the degree to which tools are detecting intrusions against an organization's critical assets'.

The last three years have seen new theories and prototypes of IDS.  The major question is whether this progress is significant enough to move IDS into the mainstream of computer security applications.  Alternatively, totally different architectures and applications may be needed to inform network administrators that hackers have targeted their system.  Ideally, the IDS should also reveal the location, the Internet address, of the intruder.

The methodology in producing this paper was a selective study of recent research publications and identification of specific vulnerabilities highlighted as weaknesses of IDS. Some of the techniques to evade detection by IDS is discussed, based on material published by security communities. A number of recent trends will be summarised from the publications.  Considering the increasing acceptance of Linux as an operating system of choice for corporate use, Linux based IDS is discussed briefly.


The hacking process starts with finding the hacking target and then scanning it to identify its vulnerabilities.  There are many scanning port scanner applications available to scan a server, a computer or a whole network from the Internet.  Port scanners and other hacking tools use known vulnerabilities in the TCP/IP protocol to reveal information about computers in the target network.  The exploits most often used by hackers are:

(McClure et al. 1999, pp. 31-6)

The presence of port scans and probes will tell the network administrator that someone from the Internet may be soon targeting his or her system.   It is relatively simple for a network administrator to block ICMP echo enquiries in the firewall system at the public interface to the Internet.  However, a network cannot be completely isolated if it is connected to the Internet.  By definition web server requests need to be processed and email requests need to be serviced.  The doors need to be kept open for a number of packet types and hackers exploit this openness.  The hacker uses automated program tools to flood the network interface or the host computers with packets.  This type of Denial of Service attacks denies the network of critical resources like Domain Name Service.

The recent CERT paper by Houle and Weaver maps the history of multiple-source DoS or Distributed Denial of Service (DDOS) attacks.  The most common types of attacks are:

(Houle & Weaver 2001, p. 3)

The packet types used by intruders have not changed since 1999.  However, CERT reports the trend of increased automation in compromised systems, first by hackers using automated scanning tools to find vulnerable systems and then launching automated scripts to compromise the targets.   Intruders use lists of networks responding to IP broadcast packets and lists of vulnerable hosts like hosts running Microsoft IIS.  The propagation of the compromised system is done by distributing a port listener program, an SMTP server or web server software to the compromised host computer.

A recent trend is Windows based handlers or agents, slaves, distributing the attack.  Intruders are known to target ISPs with large ranges of Windows based users.

The CERT paper claims that VPN users on DSL or cable connection in networks like AOL may be assigned another IP address because of the VPN.  This practice bypasses the protection of the personal firewall run by the PC: 'The security policy of the controlling end of the VPN will dictate the exposure of the VPN client system'. (Houle & Weaver 2001, p.13-4). 

Another trend recorded by CERT is the use of vendor supplied default passwords on poorly configured routers.  Using routers as a base is apparently safe from 'rival intruders'.  CERT also expresses concern about the possibility of intruders launching an attack via routers on the routing protocols of the Internet itself, with 'potentially high impact'.

The mentality of 'gang warfare' between hacker groups seems to have lead to '… a trend toward non-disclosure within intruder communities.  Rival groups will often keep new exploits and attack tools private to gain some advantage over other rival groups' (Houle & Weaver 2001, p. 14).  As a result, a large scale DDoS attack using a new exploit may be launched as a total surprise to the security community.

Some intruders have started using the IRC (Internet Relay Chat) servers for their purposes, eliminating the need for implementing special protocols and ports, which would be easier to detect by IDS.  Also, with the increase in the numbers of intermediaries like 'handlers' and 'agents', the need to forge (spoof) the source IP address has disappeared. 

CERT mentions 'collateral damage', where a large increase of network traffic has resulted in problems in backing up large volumes of log data.  Increased traffic will also usually cause increase in the ISP charges for the affected network and flooding in increased address resolution activity, which slows down and interferes with the normal operation of the network.

The purpose of a Denial of Service attack is to cause problems in the network operation, disable critical functions, take over computers and access confidential information like credit card details for financial benefit.  It is essential that network administrators and security officers get an early warning of undesirable activities targeting their network.  The purpose of Intrusion Detection Systems is to provide such warnings and identify the type of attack and where it is coming from.


5.1 Host Based IDS

Research at the end of 1980s and during early to mid 1990s concentrated on intrusion detection by analysis of significant events as they had been recorded in the system logs.

The security policy of the computer system defines the expected behaviour, controlled by the operating system and user configuration settings, set by the system administrator.  Any deviations from the security policy are recorded in the system security and audit logs. Intrusion detection system is trying to find intrusion signatures or particular intrusion patterns in the system event trace.

Kumar states that computer systems '…are likely to remain unsecure for some time to come', due to 'continually shortened software life cycle times' resulting in 'poor designs or inadequate testing' (Kumar 1995, p. 4).

Early implementations of IDS concentrated on host based analysis of security and audit logs, an activity done after the actual attack had been perpetrated. Lunt (1993) describes the NIDES (Next-Generation Intrusion-Detection Expert System) system, developed ten years ago at SRI International.  It is an intrusion detection expert system, intended to report intrusions from both outside and inside the network, based on data from user sessions once the user is inside the system. NIDES processes audit data and raises alarms based on deviation from past user behaviour.  Lunt's paper contains a summary of methods of audit trail analysis:

(Lunt 1993, p. 2)

Patterns of computer system usage, revealed in performance summaries generated by the host system, can also be analysed for security purposes.  NIDES is based on user profiles and adaptive learning.  Trend analysis of user behaviour patterns  is suggested as a defence against insider attacks (Lunt 1993, p. 6).    Lunt also suggests keystroke dynamics as another characteristic to be used to verify the identity of a user. 

5.2 Network Based IDS

A network based intrusion detection system (NIDS) monitors network traffic from a separate device.   Gula lists the method of operation of NIDS succinctly:

(Gula 2000, slide 5)

The network ID systems have several limitations, including the need to synchronise with the host system's behaviour in particular handling of IP and TCP protocol exceptions (Ptacek & Newsham 1998, p. 21-2).   The intricacies of the network protocol have been traditionally encoded into the NIDS application. 

NIDS will usually send alerts by email to an SMTP server, or it will send SNMP messages to a network monitoring system.  All the network traffic is usually passed from the Internet to the local area network through a firewall.  Therefore, in theory, at least some NIDS functionality could be placed inside the firewall application or the NIDS should at least be working closely with the firewall functions.

The architecture of an IDS solution depends on how many hosts it is designed to monitor.  A network based IDS will inspect all the traffic destined towards the hosts on a particular LAN.    Kvarnström (1999, p.7-9) mentions IPsec as a limiting factor in the analysis of network data because of the encryption, confidentiality requirement prevents the decryption of data.  It is equally significant that local area networks have started using switches instead of hubs.  Therefore the traffic between hosts or clients and hosts on the LAN is not available for inspection unless special arrangements are done.  It is possible to connect a hub to the switched network and connect a host and the PC running the IDS or IDS agent to the hub.  This arrangement will make the packets exchanged with the host visible to the IDS.  This approach would work in a small environment where the hosts are in one location but in a distributed environment special arrangements would be required to connect the IDS agents on host computers to a central system. Management ports on switches is only partial solution, not suitable for medium heavily loaded networks because of performance penalties (Kvarnström 1999, p.9).  One option to solve the problem is to provide more low level IDS functionality through interoperable networking equipment like high-performance routers and switches.

Some of the challenges in the management of NIDS are false positives and false negatives.  A false positive occurs if an innocent network event is interpreted as an intrusion attempt.  A false negative happens if an intrusion attempt goes unnoticed. False negatives or the NIDS device's inability to detect malicious activities can cause problems by letting the network to be compromised. Kvarnström (1999) refers to false positives and the possibility of spoofed attacks and warns against 'returning fire' or starting a counter attack at innocent parties.

Timm (2001, 2001a) suggests that adequate NIDS management will reduce false positives and false negatives.   Some of the causes for false negatives have been listed in Appendix 1.   With tuning of NIDS the proportion of real alarms can be increased from 10% to 60-90% of the reported events.   Signature-based NIDS are efficient in detecting known attacks but may not detect unknown or modified attacks.   In anomaly-based NIDS unusual activity, out of the normal pattern of network traffic, triggers the alarm.  Even though false negatives are reduced, this method is not flexible.


6.1 Development Cycle

A number of approaches to develop the detection function of an IDS are presented in research publications, including statistical analysis, pattern recognition, neural networks and data mining.  Because of the complexity of the network functions, a number of techniques need to be perfected before a balance is achieved between network resource requirements, accuracy of detection and overall manageability of the IDS.  

Even though Kumar (1995, p.4) speaks about computer systems in general when he refers to 'continually shortened software life cycle times' resulting in 'poor designs or inadequate testing' and concludes that 'Computer systems are therefore likely to remain unsecure for some time to come'.   The same undoubtedly applies to intrusion detection systems.

Designing a comprehensive system requires a major development effort and uncompromising level of testing in a number of network environments.  The cost of the end product may therefore become prohibitive for a small enterprise.  The time needed for the development and testing can also mean that the techniques may already be obsolete by the time the product is available for sale.   The availability of open source applications with source code can shorten the development time and ensure that more functionality can be included in the IDS product.

6.2 Open Source Environment

The commercial IDS have proprietary design and their source code is not available for analysis (Ptacek & Newsham 1998, p. 7).   The open source application base offers an interesting environment for IDS.  It has been said that the open source development environment exposes the source code to potential hackers.  On the other hand, involvement by a large number of developers means also strict peer level code inspection and quality control.  When the software environment consists of possibly millions of lines of code, it is important that all the parts of the product have been inspected at the expert level for robustness.

Godfrey and Tu have studied the evolution of the Linux kernel. They quote Lehmann's law, according to which '[…] as a system grows in size, it becomes increasingly difficult to add new code, unless explicit steps are taken to maintain the overall design.'  In practice this means that the system growth slows down. However, the Linux kernel, in spite of consisting of over 2 million lines of code, has grown geometrically.  The authors attribute the success to the exposure to a large number of developers and careful guardianship of the development (Godfrey & Tu 2000, p. 1).

Even Microsoft uses the benefits of well-tested open source code in a critical area.  Part of the TCP/IP implementation in Windows 2000 comes from the open source Unix distribution FreeBDS (Gomes 2001).

The popularity of Linux and its robust networking functions have brought Linux applications into the NIDS market.  An example is the WatchGuard Firebox system. 

Not only has the operating system itself withstood the highest levels of public scrutiny in regard to its fundamental design, but the inevitable bug fixes […] have historically been available faster than those of the commercial operating system vendors.


The WatchGuard design process releases all modifications to the operating system kernel back into the public domain.  This process enables the Linux development community to scrutinize the changes we have made to ensure that the modifications are stable and reliable.

(WatchGuard 2001, p. 10)

The WatchGuard Firebox system uses a hardened Linux kernel, where all non-essential services have been removed.

An open source intrusion detection system called Snort has become popular as a free NIDS.  It is a packet sniffer and logging application that uses rules based logging.  It is easy to install and use (Roesch 1999).  Snort has been available under the Unix and Linux platforms but it is now also available to on Windows.


7.1 Ptacek And Newsham

Fail-open system stops protecting the system when the NIDS is the target of a DOS attack  (p.  10).  Fail-closed system leaves the system protected, even if the NIDS is the target of a DOS attack.  In practice this means leaving the network protected but inaccessible (Ptacek & Newsham 1998, p.10)

There are a number of ambiguities in network traffic analysis that require the IDS to have information about the topology of the network, the operating system of the hosts and their configuration.  It is difficult to design a reliable Network IDS because of differences in timing, inherent in the network operation, when the behaviour of the network hosts is compared with the behaviour of the NIDS.  The insertion and evasion work at the network (IP) and transport (TCP) layers.  An insertion attack occurs if the IDS accepts packets rejected by the host.  If the IDS is tuned to reject packets accepted which by the network host computer, an evasion attack occurs.    This is based on small but significant differences in how the host computers and the IDS accept packets.  An example is IP fragment overlap where Windows NT 4.0 and Solaris 2.6 handle the situation one way, but the BSD, Linux and HP-UX operating systems behave in a different way (Ptacek & Newsham 1998, pp.11-5, 22)

At the network (IP) layer, reassembly and fragmentation has been used as a form of attack where differences in the handling of overlapping data are used.  None of the four tested intrusion detection systems handled packet fragmentation  (Ptacek & Newsham 1998, pp.20-1, 51).   According to Roesch (1998), the early version of Snort did not handle packet reassembly.  He presents an idea of reporting packets smaller than 128 bytes to detect possible intrusion attacks using defragmentation.  

At the transport layer, in a connection oriented TCP session the IDS keeps track of the sequence numbers of the packets, trying to remain in synchronisation.  An IDS can be 'desynchronised' or confused about the sequence numbers.  Because of the complexity of keeping track of all the TCP sessions, faults in the programming logic can create vulnerabilities in individual ID systems, which can then be targeted in attacks (Ptacek & Newsham 1998, p.24).  Ptacek and Newsham have analysed the source code of BSD UNIX distribution at both IP and TCP level, to define situations where insertion techniques could be used to mislead the IDS.  This happens because the actual network host computer behaves in a different way by accepting the packets discarded by the IDS.  The network analysis task of IDS is complicated because of differences between implementation of the TCP/IP protocol in the operating systems.

A DOS attack or disabling the IDS from responding to further packets is a result of exhaustion of resources in the IDS by the attacker.  The CPU processing, memory and disk resources can be exhausted by generating events that fill the log files or network bandwidth by overloading the network with meaningless packets (Ptacek & Newsham 1998, p.39).

Because most IDS implementations work by listening to and analysing all network traffic by using their network interface in 'promiscuous' mode, the IDS has to process far more packets than any of the hosts on the network.  The network card interfaces become saturated with much less traffic than the network media itself.  This causes the IDS to miss essential packets from the attack (Ptacek & Newsham 1998, p. 40-42).

As a reactive ID system, the IDS is designed to shut down connections or block all traffic with the IP address identified as the source the attacks.  However, because the source information in IP packets can be forged, an attack can be designed to use false positives which causes the IDS to block traffic from the faked addresses, e.g. DNS servers.  Without access to DNS servers, in practice all connectivity will stop:  'Some types of attacks may never be a legitimate basis for deployment of countermeasures, simply due to the fact that they can be performed blindly using forged addresses' (Ptacek & Newsham 1998, p. 43).  The forensic or source information of IP packets in connectionless protocols cannot be trusted (p. 23).

Ptacek concludes that the results from testing the four intrusion detection systems, where none of the four products were able to assemble packets and forged packets could be inserted into the ID, show that ID systems have not been adequately.  He states that IDS requires '…a secondary source of information […] to conclusively identify which packets have been accepted by an end-system'.  However, Ptacek and Newsham also emphasise the importance how the user of IDS configures the system tested (p. 57).

Ptacek's and Newsham's paper is a classic in the security industry.  According to Roesch, Network Flight Recorder (NFR) is capable of IP fragmentation reassembly, it '…was the first product which could defeat anti-NIDS attacks outlined by Ptacek and Newsham' (Roesch 1999).

7.2 Protecting NIDS

An important part of a testing the security of a network by hackers is the identifying the NIDS.   Gula (1999a) lists the following as typical indications revealing the type of NIDS a company is using:

- DNS information, e.g.

- reference to a security system on the company's web site

- open TCP and UDP ports, revealed in a port scan

Gula mentions that the identification and testing of the Intrusion Detection System usually starts with low intensity attacks that are gradually intensified.  The sensitivity of the IDS to probes can reveal information about the type of IDS.   Gula (2000, slide 16) lists various insertion techniques like fragmentation and low time-to-live (TTL), applicable for targeting NIDS for denial of service attacks.

The security of the NIDS is very important, otherwise the Intrusion Detection System becomes just another source of vulnerability in the network.   Intruders can compromise applications by altering the program code or adding programs into program directories, replacing standard system programs.  The NFR (Network Flight Recorder) IDS application is run in production mode from the CD-ROM, to avoid anyone tampering with the program code.

One of the favourite hacking methods is the common buffer-overflow attack where an application or the operating system is compromised by forcing them to process malformed or extremely long input data, e.g. HTTP requests.  If the application or operating system does not validate the length of its input data, data buffers can overflow into the program stack area and program code injected into the input data will be executed.  However, if the application and the operating system are compiled using a product called StackGuard, the compiled code does not allow the attacker's program code into the stack for execution (Cowan et al. 1998).  Any IDS will need to be resistant to attacks like the buffer-overflow. 


8.1 Standard Interfaces

Intrusion Detection Systems have to interoperate with the network hosts and management applications.   The detection systems need regular updates of attack patterns and rules.  Ideally, the updates will be delivered in a standard format that can be easily applied to all the IDS and NIDS applications, running on different appliances and host systems in a large network environment.  The network based IDS need to exchange data with each other about incidents and user behaviour. 

The NIDS need to raise alerts with the network administrator or Site Security Office, using a standard interface, like SNMP (which can also introduce vulnerabilities into the network), to a network management system like HP Openview (Kvarnström 1999, p.22).

Standardisation of names for vulnerabilities and exposures used in IDS has been done by a  non-profit organisation Mitre, originating from Massachusetts Institute of Technology (MIT), working in close co-operation with U.S. Department of Defence and Federal Government.  Mitre maintains a list of standard names, a common dictionary and a list of compatible products and vendors who are making efforts to include functions compatible with Mitre's list of Common Vulnerabilities and Exposures (CVE).  CVE compatible products allow the user to search for information about a vulnerability using the CVE name (Mitre 2001).  One of the IDS applications using CVE is Network Flight Recorder (NFR 2001).  Mitre has also specified standard formats for log files.

The Common Intrusion Detection Framework has specified standards, to allow:

(CIDF 2000, p. 2)

In practice this means architectural conventions for presenting components of IDS, generalised intrusion detection objects (gidos).  Gido can describe an IDS event or an instruction or query to the IDS, or it can describe an IDS component. Application program interfaces (API) are also defined, to allow re-use of CIDF components (CIDF 2000, p. 4).

8.2 Performance and Security

Improvements in the LAN performance from 100Mbps to 1Gbps mean that any software monitoring traffic inside the LAN will need to keep up with the performance increase.   A typical LAN of a small to medium size enterprise (SME) is typically connected to the Internet at speeds of 64Kb to 256Kb (ISDN) or 256Kb to 1.5Mb (ADSL) per second.   At the border between the Internet and the LAN, the speed of transactions coming in from the Internet is limited by the lower speed of the Internet connection. 

The traffic monitoring, logging and report intrusion attempts need to be also closely associated with the operating system and network security policies.   Comprehensive logging, detection and reporting functions require a lot of available network bandwidth and processing power from the NIDS.   Therefore it may not be practical to centralise all the IDS processing and monitoring.  The processors of the workstations in the network are usually idle 90% of the time.  All this number crunching power can be utilised for distributed pattern matching and rules processing, if it can be done in a secure way, without compromising the integrity of the intrusion detection.

A likely future scenario will involve running a centrally controlled, combined intrusion detection, firewall and reporting application on all workstations and servers.  Protecting the system from both outside intrusions and inside attacks will prevent the network from being used as a base for launching attacks against other networks, saving the company from possible lawsuits in the future.

A number of personal firewall and intrusion detection products are on the market for PC users, like ZoneLabs, Norton Personal Firewall and BlackICE Defender.  The BlackICE product from Network ICE (recently acquired by Internet Security Systems, Inc.) can be installed in a managed configuration where updates can be downloaded to remote workstations from behind the firewall, under the control of a central management program (Dalton 2001).  However, a secure solution would require full integration with the operating system, to prevent the user from unloading or disabling the intrusion detection product.   It remains to be seen how the Internet Connection Firewall stands up to scrutiny as an integral part of the recently released Microsoft Windows XP operating system.  It cannot be used inside a network, it runs only on the computer directly connected to the Internet, or acting as the gateway to the Internet (Microsoft 2001).

8.3 Active Networks

In the last three years there has been an increase in the number of available intrusion tools and technologies which '…launch and control tens of thousands of attacks against one or more targets' (Carver et al. 2000, p. 110).  Carver et al. states that intrusion response has remained a manual process where IDS notifies the system administrator, who initiates a response, based on existing scripts and strategies.

Active networks, developed for military purposes, have capabilities used by NIDS to detect intrusions and trace the attacker back to the source (Wang et al. 2001).   An IETF workgroup is defining a traceable ICMP (ping) message, to be generated by routers (Bellovin et al. 2001, p.2). A proposal has been put to IETF to improve the performance of the tracing of ICMP by adding 'intention-driven' information to the trace message (Mankin et al. 2001).   It is likely that the Active Network technology is transferred to special applications in the civilian networks.  In the extreme situation, the concept of Active Network Response (ANR) means tracing and 'fencing' the intruder-attacker (Maughan 2000).

8.4 Internet in Space

Research is currently being done in how to use Internet protocols in the extreme conditions of space travel (Mitre 2001a).  The large distances and timing issues make it necessary to rely on store-and-forward technology instead of the current TCP/IP protocol and the PKI infrastructure.   The issue of security is still important (Cerf et al. 2001, p. 36-41).   In the context of space travel where commercial interests dominate the availability of funding, no space agency can afford to let unsolicited traffic into the communications network supporting the space stations, the lifeline catering for the space travellers.  These networks need to be totally closed and isolated from intrusions from the general Internet.  The future need for IDS in this particular area are limited because the infrastructure and the special protocols have to cater for full trust and complete trace of the packets exchanged in the 'interplanetary' net.   Ideally, that should have been the case also in the general Internet. 


The current state of intrusion detection systems relies on careful assessment of vendor proposals and requires a trained systems and network administrator to run the IDS.

A relatively high level of networking and security skills is required before a small to medium size enterprise (SME) can install and administer their own intrusion detection system. 

One option for an SME is to obtain the security services from their ISP, who installs and fully maintains a secure connection to the Internet, combining a firewall protection with intrusion detection.  The additional cost of a secure connection is possibly from $100 to $500 per month, depending on the available bandwidth.

If an SME wants to manage its own security arrangements, the installation of a good firewall application with a GUI-based management interface may cover all the intrusion detection needs. Assuming that the company has servers accessible from the Internet, it is important that all the security, application and system log files of the host computers are saved and backed up for later analysis.  Exception logs from the on-site firewall device should be available and backed up for analysis by security experts.  The firewall approach may not be able to automatically block the traffic generated by possible intruders, but at least the firewall limits the type of packets intruders can send to the network.

The firm may have on-site expertise to manage a network intrusion detection system. Alternatively, the risk of an intrusion to the firm's assets may justify an outsourced, managed installation of an IDS.  If this is the case, the IDS application should detect known attacks, have low rate of false alarms and it should be easily updated with latest attack patterns.  The NIDS should also monitor network usage patterns, to provide internal protection from misuse of network resources.  In busy web server environments consideration should be given to NIDS proven to be fully compatible with the network hosts, to avoid introducing any of the vulnerabilities described by Ptacek and Newsham (1998).

The DARPA IDS test results from 1999 (Appendix 2) were slightly better than the results from similar test in 1998 (the details of the test results were not available from the Internet).  The problems discussed in the research articles indicate that it will take a number of years of research and development before the concept of NIDS is so mature that the NIDS products are free from false positives and false negatives.   The tracing of the source of the intrusion requires support from a new generation of network routers and the introduction of the next version of IP protocol, IPv6.   The development of the NIDS technology is likely to converge with the technology used in Active Networks.  

Once the network infrastructure makes the task of penetrating host computers difficult enough, especially if the chances of getting caught increase at the same time, hacking as a hobby will lose its kudos and the hacker communities will dwindle.  If the infrastructure of the Internet supports security and tracing the intruders through built-in functions in the routers at the ISP level, the need for specialised NIDS applications will largely disappear, except in high security environments.

Until that happens, further research should be made into the available assessments of commercial and research intrusion detection and response systems.  However, at least two years delay can be expected in accessing the latest test results from the defence community - the latest publicly available DARPA test results and the U.S. Navy's IDS database from 1999 are good examples (Stocksdale 1999).  Further study of the development of the IDS reported by Axelsson (1998, 2000) and classification and comparison with the current status of systems listed in Allen et al. (1999) would be a good starting point.


ADSL              - Asymmetric Digital Subscriber Line

AN                  - Active Network

ANR                - Active Network Response

CERT              - Computer Emergency Response Team

DDOS             - Distributed Denial of Service

DOS                - Denial of Service

GUI                 - Graphical User Interface

ICMP              - Internet Control Message Protocol

IETF                - Internet Engineering Task Force

IP                     - Internet Protocol

IPsec                - Secure IP

IPv4                 - Internet Protocol version 4, the current standard

IPv6                 - Internet Protocol version 6, the new standard with larger address base

ISDN               - Integrated Services Digital Network

ISO                  - International Standards Organisation

NAT                - Network Address Translation

OSI                  - Open Systems Interconnection, defined by International Standards Organisation (ISO)

Ping                  - Packet Internet Groper

SMTP              - Simple Mail Transfer Protocol

VPN                - Virtual Private Networking


Allen, J., Christie, A., Fithen, W., McHugh, A., Pickel, J. & Stoner, E. 1999, State of the

  Practice of Intrusion Detection Technologies [Online], Available:

 , [Accessed 23 Oct 2001]

Axelsson, S. 1998, Research in Intrusion-Detection Systems: A Survey [Online],

  Available:, [Accessed 22 Oct 2001]

Axelsson, S. 2000, Intrusion Detection Systems A Survey and Taxonomy [Online],

  Available:, [Accessed 21 Oct 2001]

Bellovin, S., Leech, M. & Taylor, T. 2001, ICMP Traceback Messages [Online],

  Available:, [Accessed 3 Nov  


Carver, A., Hill, J., Surdu, J. & Pooch, U. 2000, 'A Methodology for Using Intelligent

  Agents to provide Automated Intrusion Response', Proceedings of the 2000 IEEE

  Workshop on Information and Security, United States Military Academy, West Point,

  NY, pp. 110-6

Cerf, V., Burleigh, S., Hooke, A., Turgerson, L., Durst, R., Scott, K. , Traveis, E. &

  Weiss, H. 2001, Interplanetary Internet (IPN):  Architectural Definition [Online],

  Available:, [Accessed 3

  Nov 2001]

CERT 2001, CERT/CC Statistics 1988-2001 [Online], Available:, [Accessed 4 Nov 2001]

CIDF Working Group 2000, The Common Intrusion Detection Framework [Online],

  Available:, [Accessed 3 Nov


Coit, C., Staniford, S. & McAlerney, J. 2001, Towards Faster String Matching for

  Intrusion Detection or Exceeding the Speed of Snort [Online], Available:, [Accessed 20 Oct 2001]

Cowan, C., Pu, C., Maier, D., Hinton, H., Walpole, J., Bakke, P., Beattie, S.,  Grier, A.,

  Wagle, P. & Zhang, Q. 1998, StackGuard: Automatic Adaptive Detection and

  Prevention of Buffer-Overflow Attacks [Online], Available:,

  [Accessed 21 Oct 2001]

Dalton, C. 2001, Strategies & Issues: Managing Remote Desktop Firewalls [Online],

  Available:, [Accessed

  5 Nov 2001]

Evans, M. & Furnell, S. 2000, 'Internet-based security incidents and the potential for

  false alarms', Internet Research: Electronic Networking Applications and Policy, vol.10

  no.3 pp.238-45

Godfrey, M. & Tu, Q. 2000, Growth, Evolution and Structural Change in Open Source

  Software[Online], Available:, [Accessed 13 Oct


Gomes, L. 2001, Is Microsoft secretly using open source?   [Online], Available:,4586,2776342,00.html?chkpt=zdnn_tp,

  [Accessed 21 Oct 2001]

Gula, R. 1999, Passive Vulnerability Detection [Online], Available:, [Accessed 13 Oct 2001]

----------1999a,  Broadening the Scope of Penetration Testing "The top 14 things your

  ethical hackers for hire did not test.", Available:, [Accessed 13 Oct 2001]

Gula, R. 2000, Bypassing Intrusion Detection Systems [Online], Available:, [Accessed 13 Oct 2001]

Houle, K. & Weaver, G. 2001, Trends in Denial of Service Attack Technology [Online],

  Available:, [Accessed 4 Nov 2001]

Kumar, S.  1995,  Classification and Detection of Computer Intrusions [Online],

  Available: [Accessed 13 Oct


Kvarnström, H. 1999, A survey of commercial tools for intrusion detection [Online],

  Available:, [Accessed 13 Oct


Lee, W. 2000, Towards Cost-Sensitive Modeling of Intrusion Detection and Response

  [Online], Available:, [Accessed 23 Oct


Lippman, R., Fried, D., Graf, I., Haines, J., Kendall, K., McClung, D., Weber, D.,

  Webster, S., Wyschogrod, D., Cunningham, R. & Zissman, M. 2000, 'Evaluating

  Intrusion Detection Systems: The 1998 DARPA Off-line Intrusion Detection

  Evaluation', Proceedings DARPA Information Survivability Conference and Exhibition

  (DISCEX) 2000,  IEEE Computer Society Press, Los Alamitos, CA, pp. 12-26

Lippmann, R. P., Haines, J. W., Fried, D. J., Korba, J., and Das, K., 2000, Analysis and

  Results of the 1999 DARPA Off-Line Intrusion Detection Evaluation - Publication

  Abstract [Online], Available:,

  [Accessed 3 Oct 2001]

Lunt, T. 1993, Detecting Intruders in Computer Systems [Online], Available:, [Accessed 13 Oct 2001]

Mankin, A., Massey, D., Wu, C., Wu, S. & Zhang, L. 2001, On Design and Evaluation of

  "Intention-Driven" ICMP Traceback [Online], Available:, [Accessed 3 Nov 2001]

Maughan, D. 2000, Secure Networking [Online], Available:, [Accessed 4 Nov 2001]

McClure, S., Scambray, S. & Kurtz, G. 1999, Hacking Exposed - Network Security

  Secrets and Solutions, Osborne McGraw-Hill, Berkeley CA  (links available from

Microsoft 2001, Use Your Internet Connection Firewall to Secure Your Small Network  

  [Online], Available:,

  [Accessed 5 Nov 2001]

MIT Lincoln Laboratory 2001, DARPA Intrusion Detection Evaluation [Online],

  Available:, [Accessed 3 Nov 2001]

Mitre 2001, Common Vulnerabilities and Exposures [Online], Available:, [Accessed 23 Oct 2001]

____2001a,  2000 Project Summary - Interplanetary Internet Study [Online], Available:, [Accessed 3 Nov 2001]

NFR, NFR Network Intrusion Detection [Online], Available:, [Accessed 21 Oct 2001]

Pandey, S. 2001, Intrusion Detection Systems for the Uninitiated, Part 2: Installing and

  Configuring Snort [Online], Available:, [Accessed 20 Oct


Parmar, S. 1998, An Introduction To Security [Online], Available:, [Accessed 13 Oct 2001]

Pichnarczyk, K., Weeber, S. & Feingold, R. 1994, Unix Incident Guide:  How to Detect

  an Intrusion [Online], Available:, [Accessed 4 Nov 2001]

Ptacek, T. and Newsham, T. 1998, 'Insertion, evasion, and denial of service: Eluding

  network intrusion detection' [Online], Available:

Roesch, M. 1999, Snort - Lightweight Intrusion Detection for Networks [Online],

  Available:, [Accessed 21 Oct 2001]

Stocksdale, G. 1999,  ID-Secure Intrusion Detection and Security Tools Database   

  [Online], Available:, [Accessed 5

  Nov 2001]

Timm, K. 2001, Strategies to Reduce False Positives and False Negatives - Part One

  [Online], Available:,

  [Accessed 20 Oct 2001]


-----------2001a, IDS False Positive and False Negative Reduction Strategies and

  Techniques, Part Two [Online], Available:  bin/, [Accessed 20 Oct 2001]

Wang, X., Reeves, D., Wu, S. & Yuill, J.  2000, Sleepy Watermark Tracing: An Active

  Network-Based Intrusion Response Framework [Online], Available:, [Accessed 3 Nov 2001]

WatchGuard 2001, WatchGuard Network Security Handbook - Firebox System 4.6

  [Online], Available:           , [Accessed 3 Nov 2001]



False negatives  are defined as an intrusion detection device not issuing alerts on legitimate attacks.   According to Timm, they can happen as a result of:

(Timm 2001)




Eight sites participated in the second DARPA off-line intrusion detection evaluation in 1999. Three weeks of training and two weeks of test data were generated on a test bed that emulates a small government site. More than 200 instances of 58 attack types were launched against victim UNIX and Windows NT hosts. False alarm rates were low (less than 10 per day). Best detection was provided by network-based systems for old probe and old denial-of-service (DoS) attacks and by host-based systems for Solaris user-to-root (U2R) attacks. Best overall performance would have been provided by a combined system that used both host- and network-based intrusion detection. Detection accuracy was poor for previously unseen new, stealthy, and Windows NT attacks. Ten of the 58 attack types were completely missed by all systems. Systems missed attacks because protocols and TCP services were not analyzed at all or to the depth required, because signatures for old attacks did not generalize to new attacks, and because auditing was not available on all hosts.

                                                (Lippman, Haines et al. 2000)