Improved clustering algorithm based on high-speed network data stream (original) (raw)

Abstract

With the continuous development of network technology, the attack has become the biggest threat to the stable operation of the network. Intrusion detection technology is a proactive safety protection measure which provides real-time monitoring of internal attacks, external attacks, and misuse. Traditional intrusion detection system is short of adaptability due to the complication and scale of the network. The main problem is that the real-time performance of the network is poor and the reliability is not high. This paper designs the intrusion detection mechanism combined with data stream clustering algorithm and intrusion detection system to solve the problem in processing a large amount of high-speed data streams. The performance of processing data streams is improved through the clustering algorithm based on density and the sliding window and the experiments show that the intrusion detection efficiency is higher than DenStream algorithm.

Access this article

Log in via an institution

Subscribe and save

• Starting from 10 chapters or articles per month
• Access and download chapters and articles from more than 300k books and 2,500 journals
• Cancel anytime View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

References

• Barbara D, Daniel J Couto et al (2001) ADAM: a testbed for exploring the use of data mining in intrusion detection. Acm Sigmod Rec 30(4):15–24
  Article Google Scholar
• Bu S, Yu R, Liu P et al (2011) Distributed combined authentication and intrusion detection with data fusion in high-security mobile Ad Hoc networks. IEEE Trans Veh Technol 60(3):1025–1036
  Article Google Scholar
• Cai Y, Xie K, Ma X (2004) An improved DBSCAN algorithm which is insensitive to input parameters. Acta Sci Nat Univ Pekin 40(3):480–486
  Google Scholar
• Chen Z, Zhang H, Hatcher W, Nguyen J Yu W (2016) A streaming-based network monitoring and threat detection system. In: IEEE 14th international conference on software engineering research, management and applications (SERA), pp 31–37
• Cho Y, Moon S (2015) Recommender system using periodicity analysis via mining sequential patterns with time-series and FRAT analysis. J Converg (JoC) 6(1):9–17
  Google Scholar
• Dash T (2017) A study on intrusion detection using neural networks trained with evolutionary algorithms. Soft Comput 21(10):2687–2700
  Article Google Scholar
• Gao H, Zhu D, Wang X (2011) A parallel clustering ensemble algorithm for intrusion detection system. Ninth Int Symp Distrib Comput Appl Bus Eng Sci 2011:450–453
  Google Scholar
• Gaur M, Pant B (2015) Trusted and secure clustering in mobile pervasive environment. Hum–cent Comput Inform Sci (HCIS) 5:32
  Google Scholar
• Gu B, Sheng VS, Tay K, Romano W, Li S (2014) Incremental support vector learning for ordinal regression. IEEE T Neur Net Lear 26(7):1403–1416
  Article MathSciNet Google Scholar
• Gu B, Sheng VS, Wang Z, Ho D, Osman S, Li S (2015) Incremental learning for \(\nu \)-support vector regression. Neural Netw 67:140–150
  Article Google Scholar
• Keegan N, Ji S, Chaudhary A, Concolato C, Yu B, Jeong D (2016) A survey of cloud-based network intrusion detection analysis. Hum–cent Comput Inform Sci (HCIS) 6:19
  Google Scholar
• Kranen P, Assent I, Baldauf C, Seidl T (2009) Self-adaptive anytime stream clustering. In: The 9th IEEE international conference on data mininig (ICDM), pp 249–258
• Kranen P, Assent I, Baldauf C, Seidl T (2011) The clu-stree: Indexing micro-clusters for anytime stream mining. Knowl Inform Syst 29(2):249–272
  Article Google Scholar
• Lee W, Stolfo S, Mok K (1998) Mining audit data to build intrusion detection models. In: International conference on knowledge discovery & data mining, pp 66–72
• Liang C, Zhang Y, Shi P et al (2012) Learning very fast decision tree from uncertain data streams with positive and unlabeled samples. Inform Sci 213(23):50–67
  Article MathSciNet Google Scholar
• Liu X, Xu H, Dong Y, Wang Y, Qian J (2005) Discovering frequent pattern in the data stream. Comput Res Dev 2005(12):2192–2198
  Article Google Scholar
• Rajput R, Mishra A, Kumar S (2014) Optimize intrusion prevention and minimization of threats for stream data classification. Fourth Int Conf Commun Syst Netw Technol 2014:408–413
  Google Scholar
• Wang D, Sun J (2010) Research on data mining technology. Stat Decis 2010(7):161–162
  Google Scholar
• Wang W, Guyet T, Quiniou R et al (2014) Autonomic intrusion detection: adaptively detecting anomalies over unlabeled audit data streams in computer networks. Knowl-Based Syst 70:103–117
  Article Google Scholar
• Xia L, Jing J (2009) SA-DBSCAN: a self-adaptive density-based clustering algorithm. J Grad Sc Chin Acad Sci 26(4):530–538
  MathSciNet Google Scholar
• Xu K, Kliger M, Hero A III (2014) Adaptive evolutionary clustering. Data Min Knowl Discov 28(2):304–336
  Article MathSciNet MATH Google Scholar
• Yang D, Rundensteiner E, Ward M (2012) Shared execution strategy for neighbor-based pattern mining requests over streaming windows. ACM Trans Database Syst 37(1):5
  Article Google Scholar
• Yin C, Ma L, Feng L (2016) A feature selection method for improved clonal algorithm towards intrusion detection. Int J Pattern Recog Artif Intell 30(5):1–14
  Article Google Scholar
• Yin C, Zhang S, Xi J (2016) An improved anonymity model for big data security based on clustering algorithm. Concurr Comput Pract Exp 2016:1–13
  Google Scholar
• Yin C, Feng L, Ma L, Kim J, Wang J (2016) An effective feature selection and data-stream classification model HDP. J Internet Technol 17(4):695–702
  Google Scholar
• Yu Y, Guo S, Huang H (2007) Abnormal intrusion detection based on data stream. Comput Sci 34(5):66–71
  Article Google Scholar
• Zhen J, Hu M, Yun X (2006) Large scale network anomaly discovery based on data flow method. J Commun 27(2):1–8
  Google Scholar
• Zhu C, Dun X, Zhu L (2011) A study on the application of data stream clustering mining through a sliding and damped window to intrusion detection. Fourth Int Conf Inform Comput 2011:22–26
  Google Scholar

Download references

Acknowledgements

This work was funded by the National Natural Science Foundation of China (61373134, 61402234). It was also supported by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD), Jiangsu Key Laboratory of Meteorological Observation and Information Processing (KDXS1105) and Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology (CICAEET). Prof. Ruxia Sun is the corresponding author. We declare that we do not have any conflicts of interest to this work.

Author information

Authors and Affiliations

1. School of Computer and Software, Jiangsu Engineering Center of Network Monitoring, Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing University of Information Science & Technology, Nanjing, 210044, China
   Chunyong Yin, Lian Xia, Sun Zhang & Ruxia Sun
2. College of Information Engineering, Yangzhou University, Yangzhou, China
   Jin Wang

Authors

1. Chunyong Yin
2. Lian Xia
3. Sun Zhang
4. Ruxia Sun
5. Jin Wang

Corresponding author

Correspondence toRuxia Sun.

Ethics declarations

Conflict of interest:

Chunyong Yin declares that he has no conflict of interest. Lian Xia declares that she has no conflict of interest. Sun Zhang declares that he has no conflict of interest. Ruxia Sun declares that she has no conflict of interest. Jin Wang declares that he has no conflict of interest.

Ethical approval:

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by J. Park.

Rights and permissions

About this article

Cite this article

Yin, C., Xia, L., Zhang, S. et al. Improved clustering algorithm based on high-speed network data stream.Soft Comput 22, 4185–4195 (2018). https://doi.org/10.1007/s00500-017-2708-2

Download citation

• Published: 11 July 2017
• Issue date: July 2018
• DOI: https://doi.org/10.1007/s00500-017-2708-2

Keywords