An anomaly detection framework for DNS-over-HTTPS (DoH) tunnel using time-series analysis
Loading...
Files
Date
2020
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of New Brunswick
Abstract
Domain Name System (DNS) as a network protocol is vulnerable to several security loopholes. To cover up some of the vulnerabilities in DNS, a new protocol, named DNS over HTTPS (DoH), is created to improve privacy, and protect from various persistent attacks. The DoH protocol encrypts the DNS requests for the DoH client and sends it through a tunnel to prevent eavesdropping and man-in-the-middle attacks. This research work comprehensively studies these security vulnerabilities, proposes a taxonomy of potential DNS attacks, analyzes the security aspects of DoH protocol, and classifies DNS attacks that are applicable on DoH. To achieve these objectives, we simulated DoH tunnels. The simulated environment covers different DoH deployment scenarios includes DoH within an application, DoH proxy on the name server in the local network, and DoH proxy on a local system as suggested in RFC8484. In this research, we captured malicious and benign DoH traffic and analyzed it as a two-layered approach to classify benign and malicious traffic at first layer and characterize DoH traffic at second layer. It is observed that for statistical features, Random Forest (RF) and Decision Tree (DT) give the best classification and characterization results among prominent machine learning and deep learning classifiers at first and second layer, respectively. Moreover, for time-series features, long short-term memory (LSTM) turns out to be the best classifier for DoH traffic classification and characterization at first and second layers, respectively. The experimental results indicate that while DoH can be abused to create covert communication channels, the proposed solution is sufficient to detect these channels in a timely manner.