DGA Botnet Detection Using

Supervised Learning Methods

Hieu Mac, Duc Tran, Van Tong, Giang Nguyen, Hai Anh Tran
Bach Khoa Cybersecurity Centre
Ha Noi University of Science and Technology

1
Content

• Botnet

• Problem

• DGA botnet detection methods

 Handcrafted Features Based Methods
 Implicit Features Based Methods

• Experimental results

2
Botnet

• Botnet
 A network of computers compromised.
 3 type of structure: centralization, p2p and hybrid.
 Centralization
• C&C Server (Comand and Control Server)
• Domain name of C&C server is fixed.
• Example: Zeus (2010, 3.6m), BetaBot (2017, 192th)….
• DGA botnet
 Domain name of C&C server is generated by algorithm.
 Example: Conficker (2008, 10.5m), Kraken (2009, 0.5m), Tinba ….
• Support cyber-criminals: DDoS, stealing private data,
sending spam mail…
3
Problem

• Traditional methods
 Reversing Engineering
• Require experts
• Time-consuming.
 Intrusion Detection System
• Require updating blacklist frequently.
 Static features based methods
• Statistical features is not suitable for all kind of DGA botnets.
• Low performance.
• No single method can be the best performer for all problems.
• Our contributions:
 Investigate some handcrafted features and LSTM based methods.
 Propose some improved methods using LSTM.

4
DGA Botnet Detection Methods

Taxonomy for the supervised learning methods in DGA botnet

detection.

5
Handcrafted Features Based Methods

• Linguistic features [1][2]

 Domain length
 Entropy
𝑐𝑜𝑢𝑛𝑡 𝑡1 𝑐𝑜𝑢𝑛𝑡(𝑡1 )
𝐸 𝑑 =− ∗ log( )
|𝑝1 | |𝑝1 |
𝑡1 𝜖𝑝1

 Dictionary matching score

𝑛
𝑖=1 |𝑤𝑖 |
𝑅 𝑑 =
|𝑑|
 N-gram normality scores (n=1,2,3,4,5)
𝑡𝑛 ∈𝑝𝑛 𝑐𝑜𝑢𝑛𝑡 𝑡𝑛 ∗ 𝑓(𝑡𝑛 )
𝑆 𝑑 =
𝑑 −𝑛+1
 Merging of 3-, 4-, and 5-grams
6
Handcrafted Features Based Methods

The distinction between the linguistic attributes.

25 4

3
15
2

5 1
Alexa Ramnit Ranbyus Suppobox Banjori Alexa Ramnit Ranbyus Suppobox Banjori
Domain Length Entropy

1
100

0.5
50

0 0
Alexa Ramnit Ranbyus Suppobox Banjori Alexa Ramnit Ranbyus Suppobox Banjori
2-gram score 7
Dictionary matching score
Handcrafted Features Based Methods

• Algorithms
 C4.5 [3]
• Greedy top-down procedure.
• Attribute - Root node.
• Run until meet stopping criterion.
 SVM [4]
• Project input samples into a high dimensional space.
• Find the optimal hyperplane to separate the samples.
 ELM [5]
• Effective solution for the single hidden layer feedforward networks.
• Do not require the hidden layer to be tuned.

8
Implicit Features Based Methods

• The LSTM Algorithm

9
Implicit Features Based Methods

• The LSTM Algorithm

 Embedding layer
• Input strings  Rd*l.
• d = 128 (experiment).
• l (max length) .
 LSTM layer
• Implicit feature extraction.
• Creating attribute vector.
• 128 blocks and 1 cell per block (experiment).
 Logistic Regression
• Squash.
• Softmax.
10
Implicit Features Based Methods

• Long Short-Term Memory network [6]

𝑓𝑡 = 𝜎𝑔 (𝑊𝑓𝑥 𝑥𝑡 + 𝑊𝑓ℎ ℎ𝑡−1 + 𝑏𝑓 )

𝑖𝑡 = 𝜎𝑔 (𝑊𝑖𝑥 𝑥𝑡 + 𝑊𝑖ℎ ℎ𝑡−1 + 𝑏𝑖 )

𝑜𝑡 = 𝜎𝑔 (𝑊𝑜𝑥 𝑥𝑡 + 𝑊𝑜ℎ ℎ𝑡−1 + 𝑏𝑜 )

𝑐𝑡 = 𝑓𝑡 ⨀𝑐𝑡−1 + 𝑖𝑡 ⨀𝜎ℎ (𝑊𝑐𝑥 𝑥𝑡 + 𝑊𝑐ℎ ℎ𝑡−1 + 𝑏𝑐 )

ℎ𝑡 = 𝑜𝑡 ⨀𝜎ℎ 𝑐𝑡

𝑦𝑡 = ∅(𝑊𝑦ℎ ℎ𝑡 + 𝑏𝑦 )
11
Implicit Features Based Methods

• Recurrent SVM (Recurrent Support Vector Machine) [7]

 Softmax  SVM.
 Softmax minimizes the cross-entropy.
 SVM finds the maximum margin between samples from different classes.

• CNN (Convolutional Neural Network)-LSTM

 CNN with max-pooling finds the morphological patterns [8].
 128 blocks.

• Bidirectional LSTM [9]

 Forward and backward LSTM.
 Learn a representation from both the past and future information.
 128 blocks.
12
Dataset Specification

• Real-world dataset
Domain Type #Sample Domain Type #Sample
Geodo 58 Fobber 60
 Alexa top 1 million domains
Beebone 42 Alexa 88347
• 88,357 legitimate domains.
Murofet 816 Dyre 800  OSINT DGA feed
Pykspa 1422 cryptowall 94 • 81,490 DGA domains
Padcrypt 58 Corebot 28
Ramnit 9158 P 200
Volatile 50 Bedep 172
Ranbyus 1232 Matsnu 48
Qakbot 4000 PT Goz 6600
Simda 1365 Necurs 2398
Ramdo 200 Pushdo 168
Suppobox 101 Cryptolocker 600
Locky 186 Dircrypt 57
Tempedreve 25 Shifu 234
Qadars 40 Bamital 60
Symmi 64 Kraken 508
Banjori 42166 Nymaim 600
Tinba 6385 Shiotob 1253 13
Hesperbot 192 W32.Virut 60
Evaluation Measures

• Measures
𝑇𝑃
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
𝑇𝑃 + 𝐹𝑃
𝑇𝑃
𝑅𝑒𝑐𝑎𝑙𝑙 =
𝑇𝑃 + 𝐹𝑁
2
𝐹1 − 𝑠𝑐𝑜𝑟𝑒 =
1/𝑅𝑒𝑐𝑎𝑙𝑙 + 1/𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛
TP (True Positives), FP (False Positives), TN (True Negatives), FN (False Negatives)
 Micro-averaging
 Macro-averaging
• ROC curve
 Graphical plot illustrates the diagnostic ability of a binary classifier
system.
14
 Experiment results
HMM C4.5 ELM SVM
Precision Recall F1-score Precision Recall F1-score Precision Recall F1-score Precision Recall F1-score
Geodo 0.0127 0.4167 0.0246 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Beebone 0.0308 0.7500 0.0591 0.6250 1.0000 0.7692 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Murofet 0.8235 0.2577 0.3925 0.3810 0.4706 0.4211 0.9301 0.3803 0.5297 0.9785 0.5583 0.7109
Pykspa 0.3090 0.1937 0.2381 0.0000 0.0000 0.0000 0.7972 0.3615 0.4912 0.9625 0.2711 0.4231
Padcrypt 0.2069 1.0000 0.3429 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.1667 0.2857
Ramnit 0.1081 0.0551 0.0730 0.0000 0.0000 0.0000 0.5019 0.7098 0.5878 0.4604 0.7811 0.5794
Volatile 0.0136 0.6000 0.0267 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Ranbyus 0.0424 0.2236 0.0713 0.0000 0.0000 0.0000 0.4492 0.8699 0.5922 0.0000 0.0000 0.0000
Qakbot 0.1240 0.0587 0.0797 0.9773 0.9835 0.9804 0.6459 0.3049 0.4141 0.7262 0.315 0.4394
Simda 0.0137 0.1465 0.0250 0.7685 0.9640 0.8552 0.7094 0.0952 0.1652 0.4138 0.044 0.0795
Ramdo 0.0388 0.7250 0.0737 0.0000 0.0000 0.0000 0.2500 0.1416 0.1655 0.0000 0.0000 0.0000
Suppobox 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Locky 0.0000 0.0000 0.0000 0.3492 0.2767 0.3088 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Tempedreve 0.0015 0.8000 0.0031 0.9507 0.9766 0.9635 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Qadars 0.0309 0.7500 0.0594 1.0000 1.0000 1.0000 0.3333 0.0416 0.0740 0.0000 0.0000 0.0000
Symmi 0.0065 0.1538 0.0125 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Banjori 0.9143 0.1051 0.1885 0.6667 0.2857 0.4000 0.9977 0.9999 0.9988 0.9959 0.9995 0.9977
Tinba 0.0000 0.0000 0.0000 0.6000 0.4167 0.4918 0.7805 0.9723 0.8659 0.7545 0.9483 0.8404
Hesperbot 0.0037 0.0526 0.0069 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Fobber 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Alexa 1.0000 0.0002 0.0003 0.9899 0.9868 0.9883 0.9478 0.9867 0.9669 0.9418 0.9900 0.9653
Dyre 0.9697 1.0000 0.9846 0.1646 0.0567 0.0844 0.9717 1.0000 0.9856 1.0000 1.0000 1.0000
Cryptowall 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Corebot 0.0017 0.4000 0.0035 0.3116 0.2191 0.2573 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
P 0.2727 0.2250 0.2466 0.0645 0.0140 0.0230 0.4666 0.2083 0.2762 1.0000 0.2000 0.3333
Bedep 0.0060 0.1471 0.0115 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Matsnu 0.0000 0.0000 0.0000 0.0800 0.0435 0.0563 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
PT Goz 0.9811 0.6682 0.7950 0.9091 1.0000 0.9524 0.9774 0.9868 0.9821 0.9932 0.9970 0.9951
Necurs 0.0244 0.0729 0.0366 0.0000 0.0000 0.0000 0.2139 0.0347 0.0588 0.0000 0.0000 0.0000
Pushdo 0.0036 0.2353 0.0071 0.1071 0.0268 0.0429 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Cryptolocker 0.0163 0.6917 0.0318 0.6406 0.5538 0.5940 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Dircrypt 0.0017 0.0909 0.0034 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Shifu 0.0250 1.0000 0.0489 0.2222 0.2000 0.2105 0.1032 0.0567 0.0700 0.0000 0.0000 0.0000
Bamital 0.6316 1.0000 0.7742 0.4839 0.5797 0.5275 0.8555 0.8055 0.8226 1.0000 1.0000 1.0000
Kraken 0.0041 0.0196 0.0068 0.4545 0.4545 0.4545 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Nymaim 0.0085 0.2250 0.0165 0.3062 0.3900 0.3431 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Shiotob 0.2404 0.2749 0.2565 0.4767 0.3761 0.4205 0.8048 0.6480 0.7175 0.9074 0.5857 0.7119
W32.Virut 0.0035 1.0000 0.0070 0.4403 0.2439 0.3139 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Micro-averaging 0.8085 0.0782 0.1426 0.8652 0.8854 0.8751 0.8853 0.9045 0.8947 0.8754 15 0.8885
0.9020
Macro-averaging 0.1808 0.3510 0.2386 0.3150 0.3031 0.3089 0.3088 0.2527 0.2779 0.3193 0.2331 0.2695
 Experiment results
LSTM Recurrent SVM CNN+LSTM Bidirectional LSTM
Precision Recall F1-score Precision Recall F1-score Precision Recall F1-score Precision Recall F1-score
Geodo 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Beebone 0.4000 0.2250 0.2872 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
Murofet 0.7197 0.5509 0.6185 0.7059 0.5926 0.6443 0.9014 0.3422 0.4961 0.7459 0.5170 0.6107
Pykspa 0.8294 0.6782 0.7457 0.8744 0.6963 0.7753 0.7035 0.5977 0.6463 0.8222 0.6801 0.7445
Padcrypt 0.9242 0.5833 0.7077 1.0000 0.8000 0.8889 0.9091 0.7692 0.8333 0.7500 0.4000 0.5217
Ramnit 0.5786 0.8226 0.6793 0.5632 0.8143 0.6659 0.5657 0.7992 0.6625 0.5953 0.7869 0.6778
Volatile 0.9600 0.4000 0.5543 0.9167 0.7857 0.8462 1.0000 0.7333 0.8462 1.0000 0.8182 0.9000
Ranbyus 0.4239 0.5040 0.4593 0.3897 0.4905 0.4343 0.4313 0.6181 0.5081 0.4077 0.6883 0.5120
Qakbot 0.7005 0.5565 0.6196 0.7237 0.5160 0.6024 0.7178 0.4749 0.5716 0.7446 0.5049 0.6017
Simda 0.9067 0.8125 0.8525 0.9574 0.8982 0.9268 0.7636 0.7976 0.7802 0.7591 0.9121 0.8286
Ramdo 0.9658 0.9750 0.9702 0.9722 1.0000 0.9859 0.8864 1.0000 0.9398 1.0000 0.8780 0.9351
Suppobox 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2500 0.0556 0.0909
Locky 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Tempedreve 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Qadars 0.0000 0.0000 0.0000 1.0000 0.2727 0.4286 0.7143 0.6250 0.6667 0.0000 0.0000 0.0000
Symmi 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.5000 0.0714 0.1250 0.3333 0.0667 0.1111
Banjori 0.9992 1.0000 0.9996 0.9999 1.0000 0.9999 0.9996 1.0000 0.9998 0.9996 1.0000 0.9998
Tinba 0.8884 0.9815 0.9327 0.8823 0.9707 0.9244 0.8946 0.9377 0.9157 0.8843 0.9778 0.9287
Hesperbot 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Fobber 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Alexa 0.9727 0.9929 0.9827 0.9787 0.9924 0.9855 0.9681 0.9890 0.9785 0.9753 0.9893 0.9822
Dyre 0.9755 0.9925 0.9839 0.9742 0.9934 0.9837 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
Cryptowall 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0625 0.1176
Corebot 0.0000 0.0000 0.0000 1.0000 0.2500 0.4000 1.0000 0.1667 0.2857 0.8000 0.6667 0.7273
P 0.7521 0.3050 0.3858 0.5312 0.4595 0.4928 0.7778 0.3256 0.4590 0.7143 0.5128 0.5970
Bedep 0.8608 0.2588 0.3965 0.8182 0.2571 0.3913 0.5556 0.1667 0.2564 0.7647 0.3421 0.4727
Matsnu 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
PT Gz 0.9958 0.9994 0.9976 0.9992 0.9992 0.9992 0.9976 0.9992 0.9984 0.9985 0.9985 0.9985
Necurs 0.4673 0.0583 0.1036 0.3651 0.0922 0.1472 0.4778 0.0911 0.1530 0.4928 0.1475 0.2270
Pushdo 0.8806 0.1706 0.2744 0.5238 0.3438 0.4151 0.3158 0.3636 0.3380 0.6154 0.2051 0.3077
Cryptolocker 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2143 0.0270 0.0480 0.1538 0.0172 0.0310
Dircrypt 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Shifu 0.4064 0.3064 0.3416 0.2603 0.3800 0.3089 0.2405 0.3800 0.2946 0.3929 0.4490 0.4190
Bamital 0.7833 0.5500 0.6366 0.8000 0.6667 0.7273 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
Kraken 0.1666 0.0039 0.0076 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3077 0.0385 0.0684
Nymaim 0.2875 0.0040 0.0692 0.2432 0.0667 0.1047 0.1667 0.0084 0.0160 0.1500 0.0462 0.0706
Shiotob 0.9114 0.8845 0.8976 0.9587 0.8788 0.9170 0.9234 0.8821 0.9023 0.9472 0.8826 0.9137
W32.Virut 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000

Micro-averaging 0.9193 0.9315 0.9253 0.9206 0.9314 0.9260 0.9159 0.9253 0.9206 0.9232 0.9309 0.9270
Macro-averaging 0.4672 0.3583 0.4055 0.5115 0.4268 0.4653 0.5164 0.4254 0.4665 0.5422 16 0.4845
0.4379
Experiment results

The ROC curves related to the supervised learning

1
True Positive Rate

0.95

HMM (AUC = 0.8965)

C4.5 (AUC = 0.9935)
ELM (AUC = 0.9606)
SVM (AUC = 0.9904)
LSTM (AUC = 0.9955)
0.9 Recurrent SVM (AUC = 0.9969)
CNN + LSTM (AUC = 0.9959)
Bidirectional LSTM (AUC = 0.9964)
0 0.2 0.4 0.6 0.8 1
False Positive Rate
17
Experiment results
A. Ranks computed by the Wilcoxon test
(1) (2) (3) (4) (5) (6) (7) (8)
HMM (1) - 210.5 227 311 128 100 63 46.5
C4.5 (2) 492.5 - 406.5 423 277.5 219.5 237.5 223
ELM (3) 476 334.5 - 413.5 126 96 63.5 53
SVM (4) 392 318 289.5 - 125.5 96 64 47.5
LSTM (5) 575 425.5 577 615.5 - 199 295.5 179
Recurrent SVM (6) 603 483.5 607 607 542 - 471 281
CNN+LSTM (7) 640 503.5 639.5 639 445.5 232 - 192
Bidirectional LSTM (8) 656.5 518 688 655.5 524 422 549 -

B. Summary of the Wilcoxon test

(1) (2) (3) (4) (5) (6) (7) (8)
HMM (1) –      
C4.5 (2)  –   
ELM (3) –    
SVM (4) –    
LSTM (5)    –  
Recurrent SVM (6)      – 
CNN+LSTM (7)    – 
Bidirectional LSTM (8)       –
18
Conclusion

• We have thoroughly investigated various supervised

learning methods.
 Hidden Markov Model
 C4.5 decision tree
 Support Vector Machines
 Extreme Learning Machine
 Long Short-Term Memory network
 Recurrent SVM
 CNN+LSTM
 Bidirectional LSTM
• Bidirectional LSTM and Recurrent SVM achieve the
highest detection rate.

19
References
1. S. Schiavoni, F. Maggi, L. Cavallaro, and S. Zanero, Phoenix: DGA-based botnet tracking and
intelligence, International Conference on Detection of Intrusions and Malware, and Vulnerability
Assessment (DIMVA) (2014), LNCS 8550, 192-211.
2. Y. Shi, C. Gong and L. Juntao, Malicious Domain Name Detection Based on Extreme Machine
Learning, Neural Processing Letters (2017): 1-11.
3. L. Bilge, E. Kirda, C. Kruegel, and M. Balduzzi, EXPOSURE: Finding Malicious Domains Using
Passive DNS Analysis, Ndss. 2011.
4. J. Milgram, M. Cheriet, and R. Sabourin, “One against one” or “one against all”: Which one is
better for handwriting recognition with SVMs?, Tenth international workshop on frontiers in
handwriting recognition. La Baule, 2006.
5. G.B Huang, Q.-Y. Zhu, and C.-K. Siew, Extreme learning machine: theory and
applications, Neurocomputing 70.1 (2006): 489-501.
6. S. Hochreiter, and J. Schmidhuber, Long short-term memory, Neural computation 9(8) (1997):
1735-1780
7. Y. Tang, Deep learning using linear support vector machines, arXiv preprint arXiv:1306.0239
(2013).
8. Kim, Yoon, et al. Character-Aware Neural Language Models. AAAI. 2016.
9. A. Graves, and J. Schmidhuber, Framewise phoneme classification with bidirectional LSTM and
other neural network architectures, Neural Networks 18.5 (2005): 602-610.

20
Thank you for your listening !

21