Masscan php mysql

Many of you heard of such tool like masscan which doing network scanning really fast and can even scan the whole internet for a short period of time.

This tool is functional enough and integrated well with nmap, but from my point of view it lacks some reporting functions. There in internet you can find solution from Offensive Security with their GUI for masscan, but that didn’t fit well for my needs and so I decided to build some small custom solution based on masscan XML output parsing with help of PHP and storing results into MySQL database (MariaDB).

masscan result

First we need to create database with 8 rows and you may call them as you wish, just don’t forget to update fields in the import script:

  1. ID
  2. IP
  3. Port
  4. Protocol
  5. State
  6. Service
  7. Banner
  8. Timestamp

Our solution will consist of 2 files and will work on Debian instance with php7 and mariadb.

First file will be responsible for the data import process into database:

$filepath = $argv[1]; //argument we passing to our script (path to xml file which we want to import)
 $content =  utf8_encode(file_get_contents($filepath));
 $xml = simplexml_load_string($content, 'SimpleXMLElement', LIBXML_COMPACT | LIBXML_PARSEHUGE); //Parsing XML data
 foreach($xml->host as $host) {
     foreach($host->ports as $i) {
         $ip = $host->address['addr']; //Setting IP variable
         $port = $i->port['portid']; //Setting port variable 
         $proto = $i->port['protocol']; //Setting protocol variable
         $state = $i->port->state['state']; //Setting port state
         $service = $i->port->service['name']; //Setting service type
         $banner = preg_replace("/[A-Fa-f0-9]{6}|x[0-9][a-z]|x0[0-9]|c[0-9][a-z]|xff/", " ", $i->port->service['banner']); //Clearning banner from hex
         $import_data = mysqli_query($link, "INSERT INTO masscan (ip, port, proto, state, service, banner) VALUES ('$ip', '$port', '$proto', '$state', '$service', '$banner')"); //Storing results into database
     }
 }

To import your data you will need to do following steps:

  1. Launch execute masscan with possibility to output results into XML file (see example command below)
  2. Run php script with path to xml file in argument

Running masscan with output into XML:

masscan -p0-500 192.168.0.0/16 --banners --max-rate 100000 --source-port 61000 --output-format xml --output-filename /tmp/scan.xml

Now let’s import our results into database:

php7 import.php /tmp/scan.xml

What to do if we want to scan several IP addresses and import all results ?

Here comes another script which helps to:

  1. Truncate our table from previously stored data
  2. Scan directory for all available files with scanning results
  3. Remove old files with results after import is done
$clearTable = mysqli_query($link, "TRUNCATE TABLE masscan");
 $dir = "/tmp/";
 $list = array_diff(scandir($dir, 1), array('.', '..'));
 foreach($list as $file) {
     $import = shell_exec("/usr/bin/php7.0 /home/import.php /tmp/$file");
 }
 sleep(5);
 $remove_old_data = shell_exec("rm /tmp/*");

$clearTable variable executes command for our masscan table truncate operation so every scan will be imported as new results and we’ll not have any duplicates or outdated results.

As we configured our masscan to store results into /tmp/ directory than we going to monitor it for new files. In this case better to use IP address as name of the file. Also we may not use XML extension and file still will be parsed because of it is XML structure.

Using short timeout of 5 seconds just in case before removing old xml files from previous scans.

Official masscan repository can be found on GitHub.

RELATED ARTICLESMORE FROM AUTHOR