MalPull has received an overhaul, as it is now uses multiple threads to download the given hashes concurrently. It is now also able to download samples from VirusTotal, if you have a working API key. This release contains breaking changes compared to the last version, as the command-line arguments have been changed. At first, the usage of the new version is described, after which the patch notes are given. The program’s source code and precompiled Java Archive and can be found on GitHub. The latest release is also available on GitHub.
To use MalPull, one has to provide four command line arguments. The first argument is the amount of threads that can be used by MalPull when downloading samples. The minimum is one, and the maximum is left up to the user. Using more threads than you have (virtual) cores is unlikely to give an advantage due to the way threads are scheduled.
The second argument is the location to a file that contains API keys for all services, of which an example is given below.
virustotal=abcd1234 malshare=abcd1234 koodous=abcd1234 malwarebazaar=enabled
The order of the services in this file does not matter. If you wish to not use any of the services, simply remove them from this file. Note that Malware Bazaar does not have an API key, meaning that any value can be used. Malware Bazaar is represented this way to offer the user the option to include or exclude the service.
The third argument is the file location of a file that contains the hashes. Each hash needs to be separated by a new line. The hashes are deduplicated by MalPull, meaning duplicate entries are only downloaded once in total.
The fourth argument is the folder to store all downloaded samples in. The file name of each sample is equal to the file’s hash, as given in the list of hashes. If a file with the same name in the given location already exists, it is overwritten without warning. If the output folder does not exist, it is created, including any of the missing parent directories.
An example on how to run MallPull is given below.
java -jar /path/to/MallPull.jar 6 ~/Downloads/malpull_test/keys.txt ~/Downloads/malpull_test/hashes.txt ~/Downloads/malpull_test/output/
Hashes that cannot be found on any of the services, are printed once all hashes have been iterated through.
- Added VirusTotal support.
- Added multi-threading support to download multiple samples at the same time. The maximum thread count is configurable as a command-line setting.
- The input now requires a file that contains all hashes that are to be downloaded, separated by a newline. The command-line requires an argument that specifies the location of the input file.
- The API keys are stored in a separate file, allowing for a more efficient use of the command-line arguments.
- If a hash cannot be found on any of the enabled services, it is added to a list of missing hashes. This list is printed once all samples have been downloaded.
- The total time spent for the downloading of all samples is given once all samples have been downloaded.
- Duplicate entries in the download list are filtered prior to downloading, thus avoiding double API queries that would impact the query limit of any of the used services.
- The output folder is given via the command-line interface, where each file is written to. File names of samples are based upon the hash in the list of samples that are to be downloaded. Existing files will be overwritten without warning.
Suggestions and feedback
Feel free to give me any suggestions and/or feedback via any of the contact methods that are given below.