GitHub - MISP/misp-modules: Modules for expansion services, enrichment, import and export in MISP and other tools. (original) (raw)

MISP modules are autonomous modules that can be used to extend MISP for new services such as expansion, import, export and workflow action.

MISP modules can be also installed and used without MISP as a standalone tool accessible via a convenient web interface or using a cli tool.

The modules are written in Python 3 following a simple API interface. The objective is to ease the extensions of MISP functionalities without modifying core components. The API is available via a simple REST API which is independent from MISP installation or configuration and can be used with other tools. The API is also documented automatically via an OpenAPI end-point or a swagger file.

For more information: Extending MISP with Python modules slides from MISP training.

Installation

Installation instructions can be found in the installation documentation.

How to add your own MISP modules?

Developing a MISP module yourself is fairly easy. Start with a template or existing module and continue from there. More information can be found in the contribute section of the documentation.

Documentation

In order to provide documentation about some modules that require specific input / output / configuration, the documentation contains detailed information about the general purpose, requirements, features, input and ouput of each of these modules:

• * description - quick description of the general purpose of the module, as the one given by the moduleinfo
• requirements - special libraries needed to make the module work
• features - description of the way to use the module, with the required MISP features to make the module give the intended result
• references - link(s) giving additional information about the format concerned in the module
• input - description of the format of data used in input
• output - description of the format given as the result of the module execution

OpenAPI and API explorer

When the service is running you can discover the available endpoints in a machine-readable way via /openapi.json. An interactive Swagger UI that consumes the same specification is available at /openapi. The specification is generated during service startup, so restart misp-modules after adding or removing modules to refresh what those endpoints expose.

Licenses

For further Information see the license file.

Existing MISP modules

Expansion Modules

• Abuse IPDB - AbuseIPDB MISP expansion module
• ANYRUN Sandbox Submit - A module designed to submit URLs or files to the ANY.RUN Sandbox for analysis and return the unique analysis link and ID.
• OSINT DigitalSide - On demand query API for OSINT.digitalside.it project.
• APIVoid - Module to query APIVoid with some domain attributes.
• AssemblyLine Query - A module tu query the AssemblyLine API with a submission ID to get the submission report and parse it.
• AssemblyLine Submit - A module to submit samples and URLs to AssemblyLine for advanced analysis, and return the link of the submission.
• Backscatter.io - Backscatter.io module to bring mass-scanning observations into MISP.
• BTC Scam Check - An expansion hover module to query a special dns blacklist to check if a bitcoin address has been abused.
• BTC Steroids - An expansion hover module to get a blockchain balance from a BTC address in MISP.
• Censys Enrich - An expansion module to enrich attributes in MISP by quering the censys.io API
• CIRCL Passive DNS - Module to access CIRCL Passive DNS.
• CIRCL Passive SSL - Modules to access CIRCL Passive SSL.
• ClaamAV - Submit file to ClamAV
• Cluster25 Expand - Module to query Cluster25 CTI.
• Markdown to PDF converter - Render the markdown (under GFM) into PDF. Requires pandoc (https://pandoc.org/), wkhtmltopdf (https://wkhtmltopdf.org/) and mermaid dependencies.
• Country Code - Module to expand country codes.
• CPE Lookup - An expansion module to query the CVE search API with a cpe code to get its related vulnerabilities.
• CrowdSec CTI - Module to access CrowdSec CTI API.
• CrowdStrike Falcon - Module to query CrowdStrike Falcon.
• Cuckoo Submit - Submit files and URLs to Cuckoo Sandbox
• CVE Lookup - An expansion hover module to expand information about CVE id.
• CVE Advanced Lookup - An expansion module to query the CIRCL CVE search API for more information about a vulnerability (CVE).
• Cytomic Orion Lookup - An expansion module to enrich attributes in MISP by quering the Cytomic Orion API
• DBL Spamhaus Lookup - Checks Spamhaus DBL for a domain name.
• DNS Resolver - Simple DNS expansion service to resolve IP address from MISP attributes
• DOCX Enrich - Module to extract freetext from a .docx document.
• DomainTools Lookup - DomainTools MISP expansion module.
• EQL Query Generator - EQL query generation for a MISP attribute.
• EUPI Lookup - A module to query the Phishing Initiative service (https://phishing-initiative.lu).
• URL Components Extractor - Extract URL components
• Farsight DNSDB Lookup - Module to access Farsight DNSDB Passive DNS.
• GeoIP ASN Lookup - Query a local copy of the Maxmind Geolite ASN database (MMDB format)
• GeoIP City Lookup - An expansion module to query a local copy of Maxmind's Geolite database with an IP address, in order to get information about the city where it is located.
• GeoIP Country Lookup - Query a local copy of Maxminds Geolite database, updated for MMDB format
• Google Safe Browsing Lookup - Google safe browsing expansion module
• Google Threat Intelligence Lookup - An expansion module to have the observable's threat score assessed by Google Threat Intelligence.
• GreyNoise Lookup - Module to query IP and CVE information from GreyNoise
• Hashdd Lookup - A hover module to check hashes against hashdd.com including NSLR dataset.
• CIRCL Hashlookup Lookup - An expansion module to query the CIRCL hashlookup services to find it if a hash is part of a known set such as NSRL.
• Have I Been Pwned Lookup - Module to access haveibeenpwned.com API.
• HTML to Markdown - Expansion module to fetch the html content from an url and convert it into markdown.
• HYAS Insight Lookup - HYAS Insight integration to MISP provides direct, high volume access to HYAS Insight data. It enables investigators and analysts to understand and defend against cyber adversaries and their infrastructure.
• Intel471 Lookup - Module to access Intel 471
• IP2Location.io Lookup - An expansion module to query IP2Location.io to gather more information on a given IP address.
• IPASN-History Lookup - Module to query an IP ASN history service (https://github.com/D4-project/IPASN-History).
• IPInfo.io Lookup - An expansion module to query ipinfo.io to gather more information on a given IP address.
• IPQualityScore Lookup - IPQualityScore MISP Expansion Module for IP reputation, Email Validation, Phone Number Validation, Malicious Domain,Malicious URL Scanner,Malicious File Scanner & Compromised username, Password, Email
• IPRep Lookup - Module to query IPRep data for IP addresses.
• Ninja Template Rendering - Render the template with the data passed
• Joe Sandbox Import - Query Joe Sandbox API with a submission url to get the json report and extract its data that is parsed and converted into MISP attributes and objects.
• Joe Sandbox Submit - A module to submit files or URLs to Joe Sandbox for an advanced analysis, and return the link of the submission.
• Lastline Lookup - Deprecation notice: this module will be deprecated by December 2021, please use vmware_nsx module. Query Lastline with an analysis link and parse the report into MISP attributes and objects.
• Lastline Submit - Deprecation notice: this module will be deprecated by December 2021, please use vmware_nsx module. Module to submit a file or URL to Lastline.
• Macaddress.io Lookup - MISP hover module for macaddress.io
• Macvendors Lookup - Module to access Macvendors API.
• MalShare Upload - Module to push malware samples to MalShare
• Malware Bazaar Lookup - Query Malware Bazaar to get additional information about the input hash.
• McAfee MVISION Insights Lookup - Lookup McAfee MVISION Insights Details
• GeoIP Enrichment - A hover and expansion module to enrich an ip with geolocation and ASN information from an mmdb server instance, such as CIRCL's ip.circl.lu.
• MWDB Submit - Module to push malware samples to a MWDB instance
• OCR Enrich - Module to process some optical character recognition on pictures.
• ODS Enrich - Module to extract freetext from a .ods document.
• ODT Enrich - Module to extract freetext from a .odt document.
• Onion Lookup - MISP module using the MISP standard. Uses the onion-lookup service to get information about an onion.
• Onyphe Lookup - Module to process a query on Onyphe.
• Onyphe Full Lookup - Module to process a full query on Onyphe.
• AlienVault OTX Lookup - Module to get information from AlienVault OTX.
• Passive SSH Enrichment - An expansion module to enrich, SSH key fingerprints and IP addresses with information collected by passive-ssh
• PassiveTotal Lookup - The PassiveTotal MISP expansion module brings the datasets derived from Internet scanning directly into your MISP instance. This module supports passive DNS, historic SSL, WHOIS, and host attributes. In order to use the module, you must have a valid PassiveTotal account username and API key. Registration is free and can be done by visiting https://www.passivetotal.org/register
• PDF Enrich - Module to extract freetext from a PDF document.
• PPTX Enrich - Module to extract freetext from a .pptx document.
• Qintel QSentry Lookup - A hover and expansion module which queries Qintel QSentry for ip reputation data
• QR Code Decode - Decode QR codes from attachments OR remote URLs (Anti-Quishing).
• RandomcoinDB Lookup - Module to access the ransomcoinDB (see https://ransomcoindb.concinnity-risks.com)
• Rapid7 AttackerKB lookup - Module to lookup CVE attributes in Rapid7 AttackerKB.
• Real-time Blackhost Lists Lookup - Module to check an IPv4 address against known RBLs.
• Recorded Future Enrich - Module to enrich attributes with threat intelligence from Recorded Future.
• Reverse DNS - Simple Reverse DNS expansion service to resolve reverse DNS from MISP attributes.
• ReversingLabs Spectra Analyze - Threat intelligence enrichment module
• SecurityTrails Lookup - An expansion modules for SecurityTrails.
• Shodan Lookup - Module to query on Shodan.
• Sigma Rule Converter - An expansion hover module to display the result of sigma queries.
• Sigma Syntax Validator - An expansion hover module to perform a syntax check on sigma rules.
• SigMF Expansion - Expands a SigMF Recording object into a SigMF Expanded Recording object, extracts a SigMF archive into a SigMF Recording object.
• Socialscan Lookup - A hover module to get information on the availability of an email address or username on some online platforms.
• SophosLabs Intelix Lookup - An expansion module to query the Sophoslabs intelix API to get additional information about an ip address, url, domain or sha256 attribute.
• URL Archiver - Module to cache web pages of analysis reports, OSINT sources. The module returns a link of the cached page.
• Stairwell Lookup - Module to query the Stairwell API to get additional information about the input hash attribute
• STIX2 Pattern Syntax Validator - An expansion hover module to perform a syntax check on stix2 patterns.
• ThreatCrowd Lookup - Module to get information from ThreatCrowd.
• ThreadFox Lookup - Module to search for an IOC on ThreatFox by abuse.ch.
• ThreatMiner Lookup - Module to get information from ThreatMiner.
• Triage Submit - Module to submit samples to tria.ge
• TruSTAR Enrich - Module to get enrich indicators with TruSTAR.
• URLhaus Lookup - Query of the URLhaus API to get additional information about the input attribute.
• URLScan Lookup - An expansion module to query urlscan.io.
• Validin DNS History - Validin internet dataset expansion. Returns dns-records, web crawls, registration (WHOIS) records, and certificates from Validin's historic internet intelligence dataset.
• VARIoT db Lookup - An expansion module to query the VARIoT db API for more information about a vulnerability.
• VirusTotal v3 Lookup - Enrich observables with the VirusTotal v3 API
• VirusTotal Public API Lookup - Enrich observables with the VirusTotal v3 public API
• VirusTotal Upload - Module to push malware samples to VirusTotal
• VMRay Submit - Module to submit a sample to VMRay.
• VMware NSX Defender Enrich - Module to enrich a file or URL with VMware NSX Defender.
• VulnDB Lookup - Module to query VulnDB (RiskBasedSecurity.com).
• Vulnerability Lookup - An expansion module to query Vulnerability Lookup
• Vulners Lookup - An expansion hover module to expand information about CVE id using Vulners API.
• Vysion Enrich - Module to enrich the information by making use of the Vysion API.
• Whois Lookup - Module to query a local instance of uwhois (https://github.com/rafiot/uwhoisd).
• WhoisFreaks Lookup - An expansion module for https://whoisfreaks.com/ that will provide an enriched analysis of the provided domain, including WHOIS and DNS information.
• Wikidata Lookup - An expansion hover module to extract information from Wikidata to have additional information about particular term for analysis.
• IBM X-Force Exchange Lookup - An expansion module for IBM X-Force Exchange.
• XLXS Enrich - Module to extract freetext from a .xlsx document.
• YARA Rule Generator - The module takes a hash attribute (md5, sha1, sha256, imphash) as input, and is returning a YARA rule from it.
• YARA Syntax Validator - An expansion hover module to perform a syntax check on if yara rules are valid or not.
• Yeti Lookup - Module to process a query on Yeti.

Export Modules

• CEF Export - Module to export a MISP event in CEF format.
• Cisco fireSIGHT blockrule Export - Module to export malicious network activity attributes to Cisco fireSIGHT manager block rules.
• Microsoft Defender for Endpoint KQL Export - Defender for Endpoint KQL hunting query export module
• GoAML Export - This module is used to export MISP events containing transaction objects into GoAML format.
• Lite Export - Lite export of a MISP event.
• EQL Query Export - Export MISP event in Event Query Language
• Nexthink NXQL Export - Nexthink NXQL query export module
• OSQuery Export - OSQuery export of a MISP event.
• Event to PDF Export - Simple export of a MISP event to PDF.
• Test Export - Skeleton export module.
• ThreatStream Export - Module to export a structured CSV file for uploading to threatStream.
• ThreadConnect Export - Module to export a structured CSV file for uploading to ThreatConnect.
• VirusTotal Collections Export - Creates a VT Collection from an event iocs.
• VirusTotal Graph Export - This module is used to create a VirusTotal Graph from a MISP event.
• YARA Rule Export - This module is used to export MISP events to YARA.

Import Modules

• ANYRUN Sandbox Import - A module designed to retrieve an analysis report from the ANY.RUN Sandbox by its unique ID and extract results (such as verdict, malware tags, and IOCs), converting them into MISP attributes within your event.
• PDNS COF Importer - Passive DNS Common Output Format (COF) MISP importer
• CSV Import - Module to import MISP attributes from a csv file.
• Cuckoo Sandbox Import - Module to import Cuckoo JSON.
• Email Import - Email import module for MISP
• GoAML Import - Module to import MISP objects about financial transactions from GoAML files.
• Import Blueprint - Generic blueprint to be copy-pasted to quickly boostrap creation of import module.
• Joe Sandbox Import - A module to import data from a Joe Sandbox analysis json report.
• Lastline Import - Deprecation notice: this module will be deprecated by December 2021, please use vmware_nsx module. Module to import and parse reports from Lastline analysis links.
• MISP JSON Import - Module to import MISP JSON format for merging MISP events.
• OCR Import - Optical Character Recognition (OCR) module for MISP.
• OpenIOC Import - Module to import OpenIOC packages.
• TAXII 2.1 Import - Import content from a TAXII 2.1 server
• CSV Test Import - Simple CSV import tool with mapable columns
• ThreadAnalyzer Sandbox Import - Module to import ThreatAnalyzer archive.zip / analysis.json files.
• URL Import - Simple URL import tool with Faup
• VMRay API Import - Module to import VMRay (VTI) results.
• VMRay Summary JSON Import - Import a VMRay Summary JSON report.

Action Modules

• Export to Sentinel or Defender - Export indicators to Microsoft Sentinel or Microsoft Defender. Requires an existing installation of MISP2Sentinel or MISP2Defender.
• Mattermost - Simplistic module to send message to a Mattermost channel.
• Nextcloud talk - Simplistic module to send a message to a Nextcloud talk conversation.
• Slack - Simplistic module to send messages to a Slack channel.
• Test action - This module is merely a test, always returning true. Triggers on event publishing.