Getting started
Overview
Learn how to get started with the ReversingLabs Spectra Detect Solution for Microsoft Sentinel.
This solution is a preview state and may contain bugs, incomplete features, or limitations.
It is intended for evaluation purposes only and not recommended for production use. Features and support may change without notice. Proceed with caution and provide feedback to improve the final release.
Prerequisites
To setup and use this solution, you must have:
- A valid Spectra Detect instance
- An existing Microsoft Sentinel workspace
- An existing Log Analytics workspace
- Ability to deploy resources in Azure
Solution contents
The solution contains the following Microsoft Sentinel resources:
| Resource | Description |
|---|---|
| Function app | A Python 3.11 Azure Function app that handles processing of Spectra Detect reports into log data that can be ingested by Azure Monitor. |
| App Service Plan | The App Service plan provides hosting resources for the Function App. |
| Storage Account (x2) | Two storage accounts are deployed: Standard storage account to hold function app files Azure Data Lake Storage Gen2 (ADLS) storage for integration with Spectra Detect analysis task storage. |
| Event Grid System Topic | The system topic monitors for new blob creation in the ADLS storage account and triggers the function app. |
| Data Collection Endpoint | The DCE creates an endpoint for the Function App to send log data to for ingestion into a Log Analytics Workspace. |
| Data Collection Rule | The DCR specifies the log ingestion parameters, such as destination Log Analytics table name and table schema. |
| Log Analytics Custom Table | A custom table (default: SpectraDetectLogs_CL) is created in the provided Log Analytics Workspace. This table holds the Spectra Detect logs. |
| Microsoft Sentinel Workbook | The workbook provides visualizations of the data, enabling security teams to get insights of their Spectra Detect data at a glance. |
| Microsoft Sentinel Analytics Rule | A Microsoft Sentinel analytics rule that will generate incidents when a file receives a “malicious” classification is provided. |
Solution diagrams
Workflow overview
Cloud architecture
Installation
Deploy the ARM template
An Azure Resource Manager (ARM) deployment template has been provided to automatically deploy the solution into any Azure tenant. To deploy the template:
Deployment of the ARM template may take up to 20 minutes.
-
Download the ARM template file from the URL listed here, or click the “Deploy to Azure” button (https://strlp4j3pdo3ajy3r39hqmc.blob.core.windows.net/spectradetectsolution/azuredeploy.json):
-
The deployment menu will be presented. Provide the following values, then click "Deploy":
| Parameter Name | Example Value | Description |
|---|---|---|
subscription | Azure subscription used for billing and resource management | |
resourceGroup | (Select or create new) | Resource group to organize related resources |
region | East US | Azure region where resources will be deployed |
location | [resourceGroup().location] | Leave as-is. Location dynamically set to match the resource group's location |
logAnalyticsWorkspaceName | (Required field) | Name of the Log Analytics workspace where logs will be ingested |
logAnalyticsWorkspaceResourceGroup | (Optional) | Resource group where the Log Analytics workspace resides |
newTableName | SpectraDetectLogs | Name of the new Log Analytics custom table |
useExistingStorageAccount | false | Boolean indicating whether to use an existing storage account |
existingStorageAccountResourceGroup | (Optional) | Resource group for an existing storage account (if used) |
reportsStorageAccountName | stspectradetectlogs | Name of the storage account for storing reports |
reportsContainerName | reports | Name of the blob container for storing reports |
appServicePlanName | asp-spectradetectlogs | Name of the App Service plan |
functionAppName | funcspectradetectlogs | Name of the Azure Function App |
dataCollectionEndpointName | dce-spectradetectlogs | Name of the Data Collection Endpoint |
dataCollectionRuleName | dcr-spectradetectlogs | Name of the Data Collection Rule |
eventGridSystemTopicName | egstspectradetectlogs | Name of the Event Grid System Topic |
workbookDisplayName | ReversingLabs-SpectraDetect | Display name for the Azure Monitor workbook |
workbookResourceName | [newGuid()] | Leave as-is. Unique resource name for the workbook, generated with newGuid() |
rule1Guid | [newGuid()] | Leave as-is. Unique identifier for a specific rule, generated with newGuid() |
functionPackageUrl | <url> | URL to the zipped Azure Function app package. See below for possible values |
Use the following for the functionPackageUrl parameter:
- Version 0.3.0 (latest):
https://strlp4j3pdo3ajy3r39hqmc.blob.core.windows.net/spectradetectsolution/spectradetect-connector-0.3.0.zip
Once the deployment is complete, click the “Outputs” tab. These values will be used to configure Spectra Detect to forward task reports to the Azure Data Lake Storage container.
Configure Spectra Detect
The solution requires Spectra Detect to export analysis task reports to Azure Data Lake Storage Gen2 (ADLS). To configure Spectra Detect to send task reports to ADLS, see the following Spectra Detect documentation: https://docs.reversinglabs.com/SpectraDetect/Config/AnalysisInput/#microsoft-cloud-storage-azure-data-lake
Use the following outputs provided in the previous section for the configuration:
| Template Deployment Output | Spectra Detect Configuration Setting |
|---|---|
| adlsStorageAccountName | Storage account name |
| adlsStorageAccountAccessKey | Storage access key |
| adlsContainerName | Container |
Usage
This section provides guidance on how to use the data ingested by the solution.
Table schema
The table below describes the Columns generated in the SpectraDetectLogs_CL table:
| Field | Type | Description |
|---|---|---|
| TimeGenerated | datetime | Time the log was generated |
| task_id | string | Unique identifier for the analysis task |
| submitted | int | Unix timestamp in seconds indicating the time when the file was submitted for analysis. |
| processed | int | Unix timestamp in seconds when analysis completed |
| process_duration | int | Analysis duration in seconds |
| direct_sender | string | IP address of the machine that sent the file to Spectra Detect. This is the immediate sender, the last hop in the forwarding chain (if the request was forwarded). |
| worker_address | dynamic | List of DNS records associated with the Spectra Detect Worker that processed the file. |
| worker_hostname | string | Hostname of the Spectra Detect Worker that processed the file. |
| worker_ip | dynamic | List of IP addresses associated with the Worker that processed the file. |
| forwarded_for | dynamic | List of IP addresses that were part of the forwarding chain for that request, including the original sender. |
| info | dynamic | Contains detailed file metadata including entropy, filename, file path, type, subtype, multiple hash values (MD5, SHA1, SHA256), file size, format identification details, and unpacking status. |
| classification | dynamic | Threat classification details include threat classification level, threat name, and more. |
| index | int | The item's position in the extraction hierarchy. For example, if you submit a ZIP file, it would be index 0. Any files extracted from that ZIP would be index 1, index 2, etc. |
| parent | int | The index number of the file this was extracted from. |
| story | dynamic | Analysis summary |
| tags | dynamic | Analysis tags |
| interesting_strings | dynamic | Contains interesting strings extracted from the file, including URLs, IP addresses, and more. |
This solution processes a Spectra Detect report by taking each file (both the original submission and any extracted files) from the 'tc_report' array and creating individual records for them. Instead of keeping the nested structure, each file gets its own separate entry. The task_id value can be used to correlate related files.
Using the workbook
Included in the solution is a Microsoft Sentinel workbook which can be used to visualize the data generated by Spectra Detect.
Workbook configuration
After installing the solution, the workbook needs to be configured to point to your Microsoft Sentinel subscription and workspace. Simply click the panel buttons, select the appropriate values from the drop down list, and click the “save” icon near the top menu bar:
Example queries
Summarize file classifications for the last 7 days
This query summarizes the total number of files analyzed by Spectra Detect by their final classification:
SpectraDetectLogs_CL
| where TimeGenerated > ago(7d)
| extend reportClassification = tostring(classification.classification)
| extend Classification = case(reportClassification == 1, "goodware",
reportClassification == 2, "suspicious", reportClassification == 3,
"malicious", "unknown")
| summarize count() by Classification
Return all malicious files identified by Spectra Detect
This query returns all files with a final classification of “3” or “malicious”:
SpectraDetectLogs_CL
| extend reportClassification = tostring(classification.classification)
| extend Classification = case(reportClassification == 1, "goodware",
reportClassification == 2, "suspicious", reportClassification == 3,
"malicious", "unknown")
| where Classification == "malicious"
Identify the parent file
This query identifies the top-level parent files for all Spectra Detect analysis tasks based on the task ID.
SpectraDetectLogs_CL
| where task_id == task_id and index == 0