# ReversingLabs Documentation > Technical documentation for ReversingLabs file analysis, threat intelligence, and malware detection products. This file contains all documentation content in a single document following the llmstxt.org standard. ## General Concepts and Reference — ReversingLabs # General Concepts and Reference This section covers core concepts, terminology, and reference documentation shared across all ReversingLabs products. Use this as a starting point to understand file analysis fundamentals, classification methodology, deployment requirements, and security configuration. ## Quick Navigation - [Analysis & Classification](/General/category/analysis--classification/) — How file analysis works, classification algorithms, and threat detection - [Deployment & Integration](/General/category/deployment--integration/) — Platform requirements, network configuration, and deployment options - [Security & Access Control](/General/category/security--access-control/) — Authentication, SSO, and privacy policies - [Spectra Core](/General/Spectra-Core/) — Technical reference for indicators, tags, and analysis reports --- ## Glossary ## Products | Term | Definition | |------|------------| | Spectra Analyze | Malware analysis platform for individual analysts and small teams, powered by Spectra Core | | Spectra Detect | Automated, high-throughput malware detection solution for enterprise file analysis | | Spectra Intelligence | Cloud-based threat intelligence service providing file reputation, malware analysis, and threat feeds via APIs | | Spectra Core | ReversingLabs proprietary file analysis engine that powers static analysis across all Spectra products | | File Inspection Engine (FIE) | High-throughput file analysis solution for inspecting network traffic | | T1000 | Network appliance for real-time file inspection and threat detection | ## Analysis & Classification | Term | Definition | |------|------------| | Classification | ReversingLabs algorithm that determines the security status of files as malicious, suspicious, goodware, or unknown | | Risk score | Numeric value (0-10) indicating the likelihood that a file is malicious | | Threat level | The classification category assigned to a file after analysis: Unknown, Goodware, Suspicious, or Malicious. Corresponds to the risk score ranges on the Spectra Analyze risk score scale (0–10). | | Trust factor | Confidence level in the classification result | | Static analysis | Examination of file properties without executing the file | | Dynamic analysis | Analysis of file behavior by executing it in a sandboxed environment | | APT (Advanced Persistent Threat) | A prolonged, targeted cyberattack in which an adversary gains and maintains unauthorized access to a network while remaining undetected. APTs often use sophisticated techniques including zero-day exploits, social engineering, and lateral movement. | | False positive | A benign file or activity incorrectly classified as malicious by a security tool. In ReversingLabs products, false positives can be corrected using classification overrides on Spectra Analyze or through Spectra Intelligence reclassification requests. | | False negative | A malicious file or activity that is not detected by a security tool and is incorrectly classified as benign. False negatives can occur when malware uses novel techniques not yet covered by existing signatures or analysis heuristics. | | Heuristic analysis | A detection method that uses rules and algorithms to identify suspicious characteristics in files without relying on exact signature matches. Heuristic analysis can detect previously unknown malware variants by evaluating behavioral patterns and structural anomalies. | | Packer | Software that compresses or encrypts executable files to reduce their size or obfuscate their contents. Malware authors frequently use packers to evade signature-based detection. Spectra Core supports unpacking of over 400 packer formats during static analysis. | | PE (Portable Executable) | The standard file format for executables, DLLs, and other binary files on Windows operating systems. PE analysis is a core capability of Spectra Core, which extracts detailed header information, imports, exports, and section data from PE files. | | Obfuscation | Techniques used to make code or data difficult to understand or analyze. Common obfuscation methods include string encryption, control flow flattening, and dead code insertion. Spectra Core detects many obfuscation techniques as indicators during static analysis. | | Sandbox | An isolated environment used to execute and observe suspicious files without risk to production systems. Spectra Analyze supports integration with dynamic analysis sandboxes for behavioral analysis of samples. | | Threat propagation | The process by which a parent file inherits the classification of its extracted child files. In ReversingLabs products, if a malicious executable is found inside a ZIP archive, the archive also receives a malicious classification through upward propagation. | | Goodware override | A classification mechanism where a trusted parent sample's benign classification is propagated to its extracted child files. This is used when a known-good installer contains files that might otherwise trigger false positive detections. | ## Malware Types | Term | Definition | |------|------------| | Backdoor | Malware that provides an attacker with unauthorized remote access to a compromised system, bypassing normal authentication. Backdoors are often installed by other malware or through exploited vulnerabilities. | | Botnet | A network of compromised computers (bots) controlled by an attacker through a command-and-control (C2) server. Botnets are used for distributed denial-of-service attacks, spam distribution, and credential theft. | | Fileless malware | Malware that operates entirely in memory without writing files to disk, making it harder to detect with traditional file-based scanning. Fileless malware often leverages legitimate system tools like PowerShell or WMI. | | Keylogger | Malware that records keystrokes on a compromised system to capture sensitive information such as passwords, credit card numbers, and personal data. Keyloggers may operate at the hardware, kernel, or application level. | | Polymorphic malware | Malware that changes its code structure with each infection while maintaining the same malicious functionality. Polymorphic techniques include encrypting the payload with a different key each time, making signature-based detection less effective. | | Ransomware | Malware that encrypts files on a victim's system and demands payment for the decryption key. Ransomware typically receives a risk score of 10 (maximum severity) in ReversingLabs classification. | | Rootkit | Malware designed to hide its presence and the presence of other malicious software on a compromised system. Rootkits can operate at the user level, kernel level, or firmware level, making detection and removal challenging. | | Trojan | Malware disguised as legitimate software to trick users into executing it. Unlike viruses and worms, Trojans do not self-replicate. They are commonly used to deliver other malware payloads, steal data, or provide backdoor access. | | Worm | Self-replicating malware that spreads across networks without requiring user interaction. Worms exploit vulnerabilities in network services or operating systems to propagate and can cause significant network disruption. | ## Security Frameworks and Standards | Term | Definition | |------|------------| | CVE (Common Vulnerabilities and Exposures) | A standardized identifier for publicly known cybersecurity vulnerabilities, maintained by MITRE Corporation. Each CVE entry includes a unique ID (e.g., CVE-2024-1234), a description, and references. | | CWE (Common Weakness Enumeration) | A community-developed list of common software and hardware security weaknesses, maintained by MITRE. CWE entries categorize vulnerability types to help developers and analysts understand root causes. See the [CWE list](https://cwe.mitre.org/). | | MITRE ATT&CK | A globally recognized knowledge base of adversary tactics, techniques, and procedures (TTPs) based on real-world observations. ReversingLabs maps static analysis indicators to ATT&CK techniques in Spectra Analyze and Spectra Intelligence. | | NIST (National Institute of Standards and Technology) | A U.S. federal agency that develops cybersecurity standards and guidelines, including the [NIST Cybersecurity Framework](https://www.nist.gov/cyberframework) and SP 800 series. NIST frameworks are widely adopted for enterprise security program management. | ## Security Operations | Term | Definition | |------|------------| | C2/C&C (Command and Control) | Infrastructure used by attackers to communicate with and control compromised systems. C2 channels can use HTTP, DNS, or custom protocols. Spectra Core extracts C2 indicators from malware configurations during analysis. | | DLP (Data Loss Prevention) | Security technology designed to detect and prevent unauthorized data exfiltration from an organization. DLP solutions can integrate with ReversingLabs products to scan files for malicious content before allowing transfers. | | EDR (Endpoint Detection and Response) | Security technology that monitors endpoints for suspicious activity and provides investigation and response capabilities. EDR solutions can integrate with ReversingLabs threat intelligence for enhanced file reputation checks. | | IDS/IPS (Intrusion Detection/Prevention System) | Network security systems that monitor traffic for malicious activity. IDS detects and alerts on threats; IPS actively blocks them. File Inspection Engine can complement IDS/IPS by providing deep file analysis for network traffic. | | NDR (Network Detection and Response) | Security technology that monitors network traffic to detect threats and anomalous behavior. NDR solutions benefit from integration with ReversingLabs file analysis for inspecting files extracted from network flows. | | Reverse engineering | The process of analyzing software or malware to understand its functionality, behavior, and purpose. Spectra Core performs automated reverse engineering through static decomposition, extracting indicators, metadata, and structural information from binary files. | | SIEM (Security Information and Event Management) | A platform that aggregates and analyzes security event logs from across an organization's infrastructure. ReversingLabs integrates with SIEM platforms like Splunk and IBM QRadar. See [Integrations](/Integrations/). | | SOAR (Security Orchestration, Automation and Response) | A platform that automates security operations workflows, including incident response and threat investigation. ReversingLabs provides integrations with SOAR platforms including Splunk SOAR and Palo Alto XSOAR. See [Integrations](/Integrations/). | | SOC (Security Operations Center) | A centralized team responsible for monitoring, detecting, analyzing, and responding to cybersecurity incidents. SOC analysts use ReversingLabs products for malware triage, threat hunting, and incident investigation. | | Supply chain attack | A cyberattack that targets an organization by compromising a trusted third-party vendor, software library, or update mechanism. ReversingLabs products help detect supply chain threats by analyzing software packages and their dependencies. | | XDR (Extended Detection and Response) | An integrated security approach that correlates data across endpoints, networks, cloud, and email to provide unified threat detection and response. XDR platforms can leverage ReversingLabs threat intelligence for file-level analysis. | | Zero-day | A previously unknown vulnerability that has no existing patch or signature. Zero-day exploits are particularly dangerous because traditional signature-based detection cannot identify them. Heuristic and behavioral analysis methods provide better coverage against zero-day threats. | ## Threat Intelligence | Term | Definition | |------|------------| | IOC (Indicator of Compromise) | Observable artifact such as a hash, IP address, or domain that indicates a security breach | | YARA | Pattern-matching tool used to identify and classify malware based on textual or binary patterns | | MISP | Open-source threat intelligence sharing platform | | STIX | Structured Threat Information Expression — standard language for sharing cyber threat intelligence | | TAXII | Trusted Automated Exchange of Intelligence Information — transport protocol for sharing STIX data | | TCA | API endpoint code prefix used in Spectra Intelligence API documentation (e.g., TCA-0101) | | TCF | Feed endpoint code prefix used in Spectra Intelligence Feed documentation (e.g., TCF-0101) | ## Infrastructure | Term | Definition | |------|------------| | Helm chart | Kubernetes package used to deploy Spectra Detect and File Inspection Engine | | OVA | Open Virtual Appliance format used for deploying Spectra Analyze and T1000 as virtual machines | | ICAP (Internet Content Adaptation Protocol) | A protocol for offloading content scanning from web proxies and load balancers to external services. Spectra Detect Hub and Spectra Analyze support ICAP integration for inline file scanning. See [ICAP Integration](/Integrations/category/icap/). | | Kubernetes | An open-source container orchestration platform used for deploying and scaling containerized applications. Both Spectra Detect and File Inspection Engine support Kubernetes deployment via Helm charts. | | OCI (Open Container Initiative) | A set of industry standards for container image formats and runtimes. File Inspection Engine is distributed as an OCI-compliant container image that runs on Docker, Kubernetes, or any compatible runtime. | --- ## Getting started with Spectra Analyze This guide walks you through your first malware analysis with Spectra Analyze. You will configure the Spectra Intelligence cloud connection, upload a file, view the analysis report, and understand the classification result. ## Prerequisites Before you begin: - Spectra Analyze is deployed and accessible on your network (hardware appliance, OVA, or cloud instance) - You have login credentials (username and password) - Your browser is up to date (Chrome recommended) If you have not yet deployed Spectra Analyze, see [Setup and initial configuration](./initial-configuration.md). **Tip: Initial credentials** Your initial administrator username and password are provided by [ReversingLabs Support](mailto:support@reversinglabs.com). Change the default password after your first login. ## Step 1: Configure the Spectra Intelligence cloud connection Spectra Analyze enriches every analyzed file with data from [Spectra Intelligence](/SpectraIntelligence/) — a cloud threat intelligence service that provides scanner results, threat names, and community classifications. Without this connection, Spectra Analyze still performs local static analysis, but enrichment data will be missing from reports. 1. Navigate to **Administration > Configuration > Spectra Intelligence**. 2. Enter your Spectra Intelligence **username** and **password**. 3. Select **Test connection** to confirm the appliance can reach `appliance-api.reversinglabs.com`. 4. Select **Save**. If the connection test fails, verify that your network allows outbound HTTPS (port 443) to `appliance-api.reversinglabs.com`. You may also need to allowlist the Cloudflare IP ranges and `185.64.132.0/22`. See [IP/domain allowlisting](./initial-configuration.md#ipdomain-allowlisting) for details. **Note: If you do not yet have Spectra Intelligence credentials, contact [ReversingLabs Support](mailto:support@reversinglabs.com).** ## Step 2: Access the web interface Open your browser and navigate to the Spectra Analyze appliance URL provided by your administrator (for example, `https://spectra-analyze.example.com`). Log in with your credentials. After logging in, you land on the **Dashboard**, which shows recent analysis activity and system status. ## Step 3: Upload a file for analysis 1. Select **Submit** from the header bar. 2. Select the file you want to analyze. Supported formats include executables, archives, documents, and mobile apps — over 400 formats in total. 3. Optionally add a comment or tags to help organize your results later. 4. Select **Upload**. Spectra Analyze submits the file to [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) for static analysis. Most files complete analysis within seconds. ## Step 4: View the analysis report When analysis completes, the file appears in your results list. Select the file to open its **Sample Details** page. The report includes: - **Classification** — Malicious, Suspicious, Goodware, or Unknown, with a risk score from 0 to 10 - **Threat name** — For malicious files, the malware family name - **Indicators** — Extracted behavioral indicators mapped to [MITRE ATT&CK](https://attack.mitre.org/) tactics - **Extracted files** — Child files unpacked from archives or installers - **Metadata** — File hashes, format details, certificate information See [Sample Details](./Sample%20Details/index.md) for a full explanation of every report field. ## Step 5: Understand your results Spectra Analyze classifies every analyzed file and assigns a risk score. This section explains how to read and interpret that classification. **Info: Read more about the ReversingLabs [classification algorithm](/General/AnalysisAndClassification/Classification).** ### Color coding and sample status Spectra Analyze uses consistent color coding throughout the interface to indicate sample classification and risk. ![Four colors (left to right: red indicating malicious, green indicating goodware, orange indicating suspicious, and dark gray indicating no threats were found) used in the interface to indicate sample status](images/a1000-cheatsheet-color-intro.png) Inside each symbol is the sample's risk score. Samples with no threats found do not have a risk score. Samples with a risk score of 5 are represented using a unique icon. The indicators found during analysis were deemed insufficient to classify the file as definitively malicious or benign. These samples have a higher chance of changing classification if new information becomes available. Color-coded indicators appear in the following parts of the interface: - **YARA page** — statistics about ruleset matches ![Section of the YARA page showing color-coded status for YARA rulesets](images/a1000-cheatsheet-yara-rulesets.png) - **Sample Details page** — as the background color of the Report Summary Header ![Upper part of the Sample Details page with numbered indicators showing color-coded sample status](images/a1000-cheatsheet-circles.png) - **How We Caught This tab** — in the File Similarity and File Classifications sections ![Section of the Sample Details page with numbered indicators showing color-coded sample status](images/a1000-cheatsheet-circles4.png) - **URI stats** — on the Interesting Strings, Network References, and Malware Configurations tabs ![Network References section of the Sample Details page with color-coded indicators of sample status](images/a1000-cheatsheet-circles3.png) ### Understanding the risk score The risk score is a numerical value from 0 to 10 that indicates the severity of the classification. | Risk score | Classification | Meaning | |---|---|---| | 0 | Unknown | Not enough information to classify | | 1 | Goodware | Known-clean file | | 2–4 | Suspicious | Concerning indicators, not confirmed malicious | | 5 | Suspicious | Insufficient indicators to confirm malicious or benign; classification may change | | 6–10 | Malicious | Confirmed malware; higher scores indicate greater severity | Risk scores are assigned by the severity of the detected threat. Values from 0 to 5 are reserved for samples classified as goodware or known. Lesser threats like adware get a risk score of 6, while ransomware and trojans always get a risk score of 10. Some users may choose to ignore lower risk classifications, but those samples are still classified as malicious to warn about potential threats, such as installers that have embedded adware. Some Spectra Analyze APIs return sample classification as a numerical value: `0` for unknown, `1` for goodware, `2` for suspicious, and `3` for malicious. ### Understanding the classification reason The classification reason specifies which technology detected a threat. Files usually have multiple detections from more than one classifier, but the classification reason tile specifies which one produced the final classification. The final classification always matches one of the classifiers, but individual classifiers may have differing results between them. Due to differences in how different malicious samples and malware families behave, some samples might end up classified as malicious by one technology, and still be considered goodware by others. This does not negate or diminish the final classification — it is why Spectra Analyze uses multiple sources to classify files. Overlooking the classification reason may result in confusion. For example, adding custom YARA rules to Spectra Analyze is a powerful malware hunting feature, but improperly written rulesets can be too broad and result in a large number of samples suddenly getting classified as malicious by a YARA rule, even though everything else points to goodware. On the **Sample Details** page, the **How We Caught This** tab shows: - **File Similarity** — whether the file matches known malicious or clean samples in the Spectra Intelligence database - **File Classifications** — the specific YARA rules, threat names, or scanner results that contributed to the classification Reviewing the classification reason helps you assess whether the result is expected and whether further investigation is needed. ### Understanding threat propagation Threat propagation is the mechanism by which classification spreads from a child file to its parent. When Spectra Analyze unpacks an archive or installer, it analyzes every contained object. If a child file is classified as malicious, that classification propagates upward to the parent. This means a container file such as a ZIP archive can receive a malicious classification even if the archive itself contains no malicious code — the threat is in its contents. The **Extracted Files** section of the Sample Details page shows the full unpacked file tree and the classification of each object. ### Administering classification overrides Classifications can be overridden either through Goodware Overrides, where the classification of a high-trust parent sample is propagated down to extracted files, or manually by using the Spectra Analyze override feature. If you disagree with a classification, you can override it manually. Overrides are useful when a file is misclassified due to incomplete intelligence data or when internal tooling triggers a false positive. To override a classification: 1. Open the **Sample Details** page for the file. 2. Select **Override classification**. 3. Select the correct classification and provide a reason. 4. Select **Save**. In cases where the classification was set by the user, it is considered final and displayed in the header, as well as in the table below. Local classification overrides are visible as final to all appliance users, while the Spectra Intelligence overrides are additionally synchronized with other appliances using the same Spectra Intelligence account, and other Spectra Intelligence accounts belonging to the same company, indicated by the middle segment of the username — `u/company/user`. In cases where a sample has both overrides, the local override is displayed as the final classification. Overrides are visible to all users on the appliance and are logged for audit purposes. See [Classification](/General/AnalysisAndClassification/Classification/) for details on how scoring works and how overrides interact with threat propagation. ### Understanding the overall classification The overall classification shown at the top of the Sample Details page represents the final verdict for the submitted file, taking into account: - The file's own static analysis results - Threat propagation from any extracted child files - Any applied classification overrides - Enrichment data from Spectra Intelligence In most cases, analysis results are overwhelmingly biased towards a certain classification: a malicious file will be detected as such by Spectra Core and backed up by a large percentage (not necessarily 100%) of scanner detections. It will probably have an exact threat name, naming the malware type or the specific malware family. Certain file types might belong to a RHA File Similarity bucket with other malicious files. Even if such files get some negative results from specific technologies, it is highly unlikely that they are not malicious. On some occasions, a sample classified as goodware can still have some Spectra Intelligence scanner detections, but not enough to affect the final classification. These detections are most likely false positives, but users are still advised to check the scanner list to see which scanners detected a threat. In conclusion, while false detections are not impossible, they are much easier to identify if all of the important factors are considered and understood. For more information, see [Classification](/General/AnalysisAndClassification/Classification/) and [Risk Score Table](/General/AnalysisAndClassification/RiskScoreTable/). When a file has been overridden, the interface distinguishes between the original classification and the overridden classification so you can always see both the automated result and the manual correction. ### MITRE ATT&CK integration Spectra Analyze maps indicators detected during static analysis to the MITRE ATT&CK framework. When a sample's analysis reveals indicators that correspond to known ATT&CK tactics and techniques, they are displayed in the ATT&CK section of the Sample Details page. This mapping helps analysts correlate file-level findings with adversary behaviors documented in the ATT&CK knowledge base. ## Next steps - [Search for files](./search-page.md) — query your analyzed file collection - [YARA rules](./yara.md) — write and deploy custom detection rules - [API Documentation](./API%20Documentation/index.md) — automate file submissions and retrieve results programmatically - [Dashboard](./dashboard.md) — monitor appliance health and throughput --- ## Spectra Analyze — Malware Analysis for Threat Analysts Spectra Analyze is a malware analysis solution designed for individual analysts and small teams. It combines deep static file analysis with cloud threat intelligence to accelerate threat detection, investigation, and collaboration. Spectra Analyze is available as a hardware/software appliance or as a cloud-based service. ## Introduction Spectra Analyze is powered by [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis), the ReversingLabs engine for automated static decomposition and analysis of binary files. When you submit a file, Spectra Core unpacks it and extracts all available information from every contained object. The unpacking process supports all variants of more than 400 formats, including PE packers, archives, installation packages, firmware images, documents, and mobile applications. Once unpacked, Spectra Core extracts metadata from each file: strings, format header details, function names, library dependencies, file segments, and static behavior analysis with capability information. This data is combined with threat intelligence from Spectra Intelligence, a cloud service that provides scanner results, threat names, and community classifications, to produce a complete analysis report. The result is a classification of each analyzed file as malicious, suspicious, goodware, or unknown, along with a risk score and the indicators that drove the result. For more information about the latest features, see the [Spectra Analyze website](https://www.reversinglabs.com/products/spectra-analyze). **Tip: New to Spectra Analyze?** See [Getting Started](./getting-started.md) to configure your first cloud connection, upload a file, and interpret your first analysis report. ## Navigating the Interface This section describes the layout of the Spectra Analyze interface, including the terminology and visual indicators used throughout the appliance and its documentation. ### Global Header Bar The global header bar runs across the top of every page in the Spectra Analyze interface. It contains the most commonly used controls and the main navigation menu. From left to right: **Search Samples** is a text field for entering search queries. Clicking the field expands it across the header bar and hides other menu items to give more space for constructing queries. Performing a search navigates to the **Advanced Search** page. See [Advanced Search](./advanced-search.md). **Submit** opens the file and URL submission dialog. An upload progress bar appears in the header bar while files are uploading. Navigating away from the page or refreshing the browser during an upload will cancel it. See [File and URL Submissions](./file-submissions.md). **Dashboard** displays submission and processing statistics for a specified time range. See [Dashboard](./dashboard.md). **Search & Submissions** provides access to the Advanced Search feature for querying analyzed files across local and global Spectra Intelligence data sets. See [Search & Submissions Page](./search-page.md). **YARA** allows rule-based identification and classification of files using text or binary patterns. If this item is not visible, it can be enabled under **Administration > User Roles**. See [YARA Hunting](./yara.md). **Graph** provides an interactive visualization of relationships between malware samples, files, domains, IPs, and other entities. See [Graph Page](graph.md). **Help** contains links to appliance documentation, ReversingLabs support, and the legend for risk score indicators. **Quota Indicator** (pie chart icon) shows the current usage status of all ReversingLabs APIs configured on the appliance that have a limited quota. **Alerts Indicator** enables real-time monitoring of changes in malware classification and analysis results. If there are unresolved alerts, their count appears as a red badge on the indicator. See [Alerts](./alerts.md). **Health Status Indicator** shows the current status of the appliance. See [Health Status Indicator](#health-status-indicator) below. **User menu** shows the current username and provides access to Administration, User Profile, and Log Out. **Risk Tolerance** shows the current risk tolerance level for the appliance, if enabled. See [Risk Tolerance](./Appendix/risktolerance.md). ### Health Status Indicator The second to last item in the main appliance menu is the health status indicator, pointing out issues with the system load and showing if the appliance is connected to a file reputation service, and if the service is reachable or not. Administrators can click the icon to open a pop-up with a more granular look at the system resources: Disk Usage, Memory usage, CPU utilization, outstanding alerts and issues, and Spectra Detect Manager connection status. The pop-up also contains a link to [Open System Status page](./Administration/usage-alerts/system-status/) with detailed information on all the system monitoring services. Thresholds for these services, such as CPU/memory/disk usage or queue sizes, are configured on the [Administration > Configuration > System Health](./Administration/configuration-update/configuration/#system-health) page. If there are no issues with the system load and if the appliance is connected to Spectra Intelligence or the T1000 File Reputation Appliance, this is indicated by a black icon. A red icon with a small exclamation mark means that the reputation service is not configured or reachable or that CPU/memory/disk usage or queue sizes went over the configured thresholds. In this case, hovering over the icon lists the detected issues. ![description](images/analyze-status-indicator.png) The services used to deliver file reputation data (Spectra Intelligence or T1000 File Reputation Appliance) can be configured under **Administration > Configuration** in the [Spectra Intelligence](./Administration/configuration-update/configuration/#spectra-intelligence) and [T1000 File Reputation Appliance](./Administration/configuration-update/configuration/#t1000-file-reputation-appliance) dialogs. --- ## Getting Started with Spectra Detect — First Login, SDM Setup, and File Analysis # Getting started with Spectra Detect This guide walks you through logging in to Spectra Detect Manager (SDM), connecting to the Spectra Intelligence cloud, and running your first file analysis workflow. **What you'll accomplish:** - Log in to Spectra Detect Manager - Connect to the Spectra Intelligence cloud - Configure your first scan input - Submit a file and view analysis results ## Prerequisites Before you begin: - Access to your organization's Spectra Detect deployment (or deployment credentials if setting up from scratch) - Familiarity with your chosen deployment model (OVA/AMI, K8s Mono, or K8s Micro) - For new deployments: access to the appropriate infrastructure (VMware/AWS for OVA/AMI, Kubernetes cluster for K8s deployments) **Tip: Initial credentials** Your initial SDM administrator username and password are provided by [ReversingLabs Support](mailto:support@reversinglabs.com). Change the default password after your first login. ## Step 1: Access your deployment Spectra Detect is available in two deployment models (OVA/AMI, K8s Micro). This guide assumes you have already deployed Spectra Detect or have access to an existing deployment. For deployment instructions, see: - [AWS EKS Micro Deployment](/SpectraDetect/Deployment/AWS-EKS-Deployment-Micro/) — microservices on AWS EKS - [On-Premises Deployment](/SpectraDetect/Deployment/) — hardware and VM options ## Step 2: Log in to Spectra Detect Manager Spectra Detect Manager (SDM) is the central web interface for configuring Workers, monitoring analysis activity, and managing the cluster. It is available in OVA/AMI and K8s Micro deployments. 1. Open your browser and navigate to the SDM URL provided by your administrator (for example, `https://sdm.example.com`). 2. Log in with the administrator credentials provided by [ReversingLabs Support](mailto:support@reversinglabs.com). 3. Change the default password when prompted. **Note: SDM is not included in the K8s Micro deployment (v5.7). Refer to the [AWS EKS Micro Deployment](./Deployment/AWS-EKS-Deployment-Micro/index.md) guide for configuration in that model.** ## Step 3: Configure the Spectra Intelligence cloud connection Spectra Detect Manager must be connected to [Spectra Intelligence](/SpectraIntelligence/) to receive system updates, appliance upgrades, and cloud-enriched classification data. Without this connection, software updates cannot be delivered automatically and Deep Cloud Analysis will not be available. 1. In SDM, navigate to **Administration > Spectra Detect Manager**. 2. Scroll to the **Spectra Intelligence** section. 3. Select the **Enable Spectra Intelligence** checkbox. 4. Enter your Spectra Intelligence **username** and **password**. 5. Select **Save**. SDM will restart and begin polling Spectra Intelligence every 60 minutes. If the connection fails, verify that your network allows outbound HTTPS (port 443) to `appliance-api.reversinglabs.com`. See [Network Ports](/SpectraDetect/Admin/ManagerSettings/#network-ports) for the full list of required ports. **Note: If you do not yet have Spectra Intelligence credentials, contact [ReversingLabs Support](mailto:support@reversinglabs.com).** ## Step 4: Configure a scan input Configure at least one file input source so Spectra Detect knows where to pick up files for analysis. 1. In SDM, navigate to **Configuration > Analysis Input**. 2. Select your input type (S3 bucket, ICAP, network share, or API submission). 3. Enter connection details and authentication credentials. 4. Select **Save**. For detailed input options, see [Analysis Input Configuration](/SpectraDetect/Config/AnalysisInput/). ## Step 5: Submit a file and view results Once a scan input is configured, files submitted through that input are automatically analyzed by [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis). To verify analysis is working: 1. Submit a test file through your configured input. 2. Open the [Dashboard](/SpectraDetect/Usage/Dashboard/) in SDM. 3. Confirm the file appears with a classification (Malicious, Suspicious, Goodware, or Unknown). 4. Select the file to view its full analysis report. ## Step 6: Configure YARA rules (optional) Spectra Detect supports custom [YARA rules](/SpectraDetect/Usage/YARA/) for detection of specific file patterns. Rules can be uploaded through SDM and are automatically synchronized across all connected Workers. ## Next steps Now that you've configured your first scan input and verified analysis is working, explore the full capabilities of Spectra Detect: - **[Appliance Configuration](/SpectraDetect/Config/ApplianceConfiguration/)** — Network and system settings - **[Manager Settings](/SpectraDetect/Admin/ManagerSettings/)** — SDM configuration reference - **[Dashboard](/SpectraDetect/Usage/Dashboard/)** — Monitoring cluster status and throughput - **[Management API](/SpectraDetect/API/ManagementAPI/)** — Automate Spectra Detect via REST API - **[Updating](/SpectraDetect/Admin/Updating/)** — Software update procedures --- ## Spectra Detect — Enterprise File Analysis and Malware Detection Platform # Spectra Detect — Enterprise File Analysis and Malware Detection Spectra Detect is a file analysis system available in two deployment configurations: - **OVA/AMI deployment**: traditional virtual machine deployment with Workers, Hubs, and Spectra Detect Manager (SDM). - **K8s Micro deployment**: redesigned architecture where traditional Workers and Hubs are decomposed into specialized microservices. - Workers are broken down into individual processing components. - Hub functionality is currently supported only for S3 connector integration. - SDM is not included in this preview release. ## About Spectra Detect Spectra Detect uses a flexible cluster architecture that scales incrementally to support distributed or centralized file processing across physical and cloud environments. The cluster can incrementally scale file processing capacity from 100K to 100M files per day by adding Worker nodes. Spectra Detect is an enterprise-scale automated malware detection platform built for organizations that need to inspect millions of files per day across email gateways, file shares, S3 buckets, ICAP proxies, and other ingestion points — without creating bottlenecks or slowing down production workflows. Powered by [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis), it performs deep static analysis on over 400 file formats, identifying malware, suspicious indicators, and embedded threats in seconds per file. Analysis results include full indicator extraction, threat names, risk scores, and mapping to [MITRE ATT&CK](https://attack.mitre.org/) tactics — all delivered without executing files. ## File analysis The platform scales horizontally from 100,000 to 100 million files per day by adding Worker nodes. In virtual machine deployments (OVA/AMI), Workers can be provisioned manually to match capacity needs. In Kubernetes deployments (K8s Micro), Workers auto-scale based on queue depth. Results feed directly into existing security infrastructure — SIEM, SOAR, EDR, and threat intelligence platforms — through webhooks, S3 output, and REST APIs. Enterprise YARA rule deployment and synchronization across all Workers is managed centrally through Spectra Detect Manager. In OVA/AMI deployments, every Worker contains an instance of [Spectra Core](https://www.reversinglabs.com/products/spectra-core), a platform for automated static decomposition and analysis of files. ## Architecture In K8s Micro deployment, the Spectra Core functionality is distributed across multiple microservice components that work together to analyze files. Spectra Detect uses a three-tier cluster architecture: a central manager for control and configuration, one or more Hubs for ingestion and load distribution, and horizontally scalable Workers that perform the analysis. ![Spectra Detect architecture](./images/spectra-detect-architecture.png) Spectra Core can automatically unpack and extract information from more than 300 PE packers, archives, installation packages, firmware images, documents, and mobile application formats. The extracted information includes metadata such as strings, format header details, function names, library dependencies, file segments, and capabilities. This information is contained in the Worker analysis report (JSON file). ### Components **Spectra Detect Manager (SDM)** SDM is the central control plane for the entire cluster. It provides a web UI and REST API for monitoring appliance health, managing licenses, deploying software updates, and synchronizing YARA rules across all connected Workers and Spectra Analyze appliances. SDM connects to [Spectra Intelligence](/SpectraIntelligence/) for cloud-enriched classifications and automatic update delivery. Spectra Detect Manager (SDM) is a management platform that provides a centralized view of the status of ReversingLabs appliances, centralized software upgrades, configuration of authorized appliances, and YARA rules deployment. SDM is available in OVA/AMI deployments. It is not included in the K8s Micro preview (v5.7). **Info: SDM is available in OVA/AMI deployments, but is **not included** in the K8s Micro deployment preview (v5.7).** **Hub** The Hub is the ingestion and distribution layer between file sources and Workers. It receives files from configured input connectors — S3 buckets, file shares, ICAP proxies, email gateways, and direct API submissions — and distributes them to available Workers for analysis. In high-availability configurations, two Hubs can be deployed for redundancy. **Workers** Workers perform the actual file analysis using [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis). Each Worker unpacks and inspects submitted files, extracts indicators, assigns a risk score, and returns the result to the Hub. Workers scale horizontally: additional Workers are added manually in OVA/AMI deployments or provisioned automatically by Kubernetes based on queue depth. ### Deployment models | Model | Description | Scaling | |---|---|---| | OVA/AMI | Virtual machine with Workers, Hubs, and SDM | Manual — provision new VMs | | K8s Micro | Microservices architecture on Kubernetes | Automatic — per-component scaling | The Manager functions as a mediator between ReversingLabs appliances connected to it. When YARA rulesets are uploaded to any of the connected appliances that support them, the Manager ensures the rulesets are synchronized across all applicable appliances. ### Multi-region deployment In multi-region deployments, global load balancers are added in front of each tier to distribute traffic across geographic locations and provide fault tolerance. - Status overview for multiple ReversingLabs product types - License management for connected Spectra Analyze and Spectra Detect Worker appliances - Control for upgrading Spectra Analyze, Spectra Detect Worker and Hub - Centralized YARA rules deployment and synchronization between Spectra Analyze, and Spectra Detect Worker - Alerts for critical system services - Support for sample search across all connected and authorized Spectra Analyze appliances - Configuration modules for centralized management of Spectra Analyze, Spectra Detect Worker and Hub - Support for configuring the Connectors service on Spectra Analyze and Spectra Detect appliances A global load balancer sits in front of the SDM cluster, routing management and API traffic to the active SDM instance regardless of which region handles the request. A separate load balancer sits in front of the Hub cluster, distributing file submission traffic across Hubs. Workers are deployed in two or more independent regional clusters — each region runs its own Hub-and-Worker pool, so file analysis stays local to the region where files are submitted. The SDM cluster spans regions for centralized control while Worker clusters remain regionally isolated for performance and data residency. **Components added in multi-region deployments:** - **Global Load Balancer (SDM)** — routes SDM API and UI traffic to the active SDM node; handles failover between SDM instances across regions - **Global Load Balancer (Hub)** — distributes incoming file submissions across regional Hub clusters; can route by geography to keep analysis traffic local - **Regional Hub + Worker clusters** — each region runs an independent Hub-and-Worker pool; Workers in each region auto-scale independently based on local queue depth ## Documentation ### Getting started - [Getting started](/SpectraDetect/getting-started/) — first login, cloud connection, and first analysis ### Deployment - [AWS EKS Micro Deployment](/SpectraDetect/Deployment/AWS-EKS-Deployment-Micro/) — microservices architecture deployment on AWS EKS - [On-Premises Deployment](/SpectraDetect/Deployment/) — hardware and virtual machine deployment options ### Configuration - [Appliance Configuration](/SpectraDetect/Config/ApplianceConfiguration/) — network, storage, and system settings - [Analysis Input](/SpectraDetect/Config/AnalysisInput/) — configure file sources and input connectors - [YARA Sync](/SpectraDetect/Config/YARASync/) — synchronize YARA rules across appliances - [Certificate Management](/SpectraDetect/Config/CertificateManagement/) — TLS/SSL certificate configuration ### Usage - [Dashboard](/SpectraDetect/Usage/Dashboard/) — monitoring cluster status and analysis results - [Analysis](/SpectraDetect/Usage/Analysis/) — understanding analysis results and classifications - [YARA Rules](/SpectraDetect/Usage/YARA/) — managing and deploying YARA rulesets ### API - [Management API](/SpectraDetect/API/ManagementAPI/) — REST API for appliance management and automation ### Administration - [Manager Settings](/SpectraDetect/Admin/ManagerSettings/) — Spectra Detect Manager configuration reference - [Updating](/SpectraDetect/Admin/Updating/) — software update procedures - [Troubleshooting](/SpectraDetect/troubleshooting/) — common issues and solutions --- ## Spectra Intelligence Quick Start — First API Call in Minutes # Getting started with Spectra Intelligence This guide walks you through your first Spectra Intelligence API call — a file reputation lookup by hash. You will authenticate, send a request, and interpret the response. **What you'll accomplish:** - Obtain your API credentials - Make your first file reputation request - Parse and understand the response ## Prerequisites Before you begin: - A Spectra Intelligence account with API access - Your API credentials (username and password) - `curl` or Python 3 installed on your machine If you do not have an account, contact [ReversingLabs sales](https://www.reversinglabs.com/contact) to request access or an evaluation. ## Step 1: Get your API credentials Your Spectra Intelligence API credentials (username and password) are provided by [ReversingLabs Support](mailto:support@reversinglabs.com) when your account is activated. These credentials are used for HTTP Basic Authentication on all API requests. All API requests must be made over HTTPS to `data.reversinglabs.com`. ## Step 2: Make your first file reputation request The simplest API call is a file hash reputation lookup using the [TCA-0101](/SpectraIntelligence/API/FileThreatIntel/) endpoint. It accepts an MD5, SHA1, or SHA256 hash and returns a classification verdict instantly. **Using curl:** ```bash curl -u "your-username:your-password" \ "https://data.reversinglabs.com/api/databrowser/rl_data/query/sha1/list?format=json" \ -H "Content-Type: application/json" \ -d '{"rl": {"query": {"hash_type": "sha1", "hashes": ["3395856ce81f2b7382dee72602f798b642f14d45"]}}}' ``` **Using Python:** ```python username = "your-username" password = "your-password" hashes = ["3395856ce81f2b7382dee72602f798b642f14d45"] url = "https://data.reversinglabs.com/api/databrowser/rl_data/query/sha1/list?format=json" payload = {"rl": {"query": {"hash_type": "sha1", "hashes": hashes}}} response = requests.post(url, auth=(username, password), json=payload) print(response.json()) ``` You can also use the [ReversingLabs Python SDK](https://pypi.org/project/reversinglabs-sdk-py3/) for a higher-level interface to all Spectra Intelligence APIs. ## Step 3: Parse the response A successful response returns a JSON object with the file classification: ```json { "rl": { "entries": [ { "hash_type": "sha1", "hash_value": "3395856ce81f2b7382dee72602f798b642f14d45", "classification": { "classification": "MALICIOUS", "factor": 5, "scan_results": [] } } ] } } ``` **Key fields:** - `classification.classification` — `MALICIOUS`, `SUSPICIOUS`, `GOODWARE`, or `UNKNOWN` - `classification.factor` — Confidence level (1–5, where 5 is highest) - `scan_results` — Detections from individual AV engines If the hash is not in the database, the response returns `UNKNOWN`. ## Troubleshooting If your request fails, check the HTTP status code: - **401 Unauthorized** — Invalid credentials. Verify your username and password. - **403 Forbidden** — Your account does not have access to this endpoint. Contact support. - **429 Too Many Requests** — Daily or monthly quota exceeded. Daily quotas reset at 00:00 UTC. For a complete list of response codes and error handling, see [General Information](/SpectraIntelligence/#response-status-codes). ## Next steps Now that you've made your first API call, explore the full capabilities of Spectra Intelligence: - **[File Threat Intelligence API](/SpectraIntelligence/API/FileThreatIntel/)** — Full file analysis reports, not just reputation - **[Malware Hunting API](/SpectraIntelligence/API/MalwareHunting/)** — Search across the threat intelligence database - **[TAXII Feeds](/SpectraIntelligence/Feed/TAXII/tctf-0003/)** — Consume threat intelligence as structured feeds - **[OpenAPI Reference](/SpectraIntelligence/API-reference)** — Complete API specification - **[General Information](/SpectraIntelligence/)** — Limits, quotas, and technical reference --- ## Spectra Intelligence API — Authentication, Rate Limits & Endpoints # Spectra Intelligence API — Authentication, Rate Limits & Endpoints This section provides technical reference documentation for **Spectra Intelligence APIs and Feeds**. Each individual API reference page explains what the service does and documents available endpoints, supported query types, data formats, and response structures, along with service-specific details such as rate limits, required permissions, and field explanations. If you're looking for product overviews and datasheets, visit the [Spectra Intelligence product page](https://www.reversinglabs.com/products/spectra-intelligence). ## Introduction This section provides technical reference documentation for **Spectra Intelligence APIs and Feeds**. Each individual API reference page explains what the service does and documents available endpoints, supported query types, data formats, and response structures, along with service-specific details such as rate limits, required permissions, and field explanations. If you're looking for product overviews and datasheets, visit the [Spectra Intelligence product page](https://www.reversinglabs.com/products/spectra-intelligence). ## Spectra Intelligence API Reference Spectra Intelligence documentation is also available as an [OpenAPI Specification](/SpectraIntelligence/API-reference). ## Sending requests Requests addressed to Spectra Intelligence must use the *https* protocol. The server is `data.reversinglabs.com`. All APIs require the HTTP *Authorization* header with basic authentication. If not provided, the service will respond with status 401. ## Limits and quotas The limiting system counts the HTTP response code events for queries to ReversingLabs APIs. The limits are determined in customer contracts and are deducted only in these cases: **Single queries** – Only when HTTP code 200 is returned. **Bulk queries** - Applies only to services that support bulk queries. When HTTP code 200 is returned, the quota is deducted for each valid item sent within the bulk request. For example, a bulk request containing 100 hashes, 3 of which are invalid, will count as 97. **Next page queries** - Services that support pagination will count every successful next page query as a separate request. In case you reach the daily or monthly quota limit, the API will return HTTP code 429 and send an automated alert message to the previously provided contact email address. Daily quotas reset at 00:00 UTC, and monthly quotas reset on the first day of the month at 00:00 UTC. Customers can check their usage via the Customer Usage API or get in touch with the [Support Team](mailto:support@reversinglabs.com) to verify current API entitlements and quota limits. Use the [ReversingLabs Python SDK](https://pypi.org/project/reversinglabs-sdk-py3/) for easier programmatic interaction with the API. ## XML body The documentation provides request and response body examples in JSON, but all APIs also support XML request bodies. For **arrays in XML** , individual elements (such as objects or strings) will be wrapped inside `` tags in the response (and conversely, need to be wrapped inside `` in a request body). For example, the following JSON: ```json { "rl": { "entries": [ { "foo": "bar", "life_meaning": 42 } ] } } ``` ...should look like this in XML: ```xml bar 42 ``` ## Response status codes This table lists possible HTTP response codes, with descriptions. Individual APIs may have additional specific messages or codes. In those cases, see their documentation. Code | Description -----|--------------------------- 200 | The request has succeeded. 400 | The request could not be understood by the server due to malformed syntax. A possible reason might be that the request contains Unicode characters that cannot be processed. 401 | The request requires user authentication. The request MUST include a WWW-Authenticate header field containing a challenge applicable to the requested resource. 403 | The server understood the request, but is refusing to fulfill it. 404 | The server has not found anything matching the Request-URI. 405 | The method is known by the server but is not supported by the target resource. 409 | The request could not be completed due to a conflict with the current state of the resource. 413 | The request contained more than the maximum allowed amount of hashes (100). 429 | The daily/monthly quota has been reached, or the API rate limit exceeded (in this case, the service will return an additional "Rate limit exceeded" error message). 500 | The server encountered an unexpected condition which prevented it from fulfilling the request. 502 | The server received an invalid response from another server. 503 | The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. --- ## Getting started with File Inspection Engine This guide walks you through running File Inspection Engine (FIE) locally with Docker and scanning your first file via the HTTP API. **What you'll accomplish:** - Pull and run the FIE container - Verify the engine is ready - Submit your first file for analysis - Understand the classification response ## Prerequisites Before you begin: - Docker installed and running on your machine - A ReversingLabs license file for FIE (`.lic`) - Your ReversingLabs cloud credentials (username and password) for registry access - `curl` installed **Tip: Obtaining your license and credentials** Your FIE license file and cloud credentials are provided by [ReversingLabs Support](mailto:support@reversinglabs.com) when your FIE subscription is activated. If you do not have them, contact support before proceeding. ## Step 1: Pull the container image Log in to the ReversingLabs container registry using your cloud username and password, then pull the FIE image: ```bash docker login registry.reversinglabs.com docker pull registry.reversinglabs.com/fie/file-inspection-engine:latest ``` ## Step 2: Start the container Run FIE with your license passed as an environment variable: ```bash docker run -d \ --name fie \ -p 8000:8000 \ -e RL_LICENSE="$(cat /path/to/rl-license.lic)" \ registry.reversinglabs.com/fie/file-inspection-engine:latest ``` On first start, FIE downloads its threat database. Wait for the container to become ready: ```bash curl http://localhost:8000/readyz ``` When ready, this returns `200 OK`. The download may take a few minutes depending on your network speed. ## Step 3: Scan your first file Submit a file for analysis using the `/scan` endpoint: ```bash curl -X POST --upload-file /path/to/your/sample.exe \ http://localhost:8000/scan ``` FIE analyzes the file synchronously and returns a JSON verdict in the same response. ## Step 4: Interpret the response A typical response looks like: ```json { "classification": "OK", "message": "", "errors": [] } ``` The `classification` field indicates the verdict: | Value | Meaning | | --- | --- | | `OK` | File is goodware or unknown — no threat detected | | `malicious` | File is classified as malicious | | `suspicious` | File shows suspicious indicators (only when paranoid mode is enabled) | ### Getting additional threat details To get threat details alongside the classification verdict, start FIE with the `--with-threat-details` and `--add-file-type` flags. The enriched response includes the threat name, platform, and file type: ```json { "classification": "malicious", "message": "", "errors": [], "threat_details": { "platform": "Script", "type": "Trojan", "threat_name": "Script-JS.Trojan.Redirector" }, "file_type": "Text" } ``` ## Next steps Now that you've scanned your first file, explore the full capabilities of FIE: - **[Usage Guide](./usage.md)** — Complete API reference including all endpoints, hash lookups, status monitoring, error handling, and classification overrides - **[Configuration Reference](./configuration.md)** — CLI flags for core instances, timeouts, file size limits, and advanced options - **[Kubernetes Deployment](./Deployment/kubernetes.md)** — Deploy FIE at scale on Kubernetes - **[Air-Gapped Deployment](./Deployment/air-gapped-kubernetes.md)** — Offline environment setup For troubleshooting common issues or understanding response codes, see the [Usage Guide](./usage.md#possible-response-status-codes). --- ## File Inspection Engine File Inspection Engine (FIE) is a containerized file analysis service that performs synchronous, real-time scanning of files via an HTTP API. It is designed for integration into network security pipelines where files must be inspected inline — each request submits a file, waits for analysis to complete, and receives a verdict in the same response. FIE uses [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) for static file analysis, enabling deep inspection of over 400 file formats without executing files. It is deployed as an OCI-compliant container on Docker or Kubernetes and maintains a local threat database, so file content never leaves your infrastructure during scanning. ## Key capabilities - Synchronous HTTP API — submit a file, receive a classification verdict in one request - Containerized deployment on Docker or Kubernetes (no agent installation required) - Local threat database — all file analysis happens on-premises - Configurable Spectra Core instances for throughput scaling - Large file handling with dedicated core pools - Optional enrichment with cloud threat details via [Spectra Intelligence](/SpectraIntelligence/) hash lookups (hash only, no file upload) ## Privacy File Inspection Engine keeps all file data on-premises. Files submitted for scanning are processed locally using a bundled threat database and are not uploaded to external services. When the `--with-threat-details` option is enabled, FIE contacts Spectra Intelligence using the file hash only — the file itself is never transmitted. The threat database is updated on a regular schedule from ReversingLabs infrastructure. ## Deployment options Choose a deployment model based on your infrastructure: - [Deployment Overview and Hardware Requirements](./Deployment/) — capacity planning, Spectra Core instance sizing, and networking prerequisites - [Docker Deployment](./Deployment/docker.md) — standalone container setup for development and smaller deployments - [Kubernetes Deployment](./Deployment/kubernetes.md) — Helm chart installation for production Kubernetes clusters - [Air-Gapped Kubernetes](./Deployment/air-gapped-kubernetes.md) — deployment in offline or restricted network environments - [Helm Values Reference](./Deployment/Examples/values.md) — complete chart configuration reference ## Configuration FIE is configured via CLI flags and environment variables. Key settings include the number of Spectra Core instances, analysis timeouts, maximum file size, and network interface bindings. See the [Configuration Reference](./configuration.md) for all available options. ## Usage and API FIE exposes a REST API for file submission, result retrieval, status monitoring, and classification overrides. See the [Usage Guide](./usage.md) for API endpoint documentation, scanning workflows, response formats, and error handling. ## Related resources - [Spectra Core Analysis](/General/AnalysisAndClassification/SpectraCoreAnalysis) — how the underlying static analysis engine works - [Classification](/General/AnalysisAndClassification/Classification) — risk scores, threat levels, and classification methodology - [Platform Requirements](/General/DeploymentAndIntegration/PlatformRequirements) — hardware sizing across all ReversingLabs products --- ## Getting started with T1000 This guide walks you through authorizing the T1000 appliance, applying your license, creating an API user, and verifying the setup with a file reputation lookup. ## Prerequisites Before you begin: - T1000 appliance deployed and powered on (see [Deployment](./deployment.md)) - Network configured via the VM console (see [Management](./management.md)) - Access to a web browser on a machine that can reach the appliance IP - `curl` installed for API testing **Tip: Initial credentials** Your initial administrator username and password for the web management interface are provided by [ReversingLabs Support](mailto:support@reversinglabs.com). Contact support before proceeding if you do not have them. ## Step 1: Log in to the web management interface Open a browser and navigate to the appliance management interface on port 10000: ``` http://:10000 ``` Log in with the default credentials provided by [ReversingLabs Support](mailto:support@reversinglabs.com). On first login you are prompted to set a new password — do this before proceeding. **Note: The web management interface may be unresponsive for 10–60 minutes after each restart while the database completes internal verification. Wait for it to become available before continuing.** ## Step 2: Obtain your license (authorize the appliance) The T1000 appliance cannot retrieve database updates or respond to API requests until it is authorized. Authorization links the appliance to your ReversingLabs account. 1. In the web management interface, navigate to **RL Appliance > Authorization**. 2. Copy the values from the following fields: - **Appliance Type** - **Appliance ID** - **Appliance Key** - **Appliance Version** - **Appliance Username** - **Expiration Date** (shows "N/A" on an unlicensed appliance — include it anyway) 3. Send all of these values to [support@reversinglabs.com](mailto:support@reversinglabs.com) to request your authorization token. 4. When ReversingLabs Support responds with the token, paste it into the **Token** field on the Authorization page. 5. Select **Authorize**. 6. Restart the appliance after successful authorization (**RL Appliance > Dashboard > Reboot**). After restart, the appliance begins downloading the latest database updates from Spectra Intelligence. The Authorization page will show the license expiry date and the number of available updates. ## Step 3: Create an API user REST API access requires a dedicated user account. The default admin account cannot be used for API calls. 1. Navigate to **RL Appliance > User Management**. 2. Enter a username (alphanumeric only, must not be `admin`). 3. Select **Add User**. 4. Note the generated 8-character password shown in the **User info** section. Usernames always have the `u/` prefix — for example, if you enter `analyst`, the full username is `u/analyst`. Use this full prefixed form in all API requests. **Note: Passwords are auto-generated and cannot be manually set. Use the **Reset password** button to generate a new one if needed.** ## Step 4: Verify with an EICAR hash lookup Once the appliance has downloaded its initial database update, verify it is working by querying the EICAR test file hash — a well-known test sample that every threat intelligence database should classify as malicious. ```bash curl -u "u/:" \ "http:///api/databrowser/malware_presence/query/sha1/list?format=json" \ -H "Content-Type: application/json" \ -d '{"rl": {"query": {"hash_type": "sha1", "hashes": ["3395856ce81f2b7382dee72602f798b642f14d45"]}}}' ``` A successful response confirms the appliance is authorized, the database has loaded, and API access is working: ```json { "rl": { "malware_presence": { "entries": [ { "sha1": "3395856ce81f2b7382dee72602f798b642f14d45", "status": "MALICIOUS", "threat_level": 5, "classification": { "classification": "malware", "type": "Virus", "platform": "DOS", "family_name": "EICAR-Test-File" } } ] } } } ``` If the response returns `UNKNOWN` or an authentication error, see [Troubleshooting](#troubleshooting). ## Troubleshooting | Symptom | Likely cause | Action | |---|---|---| | Web interface unreachable on port 10000 | Database verification still in progress | Wait up to 60 minutes after restart | | Authorization page shows "N/A" for all fields | Appliance not yet networked | Configure network via VM console first | | EICAR returns `UNKNOWN` | Database update not yet complete | Wait for updates to finish; check update status on the Authorization page | | API returns `401` | Wrong username format or password | Ensure username includes the `u/` prefix | | API returns `429` | License expired | Re-authorize the appliance and contact [support@reversinglabs.com](mailto:support@reversinglabs.com) | ## Next steps - [Configuration](./configuration.md) — proxy settings, certificate management, REST API protocol - [Management](./management.md) — network settings, DNS, NTP, password reset via VM console - [File Threat Intelligence API](/SpectraIntelligence/API/FileThreatIntel/) — full API reference for hash lookups and file analysis reports --- ## T1000 # T1000 — Network Appliance for File Reputation ReversingLabs T1000 Appliance provides on-premises access to an up-to-date copy of ReversingLabs Spectra Intelligence, the industry's most comprehensive source for threat intelligence and data reputation files. With a local database, customers do not incur latency penalties and privacy risks associated with the Internet. The T1000 Appliance uses a NoSQL database optimized for data replication, and supports advanced searches across billions of file records in milliseconds. ## T1000 R1 and XG T1000 R1 and T1000 XG share the same core data model, but XG exposes additional sample metadata and analysis endpoints. While R1 supports only the TCA-0101 (Malware Presence) API, XG additionally provides TCA-0104 (File Analysis), TCA-0103 (Historic multi-AV scan records) and the XG-CFS forensic sampling service. **Info: The documentation for API endpoints titled TCA-XXXX is mirrored from ** [Spectra Intelligence](/SpectraIntelligence/API/FileThreatIntel). The functionality is equivalent in terms of requests: URL structure, request parameters, and so on. In terms of responses, certain information won't be present in T1000: - history - hashes that are not SHA256, SHA1 or MD5: - SHA384 - SHA512 - RIPEMD160 - scanner metadata: - version used for this scanning report - update timestamp Where such information is available with a direct call to Spectra Intelligence, T1000 will return `null`. When sending requests, use the username and the password created with the Appliance management interface. ### Response Status Codes | Code | Description | |----- | ------------------------------------------------------------------------------------------------------------------ | | 200 | The request has succeeded. | | 400 | The request could not be understood by the server due to malformed syntax. | | 401 | The request requires user authentication. | | 403 | The server understood the request, but is refusing to fulfill it. | | 404 | The server has not found anything matching the request URI. | | 429 | License has expired. | | 500 | The server encountered an unexpected condition which prevented it from fulfilling the request. | | 503 | The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. | --- ## Example values.yaml file ```yaml # Default values for fie. # This is a YAML-formatted file. # Declare variables to be passed into your templates. image: repository: registry.reversinglabs.com/fie/file-inspection-engine pullPolicy: IfNotPresent # Overrides the image tag whose default is the chart appVersion. tag: "" imagePullSecrets: {} # value not used for RL registry licenseFileContent: "" # FIE license received from ReversingLabs nameOverride: "" fullnameOverride: "" podAnnotations: {} podSecurityContext: {} securityContext: {} storage: existingPvcName: "" # set name to use existing pvc size: 32Gi # min. 22Gi atm. className: gp2 rlTmpInRam: true tmpfsSize: 20Gi service: annotations: {} # To set an internal load balancer, refer to your cloud service provider documentation. type: ClusterIP port: 8000 ingress: enabled: false className: "alb" annotations: {} # There are different features supported by various Ingress controllers. Please refer to # documentation on your platform specific Ingress controller to configure it in your environment. hosts: - host: fie.local.lan paths: - path: / pathType: Prefix tls: [] # - secretName: fie-tls # hosts: # - fie.local.lan resources: requests: cpu: 8 memory: 32Gi ephemeral-storage: 100Gi nodeSelector: {} tolerations: [] affinity: {} settings: # This option is available only for alpha6 and above. Possible values: enabled / disabled / force addFileType: "disabled" # Cloud account password, used only for default registry auth cloudPassword: "" # Desired frequency of cloud threat data updates, between 1 minute and 24 hours. Use m for minutes # and h for hours, e.g. 45m or 6h. cloudUpdateInterval: "5m0s" # Automatic updates of cloud threat data cloudUpdates: true # Cloud account username, used only for default registry auth cloudUsername: "" # Maximum concurrent requests performing file analysis, across all HTTP endpoints. Allowed values are from 0 (unlimited) to 100 concurrencyLimit: 20 # The address and port on which the HTTP server will listen. httpAddress: ":8000" # Files larger than this will be analyzed by large Spectra Core instances (0 to 10240 MiB). When 0, there is no distinction between instances. largeFileThreshold: 10 # Set the max decompression factor to limit resource usage during decompression, with 0 meaning no limit. maxDecompressionFactor: 1.0 # The value needs to be between 1 and 10240 MiB. Uploads larger than this will be rejected. maxUploadFileSize: 2048 # The number of Spectra Core instances that will process large files (0 to 100) numberOfLargeCores: 2 # The number of Spectra Core instances that will process regular files (1 to 100) numberOfRegularCores: 4 # Whether suspicious samples should be classified as malicious paranoidMode: false # cgroup v2 memory use percentage that triggers rejection of new file uploads. Allowed values are from 0 (disabled) to 100. processingUnavailableAtMemoryPercent: 0 # The host and port of a proxy for outgoing HTTP connections. It can optionally include one of the # following three schemes: http, https, socks5. Example: socks5://host:port proxyAddress: "" # Maximum analysis time, for example 10s (seconds) or 1m (minute). The default is 0, which means unlimited. timeout: "0" # The maximum number of file layers to unpack when performing static analysis. Valid values # are from 0 (unlimited) to MaxInt32. Default 17 unpackingDepth: 17 # Includes detailed information about malicious samples in the HTTP response withThreatDetails: false # Do not look up samples in the malicious threat data. The files don't need to be present locally either. withoutMaliciousThreatData: false ``` --- ## Air-Gapped Kubernetes Deployment If the network topology of a Kubernetes cluster prevents access to the ReversingLabs registry and APIs, several objects must be manually transferred and uploaded to the cluster. It is crucial to have Kubernetes API access available since `kubectl` will be used throughout this process. **Steps:** - **Download the threat data manually**: Use a FIE instance with internet access to download threat data. - **Deploy FIE in production**: Deploy the production FIE application with cloud updates disabled - **Transfer threat data**: Copy the downloaded threat data to the air-gapped FIE instance. To complete this process, you will need: - The [FIE Helm Chart](./kubernetes.md#appendix-fie-helm-chart) - The FIE container image [pulled from the ReversingLabs registry](./docker.md#pulling-the-docker-image), which must be made available to the Kubernetes cluster via a client-provided registry. - A valid license (provided by ReversingLabs). ### Manually download the threat data The detailed process for downloading threat data is available [here](./docker.md#manual-threat-data-synchronization). ### Deploy the FIE application After the threat data downloads, deploy FIE using the [Helm chart](./kubernetes.md#appendix-fie-helm-chart). #### Making the container image available to Kubernetes To make the FIE container image available to the Kubernetes cluster, you need to [pull it from the ReversingLabs registry](./docker.md#pulling-the-docker-image) and push it to your own registry. Follow these steps: 1. **Load the image**2. **Tag the image**3. **Push the image to your registry****Note: Podman is used in this example, but the syntax should be similar if using Docker.** #### Installing FIE Using Helm Prepare a custom values file to configure the deployment using the [FIE Helm Chart](./kubernetes.md#install-fie-using-helm). Consult with your Kubernetes administrator to decide how to expose the FIE service (e.g., LoadBalancer, Ingress). For the full list of available values, see the [example values.yaml](./Examples/values.md) file. In this example, we use a LoadBalancer service to expose FIE, and we override the default image repository and tag with the settings from the previous step. **Example Configuration (configuration.yaml):**Once you have prepared the values file, you can proceed to install the Helm chart. The Helm chart can be pushed to a chart repository, an OCI repository, or used directly as shown below: **Example Helm installation command** ```bash helm install fie ./fie-0.2.1.tgz --create-namespace --namespace fie-gapped \ --set settings.cloudPassword="$RL_CLOUD_PASSWORD" \ --values configuration.yaml --set-file licenseFileContent=RL-license.enc ``` ```yaml NAME: fie LAST DEPLOYED: Mon Aug 26 11:57:56 2024 NAMESPACE: fie-ag STATUS: deployed REVISION: 1 TEST SUITE: None ``` **Note: CPU Requests: The `RL_CPU_REQUEST` environment variable exposes the number of CPU requests configured in Kubernetes (`resources.requests.cpu`) to the FIE container.** FIE uses this value to report instance usage in `/status` as a percentage relative to the allotted CPU. - **If deploying with our Helm chart**: the chart template maps `resources.requests.cpu` into this variable (see the [Kubernetes deployment guide](./kubernetes.md#cpu-requests)). - **If deploying without Helm**: define `RL_CPU_REQUEST` in the container's environment section of your Pod or Deployment manifest. ### Copy the threat data There are multiple ways to transfer the threat data to the air-gapped environment. Below is one example workflow: 1. **Download the tar package** We will store the threat data into a .tar file. This requires the `tar` package to be installed in the FIE pod. Since this is an air-gapped environment, the `tar` package must be downloaded externally and then transferred to the pod: ```bash curl -O https://cdn-ubi.redhat.com/content/public/ubi/dist/ubi8/8/x86_64/baseos/os/Packages/t/tar-1.30-9.el8.x86_64.rpm ``` This is an example command, make sure to check that you're downloading the latest available version. 2. **Upload the tar package to FIE and install it** ```bash cat tar-1.30-9.el8.x86_64.rpm | kubectl -n fie-gapped exec -it deploy/fie -- cp /dev/stdin /tar-1.30-9.el8.x86_64.rpm ``` ```bash kubectl -n fie-gapped exec -it deploy/fie -- rpm -ihv /tar-1.30-9.el8.x86_64.rpm ``` 3. **Store and transfer threat data** Once `tar` installs, threat data can be stored into a .tar archive and moved over to the pod: ```bash $ cd /external/dir $ tar cvf - * | kubectl -n fie-gapped exec -i deploy/fie -- tar xf - -C /rl/threat-data --no-same-owner ``` 4. **Restart the pod** After everything is installed and copied over, restart the pod: ```bash kubectl -n fie-gapped rollout restart deploy/fie ``` Contact [ReversingLabs Support](mailto:support@reversinglabs.com) for more information and guidance. --- ### Get the application URL To confirm that the File Inspection Engine is up and running, retrieve the application URL and perform a test file submission. You can follow the steps provided in the [Kubernetes Deployment guide](./kubernetes.md#get-the-application-url). --- ## Docker Deployment — File Inspection Engine ## Docker image The File Inspection Engine Docker image can be obtained from the ReversingLabs container registry. ### Pulling the Docker image To pull the Docker image from the ReversingLabs container registry: 1. **Log in to the Docker Registry** Log in using your cloud username and password: ```bash docker login registry.reversinglabs.com ``` 2. **Pull the Docker image** Pull the `file-inspection-engine` Docker image with the specified tag:## Running the application The File Inspection Engine (FIE) reads its license from an environment variable called `RL_LICENSE`. This license, provided by ReversingLabs, must be passed to the application at startup. To start the application, use the following commands.In this example, the container runs on the host network, so no port mapping is needed. **If you're not using the host network, you'll need to map the container's port to the host.** The HTTP server uses port 8000 by default, but you can change it: - To map the port to a different host port:- To change the HTTP port used by the container:## Storage and mounting considerations FIE uses two directories inside the container for storage: 1. `/rl/threat-data`, which it uses to assign file classifications. 2. `/rl/tmp`, which it uses to store file uploads, unpacked files, and file analysis reports. The `/rl/threat-data` directory contains roughly 20 GiB for malicious data and 1 GiB for suspicious data, and additional space is needed during updates, as files are downloaded fully before replacement. Threat data synchronization starts shortly after the application is up and running and continues at regular intervals, configurable via the `--cloud-update-interval` parameter. Initial synchronization involves larger files, while subsequent updates use incremental changes (typically < 100 KB per segment). Data is divided into 256 segments per classification, and each segment may require multiple updates, which can increase the total download size to several megabytes, especially with less frequent updates. This means that a container started "bare" - without any threat data mounted upon start - will first need to pull in around 20 GiB of data, every time it is started. This radically decreases the performance of FIE, so **mounting an external volume is essential**, for example:This allows reusing threat data between containers, for example by transferring it to an [air-gapped instance](#air-gapped-manual-threat-data-synchronization). Mounting an external volume also means that you avoid the [performance costs](https://docs.docker.com/engine/storage/drivers/#copying-makes-containers-efficient) associated with writing to disk inside the container. **Warning: Reusing threat data must be done in **read-only** mode.** Individual FIE containers will, by default, continuously monitor and update their `/rl/threat-data` directory. Reusing threat data between several containers can lead to an issue with how containers interact with that directory. Therefore, make sure that containers which reuse the same source of threat data **do not write to it**. This can be accomplished by turning off [cloud updates](../../configuration/#--cloud-updates--rl_cloud_updates) for all containers which reuse the same data. **Note**: Do not reuse threat data even when only one container is writing. Even in this case, the read-only containers could potentially use old data *while it is being updated*. Since containers are only aware of their own threat data updates, they cannot detect another container being in the middle of an update. ### Selecting the mount type The two main factors to consider when choosing a mount type are **persistence** and **speed**. You want the `/rl/threat-data` directory to be persistent and have good read speed (as that's where the application will look when classifying files), and you want good write speed for `/rl/tmp`. If you're working directly with the threat data (as described in the [air-gapped instance](#air-gapped-manual-threat-data-synchronization) section), select a regular [bind mount](https://docs.docker.com/engine/storage/bind-mounts/). This allows you to freely interact with the downloaded data from the host system. You could also select a [Docker volume](https://docs.docker.com/engine/storage/volumes/#when-to-use-volumes) if you need a persistent source of data, but do not intend to directly interact with it. For `/rl/tmp`, persistence is not important, but write speed is. A possible choice here is [tmpfs mounts](https://docs.docker.com/engine/storage/tmpfs/). This also allows the highest throughput, as the underlying static analysis engine performs a lot of disk writes, and `tmpfs` mounts are RAM-only - which means that the write speeds will be faster. Note, however, that this requires allocating more RAM than e.g. using a bind mount. ## Manual threat data synchronization The File Inspection engine retrieves updates automatically. If you want to pre-download threat data so your customers can start using it immediately, or if you prefer to manually sync the data, use the `threat-data` command included in the image. This command is also used to [download threat data in air-gapped environments](#air-gapped-manual-threat-data-synchronization). If manual threat data updates occur less than once per week, incremental updates may take longer than a full database download. Performance depends on system resources, network bandwidth, and the deployment environment. Incremental updates are recommended by default, but if they are slow, consider these factors and opt for a full download if necessary. ### Supported Options The `threat-data` command supports the following options in addition to username and password: - `RL_PARANOID_MODE` Download data collection for suspicious files. - `RL_PROXY_ADDRESS` Specify a proxy server address if you need to connect to the cloud via a proxy. - `RL_RETRY_COUNT` The number of retries if a segment fails to download during update. - `RL_LOG_JSON` Defines the log output format as either JSON or colored plain text. ### Sync Command To manually sync the threat data, use the `sync` sub-command, which requires specifying the threat data directory: ```bash ./threat-data sync /threat/data/dir ``` To execute this via Docker, run:If you need to treat suspicious files as malicious, make sure to set the `RL_PARANOID_MODE` option to `true` in the command. **Important**: - The `threat-data` command only supports configuration via environment variables. - We recommend pre-downloading the threat data once and including it in your distribution for multiple users, as a full threat data download is more resource-intensive compared to incremental updates. - **Do not** run the `threat-data` command concurrently with the application if both are accessing the same directory. - **Always use the `threat-data` binary from the same FIE version** as your running container. Older binaries are not compatible with newer threat databases. Extract the binary from the container if needed: `docker cp :/rl/app/threat-data ./threat-data` ## Air-gapped manual threat data synchronization For air-gapped environments, follow the process below to synchronize threat data. First, download the threat data on a machine with internet access, then transfer the data to the air-gapped instance. 1. Start a File Inspection Engine (FIE) instance on a machine with internet access. Once the data sync is complete, stop the FIE instance that was used for downloading, and then proceed to step 2. Alternatively, run the following command to manually sync the threat data:`/external/dir` represents the path on the host system where the threat data is stored. If the directory contains older threat data, it will be incrementally updated. **Note: If using paranoid mode, set the environment variable `RL_PARANOID_MODE=true`.** Upon successful synchronization, the log should show `Threat data fully updated`. In case of errors, rerun the command to retry. Proceed to step 2. 2. Stop a production FIE instance (or create a new one) in the air-gapped environment. 3. Copy the threat data from `/external/dir` on the internet-connected machine to the corresponding threat data directory used by the air-gapped FIE instance. Ensure that the transferred data is placed in the directory where the application would normally download it if it were online. For further assistance, contact [ReversingLabs Support](mailto:support@reversinglabs.com). 4. Restart or deploy the air-gapped FIE instance with the updated threat intelligence data. --- ## Deploying File Inspection Engine: Requirements & Architecture ## Hardware Requirements To handle files of up to 2 GB, we recommend the following: - **Memory (RAM)**: - Provision **at least 32 GB of RAM**. File processing may require up to 8 times the file size in RAM, especially to accommodate large file handling and concurrent requests. - **Disk Size**: - Allocate **at least 100 GB of disk space** to support scanning of larger files and threat intelligence database storage requirements. ### Production Deployment A single deployment can handle files of all sizes using multiple Spectra Core instances. - The total number of instances is configured with `--number-of-regular-cores` and `--number-of-large-cores`. - All instances are identical. The "large" group is simply a reserved subset of instances that only process files above the configured threshold (`--large-file-threshold`). - If no large instances are configured (set to 0), all files are processed by the regular pool. Each Spectra Core instance consumes memory even when idle (approximately 1.4 GB per instance), so factor this into your capacity planning. This approach improves CPU utilization and throughput while keeping the deployment architecture simple. --- ## Kubernetes Deployment — File Inspection Engine ## Introduction A typical File Inspection Engine (FIE) installation is performed on Kubernetes using Helm. Throughout this document, we'll be using Google Kubernetes Engine (GKE) as an example. For managed Kubernetes solutions, you may also need to use vendor-specific tools to interact with the cluster. In our example, this will be `gcloud`. To install `gcloud`, follow [these steps](https://cloud.google.com/sdk/docs/install). ------ ## Deploying FIE Helm Chart to GKE Here is an overview of deploying the FIE Helm chart to a GKE cluster: ### Prerequisites - A GKE cluster is available. - `kubectl` is configured to work with your cluster. - Helm is installed. ------ ### Example: Configuring `kubectl` for a Specific Cluster 1. **List Available GKE Clusters:** ```bash gcloud container clusters list ``` **Example Output:** ```bash NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS gke-autopilot-ado-dev us-east4 1.28.8-gke.1095000 35.199.55.139 e2-small 1.28.8-gke.1095000 2 RUNNING ``` 2. **Get Cluster Credentials:** Run the following command to fetch cluster endpoint and authentication data: ``` gcloud container clusters get-credentials gke-autopilot-ado-dev --region us-east4 ``` **Output:** ```bash Fetching cluster endpoint and auth data. kubeconfig entry generated for gke-autopilot-ado-dev. ``` ------ The FIE Helm chart requires valid Spectra Intelligence credentials, which will be provided by ReversingLabs. ## Install FIE Using Helm The examples provided use a placeholder account (`u/example/fie`). Be sure to replace this with your actual credentials wherever applicable. ### Customize the Installation with a Values File For the full list of available values, see the [example values.yaml](./Examples/values.md) file. You can modify values such as ingress or storage class according to your needs. This example exposes the application internally using a load balancer service. ### Set the password and install the Helm Chart 1. Store the password in a variable: ```bash read -rs SPECTRA_INTELLIGENCE_PASSWORD ``` 2. Log in to the ReversingLabs container registry: ```bash echo "${SPECTRA_INTELLIGENCE_PASSWORD}" | helm registry login -u "u/example/fie" --password-stdin registry.reversinglabs.com ``` 3. Install the Helm chart: ```bash $ helm install fie oci://registry.reversinglabs.com/fie/charts/fie \ --create-namespace --namespace fie \ --set settings.cloudPassword="${SPECTRA_INTELLIGENCE_PASSWORD}" \ --values values-deploy-example-gcp.yaml \ --set-file licenseFileContent=rl-license.enc ``` **Expected Output:** ```bash Pulled: registry.reversinglabs.com/fie/charts/fie:0.2.1 Digest: sha256:61ed7f0761912cc5052ceac1d71654f3c1f89f543df0ab6ae3d199070ab02084 NAME: fie LAST DEPLOYED: Tue May 28 11:20:34 2024 NAMESPACE: fie STATUS: deployed REVISION: 1 TEST SUITE: None ``` ### CPU Requests The `/status` endpoint shows both counts (`available_*`) and percentages (`percentage_*`) for Spectra Core instances. The percentages are based on the value of the `RL_CPU_REQUEST` environment variable. - **Helm deployments**: Helm automatically maps `resources.requests.cpu` into this variable. No extra configuration is needed. - **Non-Helm deployments**: You must set `RL_CPU_REQUEST` yourself. This does not affect how many Spectra Core instances are created or how Kubernetes schedules the pod. It only affects how percentages are reported in `/status`. You can provide this value in three ways: 1. **Command-line flag** ```yaml args: ["--cpu-request=8"] ``` 2. **Environment variable** ```yaml env: - name: RL_CPU_REQUEST value: "8" ``` 3. **Kubernetes Downward API** ```yaml resources: requests: cpu: "8" env: - name: RL_CPU_REQUEST valueFrom: resourceFieldRef: containerName: fie resource: requests.cpu ``` In this last example, the Pod requests 8 CPUs, and Kubernetes injects that value into the container as `RL_CPU_REQUEST`. FIE then uses it only for calculating the `percentage_*` fields in `/status`. ------ ### Get the application URL After deployment, obtain the application URL and port by running one of the following commands: 1. **LoadBalancer IP:** > **Note:** It may take a few minutes for the LoadBalancer IP to be available. You can watch the status by running `kubectl get --namespace fie-ag svc -w fie`. ```bash export SERVICE_IP=$(kubectl get svc --namespace fie fie \ --template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}") echo http://$SERVICE_IP:8000 ``` 2. **Verify the deployment** ```bash kubectl -n fie get svc/fie ``` **Expected output** ```bash NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE fie LoadBalancer 34.118.234.76 10.128.0.24 8000:32132/TCP 32m ``` --- Once you have the service IP and port, you can send a test query to the liveness/readiness endpoints: ```bash curl -v http://10.128.0.24:8000/livez curl -v http://10.128.0.24:8000/readyz ``` **Note: Check the [Usage](/docs/FileInspectionEngine/usage.md#check-application-liveness) section for more information on these two endpoints.** --- Alternatively, submit a file for analysis. This only works after the [threat data is fully downloaded](#monitoring-the-threat-data-download): ```bash curl -sS -X POST --upload-file eicar.com http://10.128.0.24:8000/scan | jq ``` **Expected output** ```json { "classification": "malicious", "message": "" } ``` ------ ### Monitoring the Threat Data Download After deployment, the FIE application will begin downloading threat data. This process can take between 30 and 90 minutes, depending on your network speed. You can monitor the download process by running: ```bash kubectl -n fie logs -f deploy/fie ``` Look for the following message, which indicates that the download process completed successfully: ```json {"level":"info","component":"threatdata.UpdateManager","time":"2024-09-18T22:32:58.346353125Z","message":"Cloud update run finished"} ``` ## Appendix: FIE Helm Chart ### Download the Helm Chart To download the Helm Chart, run the following commands: ```bash helm registry login -u "$RL_CLOUD_USERNAME" -p "$RL_CLOUD_PASSWORD" registry.reversinglabs.com helm pull oci://registry.reversinglabs.com/fie/charts/fie ``` --- ## File Inspection Engine Configuration Reference The `--help` flag output lists all available command-line options along with their default values.**Note that default values may be overridden by environment variables.** - **Boolean flags** must include an equals sign (=) when explicitly set to `true`/`1` or `false`/`0` (e.g., `--cloud-updates=true`, `--cloud-updates=false`). Alternatively, to enable a boolean flag, use the shortcut by specifying the flag name alone (e.g., `--cloud-updates` for `true`). Non-boolean flags don't need an equals sign. Both `--timeout 10s` and `--timeout=10s` are fine. - **Time Duration Options:** For configuration options containing time durations, the supported suffixes are `ms`, `s`, `m`, and `h` (e.g., `--timeout 10s` or `--cloud-update-interval 1m`). **Info: **Environment variables:** Command line flags can also be passed as environment variables, by using `RL_` as the prefix and replacing the dashes in between the words with underscores. For example, `--http-address` becomes the environment variable `RL_HTTP_ADDRESS`.** When a default value is not mentioned in the help output, it is empty (either an empty string or `false` for boolean options). ## Configuration options ### `RL_LICENSE` - **Description**: Set the contents of your license file. - **Default**: N/A - **Notes**: This option is **only** available as an environment variable. --- ### `RL_RETRY_COUNT` - **Description:** Configures the maximum number of retries for failed threat data segment downloads when using the `threat-data` command. - **Default:** 3 - **Possible Values:** 0 to 100 --- ### `--add-file-type` / `RL_ADD_FILE_TYPE` - **Description**: Controls whether `file_type` information is returned in the [`/scan` API response](./usage.md#file-submissions). - **Default**: `disabled` - **Possible Values**: `disabled`, `enabled`, `force` - **Notes**: - When `enabled`, the correct `file_type` will only be returned if static analysis was performed. - When `force` is set, static analysis is always performed. --- ### `--concurrency-limit` / `RL_CONCURRENCY_LIMIT` - **Description**: Maximum concurrent requests when performing file analysis, across all HTTP endpoints. - **Default**: 20 - **Possible Values**: From `0` (unlimited) to `100` - **Notes**: Even if the limit is set to 0 (unlimited), the system will still track the number of active concurrent requests. The `active_concurrency` field will always reflect the actual number of concurrent requests, regardless of the limit setting. The `active_concurrency` field is logged in the format: `active_concurrency={value}/{limit}`. --- ### `--cpu-request` / `RL_CPU_REQUEST` - **Description**: Informs the application how many CPUs were requested for the container. This value does not control how many Spectra Core instances are created. Those are configured explicitly with `--number-of-regular-cores` and `--number-of-large-cores`. Instead, it is used only for reporting in the `/status` endpoint. - **Default**: Not set (FIE will use the total number of CPUs detected on the node). - **Notes**: - When set, the `percentage_*` fields in the `/status` endpoint are calculated relative to this value. - The `available_*` fields show counts of available instances and are not affected by this value. - This option is most useful in Kubernetes, where you typically specify `resources.requests.cpu`. Docker does not have a concept of CPU requests. - You can provide the value in three ways: 1. Command-line flag: ```yaml args: ["--cpu-request=8"] ``` 2. Environment variable: ```yaml env: - name: RL_CPU_REQUEST value: "8" ``` 3. Kubernetes Downward API (avoids duplicating the number): ```yaml env: - name: RL_CPU_REQUEST valueFrom: resourceFieldRef: containerName: fie resource: requests.cpu ``` --- ### `--proxy-address` / `RL_PROXY_ADDRESS` - **Description**: Specifies the address of a proxy server for contacting the cloud API. - **Default**: N/A - **Possible Values**: - `https://host:port` - `http://host:port` - `socks5://host:port` - **Notes**: You can include credentials in the proxy URL, for example: - `http://user:password@localhost:8080` --- ### `--cloud-update-interval` / `RL_CLOUD_UPDATE_INTERVAL` - **Description**: Sets how frequently the application checks for cloud threat data updates. - **Default**: `5m` - **Possible Values**: From `1m` to `24h` (e.g., `45m`, `6h`) --- ### `--cloud-updates` / `RL_CLOUD_UPDATES` - **Description**: Enables or disables automatic updates for threat data. Cloud updates are automatically disabled when `--without-malicious-threat-data` is set to `true` and `--paranoid-mode` is set to `false`, as threat data is not used in that case. - **Default**: `true` - **Possible Values**: `true`, `false` --- ### `--http-address` / `RL_HTTP_ADDRESS` - **Description**: Defines the host and port for the HTTP server. - **Default**: :8000 - **Possible Values**: - Port only. Example: `:9000` - Host and port. Example: `127.0.0.1:8080` --- ### `--log-json` / `RL_LOG_JSON` - **Description**: Defines the log output format as either JSON or colored plain text. - **Default**: `true` - **Possible Values**: `true`, `false` --- ### `--max-decompression-factor` / `RL_MAX_DECOMPRESSION_FACTOR` - **Description:** Spectra Core has a set of mechanisms that protect the user from intentional or unintentional archive bombs, ranging from checks that prevent a file from making identical copies of itself during unpacking, to the maximum allowed decompression ratio for any given file. These protection measures enable the engine to terminate the archive decompression if the size of unpacked content exceeds a set quota. The maximum decompression ratio is calculated as ``` MaximumDecompressionFactor * (1000 / ln(1 + InputFileSize * pow(10, -5))) ``` where `InputFileSize` must be in bytes. To calculate the maximum decompressed file size, multiply this ratio by the `InputFileSize`. In practice, this means that the unpacking will stop once the size of all extracted content exceeds the theoretical maximum of the best performing compression algorithm. - **Default**: 1.0 - **Notes:** When a file exceeds the decompression ratio, the unpacking will stop and the partially unpacked content will be sent for analysis. If set to a negative value, a warning is printed, and the value defaults to 1.0. Setting this to 0 disables decompression management, but this is strongly discouraged as it leaves the system vulnerable to resource exhaustion attacks. --- ### `--max-upload-file-size` / `RL_MAX_UPLOAD_FILE_SIZE` - **Description**: Maximum file size (in MiB) the application will accept. - **Default**: 100 - **Minimum**: 1 - **Maximum**: 10240 --- ### `--number-of-regular-cores` / `RL_NUMBER_OF_REGULAR_CORES` - **Description**: Configures how many Spectra Core instances are allocated to handle files up to the size threshold (`--large-file-threshold`). - **Default**: 4 - **Possible Values**: 1-100 --- ### `--number-of-large-cores` / `RL_NUMBER_OF_LARGE_CORES` - **Description**: Configures how many Spectra Core instances are reserved for files larger than the size threshold (`--large-file-threshold`). - **Default**: 2 - **Possible Values**: 0-100 - **Notes**: If set to 0, no instances are reserved for large files, and all files are processed by the pool of "regular" instances. --- ### `--large-file-threshold` / `RL_LARGE_FILE_THRESHOLD` - **Description**: File size threshold (in MiB) that determines when a file is routed to the reserved large-file instances. - **Default**: 10 - **Possible Values**: 0-10240 - **Routing rules**: - Files **larger than** the threshold go to the large-file instances. - Files **equal to or smaller than** the threshold stay in the regular pool. - **Notes**: - When set to 0, size-based routing is disabled and all files are distributed across available instances. In this case, the system routes files to the instance with the fewest active analyses, rather than using file size. - File size is only an approximation of processing cost. Real resource usage depends on file complexity (number of unpacked children, nesting depth). Choosing an optimal threshold and timeout may require experimentation based on your workload. --- ### `--paranoid-mode` / `RL_PARANOID_MODE` - **Description**: Enables an additional classification for suspicious files, allowing them to be flagged as `suspicious` instead of `OK`. With this option, the possible response classifications are `OK`, `malicious` (if malicious threat data is not disabled), and `suspicious`. - **Default**: `false` - **Possible Values**: `true`, `false` - **Notes**: - Requires an additional 1 GiB of cloud threat data for suspicious classification. - When malicious or suspicious threat data is enabled, goodware classification is automatically enabled as well and requires 64 MiB of threat data. Goodware classification cannot be directly enabled or disabled. --- ### `--processing-unavailable-at-memory-percent` / `RL_PROCESSING_UNAVAILABLE_AT_MEMORY_PERCENT` - **Description**: Defines the memory usage threshold (in percentage) at which the application will reject new file uploads and return an error on the `/readyz` endpoint. This helps prevent overloading the system when memory usage is high. For example, to reject uploads once memory usage reaches 80%, use: `--processing-unavailable-at-memory-percent=80`. - **Default**: 0 (disabled) - **Possible Values**: 0–100 - **Notes**: The threshold is based on `cgroup v2` memory usage within the container. If your system doesn't support `cgroup v2`, you can disable this feature by setting the parameter to `0`. --- ### `--with-threat-details` / `RL_WITH_THREAT_DETAILS` - **Description**: Determines whether detailed threat information is included in the JSON HTTP response for malware classification. - **Default**: `false` - **Possible Values**: `true`, `false` - **Notes**: Slows down the response as it contacts the cloud API by submitting the file hash to Spectra Intelligence. If no additional threat information is available, the `threat_details` property won't be present. --- ### `--unpacking-depth` / `RL_UNPACKING_DEPTH` - **Description**: The maximum number of file layers to unpack when performing static analysis. - **Default:** `17` - **Possible values**: From `0` (unlimited) to MaxInt32. --- ### `--timeout` / `RL_TIMEOUT` - **Description**: Configures the timeout limit for file analysis, in seconds. The countdown starts when a Spectra Core instance begins processing a file. - **Default**: 0 (unlimited) - **Examples**: `--timeout=30s`, `--timeout=5m`, `--timeout=1h` - **Notes**: - When the timeout is reached, the Spectra Core instance is terminated and restarted. - If the instance was processing multiple files, all analyses are aborted. - Logs contain information about which files were impacted. - Before restart, the instance cleans up its temporary files. Restart time depends on the number of files and disk performance, but typically takes a few seconds. - Because restart takes time, very short timeout values are not recommended. --- ### `--without-malicious-threat-data` / `RL_WITHOUT_MALICIOUS_THREAT_DATA` - **Description**: Allows the application to run without downloading malicious threat data. When enabled, malicious threat data updates are disabled. If `--paranoid-mode` is also enabled, suspicious threat data will still be downloaded. When both malicious and suspicious threat data are disabled, files are classified based purely on static analysis. - **Default**: `false` - **Possible Values**: `true`, `false` - **Notes**: When malicious or suspicious threat data is enabled, goodware classification is automatically enabled as well and requires 64 MiB of threat data. Goodware classification cannot be directly enabled or disabled. --- **Example - Running with proxy and additional settings**--- ## File Inspection Engine Troubleshooting Guide # Troubleshooting This guide covers common issues with [File Inspection Engine](./index.md) (FIE) and the steps to resolve them. --- ## Container exits immediately on startup **Symptom** The FIE container starts and exits within a few seconds. `docker ps` shows it in an `Exited` state. The container never becomes ready to accept requests. **Cause** - A fatal configuration error occurred during initialization, such as a missing or malformed `RL_LICENSE` environment variable. - A required CLI flag has an invalid value (for example, an invalid duration format for `--timeout`). - The container cannot bind to the configured HTTP port because it is already in use. - The static analysis engine (Spectra Core) failed to initialize due to insufficient resources. **Solution** 1. Inspect the container logs immediately after exit: ```bash docker logs ``` or for a Kubernetes pod: ```bash kubectl logs --previous -n ``` 2. Look for startup error messages. A missing license produces: ``` FATAL: License validation failed ``` Confirm that the `RL_LICENSE` environment variable is set and contains the full license file content: ```bash docker run -e RL_LICENSE="$(cat /path/to/license.lic)" \ registry.reversinglabs.com/fie/file-inspection-engine: ``` 3. Check for port conflicts if the log shows a bind error: ``` Error: listen tcp :8000: bind: address already in use ``` Change the host port mapping or stop the process occupying the port: ```bash docker run -p 9000:8000 ... # Map to a different host port ``` 4. Review the [Configuration Reference](./configuration.md) to verify that all provided flags use correct formats. Boolean flags require `=true` or `=false` when explicitly set (for example, `--cloud-updates=false`, not `--cloud-updates false`). 5. For Kubernetes deployments, check the Helm values for misconfigured environment variables or resource limits that are too low. See the [Helm Values Reference](./Deployment/Examples/values.md). --- ## `/readyz` returns 503 — not ready **Symptom** After starting the container, polling the readiness endpoint returns a non-200 status: ```bash curl http://localhost:8000/readyz # Returns 503 or another 5xx/4xx status ``` The container is running but not accepting file submissions. **Cause** - Threat data has not finished downloading. FIE requires the threat database to be available before it becomes ready, as described in [Starting the File Inspection Engine](./usage.md#starting-the-file-inspection-engine). - All Spectra Core instances are currently busy (high load or concurrency limit reached). - The license is invalid or has expired, preventing the engine from completing initialization. **Solution** 1. Check the container logs for readiness-related messages: ```bash docker logs -f ``` On first startup, you will see threat data download progress. Wait for the download to complete. The container logs `Instance is ready` when at least one analysis instance is available: ```json {"level":"info","process":"fie","instance_id":"core-regular-0.abc12","message":"Instance is ready"} ``` 2. Check the `/status` endpoint for more detail on the current state: ```bash curl http://localhost:8000/status ``` Review the `license.valid_until` field and the `spectra_core.available_regular_cores` field. If `available_regular_cores` shows `0/N`, all instances are busy or failed to initialize. 3. If the license has expired, update the `RL_LICENSE` environment variable and restart the container. See [License validation error on startup](#license-validation-error-on-startup). 4. For the `/readyz` endpoint behavior when under load, see [Request Rejection](./usage.md#request-rejection). The endpoint returns a non-200 status when memory or concurrency limits are exceeded — this is expected behavior, not a fault. --- ## Threat database download fails or is slow **Symptom** The container starts but remains in a not-ready state for a long time. Logs show repeated download failures or slow progress: ```json {"level":"warn","process":"fie","message":"Failed to download threat data segment, retrying"} ``` Or the threat data download does not start at all. **Cause** - Outbound HTTPS connectivity from the container to the ReversingLabs update infrastructure is blocked by a firewall or requires a proxy. - The license does not include the appropriate threat data entitlement. - The `--without-malicious-threat-data` flag is not set, but network access to the update server is unavailable. - The container's DNS is not resolving the update server hostname. **Solution** 1. Test outbound connectivity from inside the container: ```bash docker exec curl -I https://updates.reversinglabs.com ``` If this fails, the container cannot reach the update server. Check firewall egress rules and ensure that outbound HTTPS (port 443) is permitted. 2. If a proxy is required, configure it using the `--proxy-address` flag or the `RL_PROXY_ADDRESS` environment variable: ```bash docker run -e RL_PROXY_ADDRESS="http://proxy.company.internal:8080" \ -e RL_LICENSE="..." \ registry.reversinglabs.com/fie/file-inspection-engine: ``` 3. For air-gapped environments, use the offline threat data download process. See [Air-Gapped Kubernetes Deployment](./Deployment/air-gapped-kubernetes.md) for the procedure to pre-load threat data without internet connectivity. 4. The `RL_RETRY_COUNT` environment variable controls how many times FIE retries failed segment downloads (default: 3). For flaky connections, increase this value: ```bash docker run -e RL_RETRY_COUNT=10 ... ``` 5. If you want to run FIE without downloading malicious threat data (relying on static analysis only), set `--without-malicious-threat-data=true`. See the [Configuration Reference](./configuration.md) for the implications of this option. --- ## Analysis returns 503 Service Unavailable **Symptom** `POST /scan` requests return: ```http HTTP/1.1 503 Service Unavailable ``` or: ```http HTTP/1.1 429 Too Many Requests {"error":"The concurrency limit has been reached"} ``` or: ```http HTTP/1.1 429 Too Many Requests {"error":"Analysis not accepted due to high processing load"} ``` **Cause** - All Spectra Core instances are busy processing other files (high load). - The concurrency limit configured with `--concurrency-limit` has been reached. - Memory usage has exceeded the `--processing-unavailable-at-memory-percent` threshold. **Solution** 1. Review the [response status codes](./usage.md#possible-response-status-codes). A 429 with `"The concurrency limit has been reached"` means too many simultaneous requests are active; retry after a short delay. 2. Monitor the `/status` endpoint to see current instance availability: ```bash curl http://localhost:8000/status | python3 -m json.tool | grep -A4 "spectra_core" ``` The `available_regular_cores` and `available_large_cores` fields show how many instances are currently free. 3. Implement retry logic with backoff in your client for 429 responses. Do not retry at a constant rate under load — this worsens congestion. 4. Increase the number of Spectra Core instances (`--number-of-regular-cores`) to handle higher concurrency, subject to available CPU and memory. See the [Configuration Reference](./configuration.md). 5. Check logs for the high-load indicators described in [Logging](./usage.md#logging): ```json {"level":"warn","process":"core","message":"High processing load"} ``` Wait for `"High processing load over"` before resuming normal submission rates. 6. For sustained high throughput, consider deploying multiple FIE instances behind a load balancer, with each instance's `/readyz` endpoint used as the health check. --- ## Port binding conflict **Symptom** The container fails to start with an error in the logs: ``` Error: listen tcp :8000: bind: address already in use ``` or Docker reports: ``` docker: Error response from daemon: driver failed programming external connectivity: Bind for 0.0.0.0:8000 failed: port is already allocated. ``` **Cause** - Another process on the host is already using port 8000 (the default FIE HTTP port). - A previous FIE container is still running and holding the port. - The Docker daemon has reserved the port range that includes 8000. **Solution** 1. Identify what is using the port: ```bash sudo lsof -i :8000 sudo ss -tlnp | grep 8000 ``` 2. If an old FIE container is occupying the port, stop it: ```bash docker ps -a | grep fie docker stop docker rm ``` 3. Map the container to a different host port without changing the internal port: ```bash docker run -p 9001:8000 \ -e RL_LICENSE="..." \ registry.reversinglabs.com/fie/file-inspection-engine: ``` 4. To change the port the FIE process listens on internally, use the `--http-address` flag: ```bash docker run -p 9001:9001 \ -e RL_HTTP_ADDRESS=":9001" \ -e RL_LICENSE="..." \ registry.reversinglabs.com/fie/file-inspection-engine: ``` See the [Configuration Reference](./configuration.md) for the `--http-address` option. --- ## Out of memory (OOM) — container killed **Symptom** The container is killed abruptly during analysis. Docker events or Kubernetes events show: ``` OOMKilled ``` or the host `dmesg` contains: ``` Out of memory: Kill process (fie) score or sacrifice child ``` **Cause** - The container memory limit is too low for the number of Spectra Core instances and the file types being analyzed. - Files with very high decompression ratios (deeply nested archives) are consuming more memory than expected. - The temporary directory is mounted as `tmpfs`, which counts toward container memory usage. **Solution** 1. Increase the container memory limit. As a general guideline, allocate at least 1–2 GB of memory per Spectra Core instance, plus overhead for the FIE process itself. For Docker: ```bash docker run --memory="8g" ... ``` For Kubernetes, update the resource limits in the Helm values. See the [Helm Values Reference](./Deployment/Examples/values.md). 2. If `tmpfs` is used as the temporary directory, its contents count toward container memory. Consider switching to a host-mounted volume for temporary files to avoid this. 3. Enable the memory threshold check using `--processing-unavailable-at-memory-percent`. This causes FIE to reject new submissions when memory usage is high, preventing OOM rather than being killed: ```bash docker run -e RL_PROCESSING_UNAVAILABLE_AT_MEMORY_PERCENT=85 ... ``` When memory exceeds 85%, the engine logs: ```json {"level":"warn","message":"Memory use is above the threshold of 90%"} ``` and starts returning HTTP 429 to new submissions. See [Memory Usage](./usage.md#memory-usage). 4. Reduce the number of concurrent Spectra Core instances (`--number-of-regular-cores`) to lower peak memory consumption. 5. Review the [platform requirements](/General/DeploymentAndIntegration/PlatformRequirements) for recommended memory allocations based on instance count and expected file types. --- ## Container restarts because of cgroup v2 `memory.oom.group` **Symptom** The Kubernetes node restarts the entire FIE pod when one Spectra Core engine hits its memory limit. Logs show a single engine OOM, but the container is removed rather than just the failing process. **Cause** - Kubernetes v1.28+ defaults the node-level `memory.oom.group=1`, so any OOM in the pod kills every process in that cgroup. FIE enables concurrent Spectra Core instances inside the same pod, and cgroup v2 enforces the group-wide kill. This differs from cgroup v1 behaviour where only the oom-ing process (engine) was restarted. - The behavior is driven by the node’s kubelet configuration and is not something the FIE image can change. **Solution** If you are deploying on Google Kubernetes Engine (GKE), you can restore the cgroup v1-style behavior where only the offending process is killed: 1. Enable the kubelet `singleProcessOOMKill` option on your node pools. This setting is available starting with GKE versions `1.32.4-gke.1132000` and `1.33.0-gke.1748000`. 2. Follow the Google Cloud documentation for [Customizing node system configuration](https://cloud.google.com/kubernetes-engine/docs/how-to/node-system-config) to apply the `singleProcessOOMKill: true` toggle. 3. After the nodes pick up the new kubelet config, pods experiencing isolated engine OOMs should only restart the affected Spectra Core process instead of the entire container. The pod will still log the original OOM event and should recover once the engine restarts. 4. Continue sizing memory and core counts according to the [platform requirements](/General/DeploymentAndIntegration/PlatformRequirements), since `singleProcessOOMKill` only affects how kubelet responds to the OOM—it does not prevent the underlying memory condition. --- ## Large files time out during analysis **Symptom** Analysis of files above a certain size returns: ```http HTTP/1.1 524 {"error": "The analysis could not be completed within the configured maximum analysis time"} ``` The container logs show: ```json {"level":"warn","message":"Analysis aborted due to a timeout"} {"level":"warn","message":"Analysis has timed out"} ``` **Cause** - The `--timeout` value is too short for the complexity of the file being analyzed. - A very large or deeply nested archive requires more time to unpack and analyze than the timeout allows. - After a timeout, the Spectra Core instance handling the file is restarted, temporarily reducing available capacity. **Solution** 1. Increase the analysis timeout using the `--timeout` flag. Duration values use `s`, `m`, or `h` suffixes: ```bash docker run -e RL_TIMEOUT="5m" ... ``` Note: very short timeout values are not recommended because instance restarts after a timeout can cause cascading delays. See [Timeouts](./usage.md#timeouts). 2. After a timeout, the affected instance restarts automatically. Monitor logs for `"Instance is ready"` to confirm recovery: ```json {"level":"info","message":"Instance is ready"} ``` 3. For predictably large files, configure a dedicated large-file instance pool using `--number-of-large-cores` and `--large-file-threshold`. These instances process one file at a time, and their separate timeout can be tuned independently: ```bash docker run \ -e RL_NUMBER_OF_LARGE_CORES=2 \ -e RL_LARGE_FILE_THRESHOLD=50 \ -e RL_TIMEOUT="10m" \ ... ``` See the [Configuration Reference](./configuration.md) for all large-file pool options. 4. Use the [Check for Hard Timeout](./usage.md#check-for-hard-timeout) procedure to distinguish regular timeouts from hard timeouts caused by Spectra Core process termination. --- ## License validation error on startup **Symptom** The container exits immediately or the `/readyz` endpoint returns a non-200 status. Container logs contain: ``` FATAL: License validation failed ``` or: ``` License expired ``` The `/status` endpoint returns a `valid_until` date in the past. **Cause** - The `RL_LICENSE` environment variable is not set. - The license file content is truncated, incorrectly formatted, or was copied with extra whitespace or line breaks. - The license has reached its expiration date. - For network-validated licenses, the container cannot reach the ReversingLabs license server. **Solution** 1. Confirm the `RL_LICENSE` environment variable is set. Pass the license as the entire file contents: ```bash # Using a license file on disk docker run -e RL_LICENSE="$(cat /path/to/rl-license.lic)" \ registry.reversinglabs.com/fie/file-inspection-engine: ``` For Kubernetes, store the license as a Secret and reference it in the pod spec: ```bash kubectl create secret generic fie-license \ --from-file=RL_LICENSE=/path/to/rl-license.lic ``` 2. Verify the license has not expired using the `/status` endpoint: ```bash curl http://localhost:8000/status | python3 -m json.tool | grep valid_until ``` 3. If the license is expired, contact your ReversingLabs account manager or [support@reversinglabs.com](mailto:support@reversinglabs.com) to obtain a renewed license. 4. Note that `RL_LICENSE` is only available as an environment variable, not as a CLI flag. See the [Configuration Reference](./configuration.md) for the `RL_LICENSE` parameter notes. --- ## Analysis results show UNKNOWN for all files **Symptom** All files submitted to `/scan` return `"classification": "OK"` regardless of file type, and no malicious verdicts are produced even for files known to be malicious. **Cause** - The `--without-malicious-threat-data=true` flag is set, which disables downloading of malicious threat data and prevents malicious classifications from threat data matching. - Threat data has not yet downloaded successfully, so the engine is operating without a populated database. - The threat database timestamp is very old (stale), indicating updates have not been applied for an extended period. **Solution** 1. Check the current threat data configuration and status using `/status`: ```bash curl http://localhost:8000/status | python3 -m json.tool ``` Review the `threat_data.enabled_classifications` field. If it shows an empty array (`[]`), malicious classification from threat data is disabled. The `version.threat_data` field shows when the database was last updated. 2. If `enabled_classifications` is empty, check whether `--without-malicious-threat-data=true` is set in your configuration. Remove this flag (or set it to `false`) if you want malicious threat data to be used: ```bash docker run -e RL_WITHOUT_MALICIOUS_THREAT_DATA=false ... ``` 3. If threat data is enabled but stale, verify that cloud updates are working. Check `--cloud-updates` is not set to `false` and that the container can reach the update server. See [Threat database download fails or is slow](#threat-database-download-fails-or-is-slow). 4. Note that with `--without-malicious-threat-data=false` (the default), FIE still classifies files using [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) static analysis, so some malicious files will be detected even without threat data. However, threat data significantly improves detection coverage. --- ## `/status` endpoint shows zero available instances **Symptom** The `/status` endpoint shows all Spectra Core instances as unavailable: ```json { "spectra_core": { "available_regular_cores": "0% (0/4)", "available_large_cores": "0% (0/2)" } } ``` All `/scan` requests are being rejected with 429 or 503. **Cause** - All instances are busy processing files submitted simultaneously. - One or more instances have timed out and are in the process of restarting. - All instances failed to initialize during startup (for example, due to resource exhaustion). **Solution** 1. Wait briefly and re-check `/status`. Instances that are restarting after a timeout typically recover within a few seconds. Look for `"Instance is ready"` log messages: ```bash docker logs -f | grep "Instance is ready" ``` 2. If instances are busy (not restarting), reduce the rate of incoming requests and allow in-flight analyses to complete. Check the concurrency limit (`concurrency_limit` in `/status`) and compare it to the number of active instances. 3. If instances failed during startup, check logs for initialization errors: ```bash docker logs 2>&1 | grep -i "error\|fatal\|failed" ``` 4. Check for [OOM conditions](#out-of-memory-oom--container-killed) — if instances are being killed by the kernel before they can finish initializing, the available count will remain at zero. 5. For a persistent `0/N` state where all instances are stuck, restart the container. If this state recurs, review the [platform requirements](/General/DeploymentAndIntegration/PlatformRequirements) to ensure the host has sufficient CPU and memory for the configured number of instances. 6. For Kubernetes deployments, check whether the pod itself is in a degraded state: ```bash kubectl describe pod -n kubectl top pod -n ``` --- ## File Inspection Engine API Reference ## General ### Starting the File Inspection Engine The File Inspection Engine starts the main application process separately from its analysis instances. The application will start successfully in most cases, except for fatal configuration errors. - The license is checked during startup. - Threat data must be downloaded before analysis can begin. - Analysis instances initialize independently. The application is considered ready when at least one instance is ready, which can be seen in the logs or through the `/status` endpoint. To verify overall readiness, use the readiness endpoint: ```bash curl http://:/readyz ``` If the application is ready, this returns `200 OK`. The `/readyz` endpoint is the recommended way to confirm readiness, because it checks not only instance availability but also license validity, threat data availability, and the concurrency limit. **Note: When starting for the first time, the application needs to download threat data. This process may take some time, and the application will only become fully usable once the threat data download is complete, regardless of the messages displayed.** If the static analysis engine fails to initialize or another fatal configuration error occurs, the application will exit. For all other errors, logs will be generated, and the application will continue to run. ### Possible Response Status Codes If present, the `errors` and `message` fields may contain soft errors (e.g., failing to get detailed threat information from the cloud), but are often empty. Hard errors will be returned as HTTP status `500 Internal Server Error`. |Code | Description | Message| |-----|--------------|------------ |200 | The request has succeeded. | N/A| |400 | File size error. | `{"error": "Maximum upload file size in bytes is {configured_value}" }` | |429 | Concurrency limit reached. | `{"error":"The concurrency limit has been reached"}` | |429 | High processing load | `{"error":"Analysis not accepted due to high processing load"}` | |524 | A timeout occurred. | `{"error": "The analysis could not be completed within the configured maximum analysis time"}`| ## Classification and Threat Data Configuration The File Inspection Engine's classification behavior is controlled by two key configuration options that work together: `--without-malicious-threat-data` and `--paranoid-mode`. When `--without-malicious-threat-data` is enabled, the engine skips downloading malicious threat data and relies on static analysis for classification. The `--paranoid-mode` option adds a third classification level (`suspicious`) and requires additional threat data. When both `--without-malicious-threat-data=true` and `--paranoid-mode=false`, cloud threat data updates are automatically disabled since no threat data is needed. However, if `--paranoid-mode=true`, suspicious threat data will still be downloaded even when malicious data is disabled. Related configuration options: - [`--without-malicious-threat-data`](./configuration.md#--without-malicious-threat-data--rl_without_malicious_threat_data) - [`--paranoid-mode`](./configuration.md#--paranoid-mode--rl_paranoid_mode) ### Classification Overrides The File Inspection Engine supports the following types of classification overrides: - **Goodware overrides**: Part of the goodware threat data classification. These are files marked as goodware by ReversingLabs analysts and are automatically downloaded when cloud updates are enabled. The goodware classification requires 64 MiB of threat data. Analyst overrides are always goodware classifications. - **User overrides**: Custom classifications by other users within the same organization (indicated by the middle segment of the username - u/**company**/user). Users can change a file's classification to malicious, suspicious, or goodware. They are automatically downloaded when cloud updates are enabled, their count is logged at application startup. **Creating User Overrides** User overrides can be created in the following ways: 1. **Spectra Analyze**: Override classifications **in [Spectra Intelligence](/SpectraIntelligence/)** for specific files through the [Spectra Analyze user interface](/SpectraAnalyze/#administering-classification-overrides). 2. **Spectra Intelligence API**: Use the [TCA-0102 File reputation override](/SpectraIntelligence/API/FileThreatIntel/tca-0102) service to programmatically create user overrides. Both types of overrides apply only to container files, i.e. to the top-level file being scanned, and not to unpacked children within archives. For example, a file with a user override that is in a ZIP archive will not have the override applied. ## File Submissions ``` POST /scan ``` To scan a file, make a POST request to `http://:/scan`, with the file contents as the raw request body. - You don't need to set the `Content-Type` header. If set, it will be included alongside every log message related to that submission. - It is recommended to set the `Content-Length` header to prevent files larger than the maximum upload size from partially uploading before getting rejected. - Optionally, you can provide an `external_id` query string parameter that will also be included alongside log messages related to the submission. This parameter can contain any value meaningful to the client application, such as a file name, database ID, or other identifying information. ### Examples using `curl` **Example request:** ```bash curl -X POST --upload-file example.docx http://:/scan ``` **Example response:** ```json5 { "classification": "OK", "message": "", "errors": [] } ``` **Classification**: The `classification` string will be either `"OK"` or `"malicious"`. If [paranoid mode](/FileInspectionEngine/configuration/#--paranoid-mode--rl_paranoid_mode) is turned on, the classification could also be `"suspicious"`. If `with-threat-details` and `add-file-type` options are enabled, the response may look like: ```json { "classification": "malicious", "message": "", "errors": [], "threat_details": { "platform": "Script", "type": "Trojan", "threat_name": "Script-JS.Trojan.Redirector" }, "file_type": "Text" } ``` **Logging:** The following example provides `Content-Type` and a custom external ID in the request, both of which can be visible in application logs: ```bash curl -X POST -H 'Content-Type: application/x-tar' --upload-file archive.tar 'http://localhost:8000/scan?external_id=my%20external%20id' ``` Log example: This request would create the following log entry, including the external ID: ```json { "level": "info", "process": "fie", "request_id": "2fd7364b-50c5-4128-ae08-35a819dda62f", "external_id": "my external id", "content_type": "application/x-tar", "component": "http.api", "request_path": "/scan", "content_length": 244244480, "active_concurrency": "1/20", "time": "2025-09-24T15:25:06.350178812+02:00", "message": "Upload started" } ``` ### Error handling If there are any errors, they will be returned in the `message` field (deprecated), as well as the `errors` field. The `message` field will contain the same errors as the `errors` array, only it will be a semicolon-concatenated string. For example: ```json5 { "message": "error one; error two; error three", "errors": ["error one", "error two", "error three"] } ``` In some cases, certain errors are expected, and are converted to additional properties inside `analysis_information`. For example, if a file hits the [decompression factor limit](/FileInspectionEngine/configuration/#--max-decompression-factor--rl_max_decompression_factor), this error will be logged in `errors` and `message`, but also present in `analysis_information.partial_unpacking`. ```json5 { "errors": ["Exceeds decompression ratio."], "message": "Exceeds decompression ratio.", "analysis_information": { "partial_unpacking": true } } ``` ## Hash Lookups Use the following endpoints to check sample classification by sample or hash without triggering static analysis. ### Compute hash and perform lookup ``` POST /check-sample/upload ``` To use a file to compute its hash and check its classification, make a POST request with the file sample as the raw request body. **Note: Even though the sample is not sent for static analysis, the configured file size limit still applies.** #### Examples using `curl` **Example request:** ```bash curl -X POST --upload-file sample 'http://localhost:8000/check-sample/upload' ``` **Example response:** ```json5 { "classification": "malicious" } ``` **Classification**: The `classification` string will be either `"OK"` or `"malicious"`. If [paranoid mode](/FileInspectionEngine/configuration/#--paranoid-mode--rl_paranoid_mode) is turned on, the classification could also be `"suspicious"`. If the `with-threat-details` option is enabled, the response may look like: ```json { "classification": "malicious", "threat_details": { "platform": "Win32", "type": "Malware", "threat_name": "Win32.Malware.Heuristic" } } ``` ### Provide hash and perform lookup ``` GET /check-sample/hash/{name}/{value} ``` To check a file's classification by providing its hash, make a GET request using the following path parameters: - `name`: Hash type; currently supports only `sha1` - `value`: SHA1 hash string #### Examples using `curl` **Example request:** ```bash curl -X GET 'http://localhost:8000/check-sample/hash/sha1/74577262dad60dc5bf35c692f23300c54c92cb53' ``` **Example response:** ```json5 { "classification": "malicious" } ``` **Classification**: The `classification` string will be either `"OK"` or `"malicious"`. If [paranoid mode](/FileInspectionEngine/configuration/#--paranoid-mode--rl_paranoid_mode) is turned on, the classification could also be `"suspicious"`. If the `with-threat-details` option is enabled, the file hash will also be submitted to the cloud API to retrieve additional threat details. The response may look like: ```json { "classification": "malicious", "threat_details": { "platform": "Win32", "type": "Malware", "threat_name": "Win32.Malware.Heuristic" } } ``` ## Request Rejection When a `/scan` or `/check-sample` upload request reaches an FIE instance, the engine first performs a readiness check before accepting the file for processing. Depending on the setup, readiness checks can be approached in two ways: - Kubernetes Readiness: Point your container's readiness probe to `/readyz`. When a container fails its readiness check, Kubernetes marks the Pod as not ready, and Services automatically stop routing traffic to it. - External Load Balancer: Configure your load balancer's health check on `/readyz` so only ready instances receive traffic. These probes are helpful to keep traffic away from busy nodes, but they are optional. The engine also performs its own readiness check for each file upload. **Note: Depending on the delay between the readiness check and the file submission, it is possible that the application returns a ready state, but the file can still be rejected if the conditions change. Such files will have to be resubmitted.** The engine evaluates several conditions to determine whether a file can be accepted. Some conditions are influenced by system state, such as memory usage or processing load, while others, such as concurrency, are driven by the volume of incoming requests. Logging occurs when any of these conditions change, regardless of file submission activity. ### Memory Usage The system can **optionally track memory usage** and compare it to a configured threshold (`processing-unavailable-at-memory-percent`). When enabled, memory usage is calculated as a percentage of either: - the [memory limit defined for the container](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) (if set), or - the total available system memory (if no container limit is present). If this threshold is exceeded at the time of file submission, the application will reject the file until memory usage drops below the limit. **Note: If the temporary directory is configured as `tmpfs`, it will be counted toward memory usage.** Because memory usage depends on the complexity and unpacking behavior of previously submitted files, it may remain elevated even after no new uploads occur. The system logs when it enters and exits high-memory conditions independently of file submissions. If the threshold is not configured, memory usage is **not tracked or logged**, and file rejection based on memory use is disabled. ### Concurrency Limit The system enforces a limit on concurrent requests, defined by the `concurrency-limit` setting. If the number of active concurrent requests exceeds this limit, new submissions are temporarily rejected. The concurrency limit applies **globally** to all uploads, regardless of whether they are routed to the regular core pool or cores reserved for large files. Even if the concurrency limit is set to `0` (unlimited), the system still tracks the number of active concurrent requests. Concurrency is controlled by the number of active file submissions at any given moment. This value is directly influenced by the client's submission behavior, making it a more predictable limit compared to memory or processing-based conditions. ### Multiple Cores Requests are assigned to [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) instances. Each file is always processed by a single instance. - All instances are identical in capability. - You can configure a subset of instances to be reserved for files larger than the size threshold (`--large-file-threshold`). - Large-file instances process only **one file at a time**, regardless of the global concurrency limit. - If no large instances are configured, all files are handled by the same pool. The `/readyz` endpoint returns `200 OK` if at least one instance (regular or large-file) is available. For detailed availability per group, use the `/status` endpoint. ## Logging If a file upload is rejected due to memory or processing conditions, the application will return the HTTP status `429 Too Many Requests`. The application logs when it enters and exits high-load or high-memory states. These log entries are independent of file submission attempts, since resource usage is influenced by the complexity of previously submitted files, not just their size or frequency. ### Log Examples **Processing Load: High and Normal** When a Spectra Core instance becomes busy, logs indicate that it cannot accept new files: ```json { "level": "info", "process": "fie", "instance_id": "core-large-0.8qczl", "time": "2025-09-24T15:39:07.618375188+02:00", "message": "Instance is not ready" } { "level": "warn", "process": "core", "instance_id": "core-large-0.8qczl", "time": "2025-09-24T15:39:58.946518488+02:00", "message": "High processing load" } ``` When the load subsides and the instance can accept new files again, logs show a return to normal: ```json { "level": "info", "process": "core", "instance_id": "core-large-0.8qczl", "time": "2025-09-24T15:40:15.592637095+02:00", "message": "High processing load over" } { "level": "info", "process": "fie", "instance_id": "core-large-0.8qczl", "time": "2025-09-24T15:40:15.677907694+02:00", "message": "Instance is ready" } ``` - `Instance is not ready` – The instance cannot accept new files right now. The `/status` endpoint will show it as unavailable, and new submissions will be rejected until it becomes free again. - `High processing load` – The instance is busy. This is an additional signal but less important for deciding whether to submit new files. - `High processing load over` – The instance has returned to a normal state. - `Instance is ready` – The instance is available again, either because load subsided or after a restart. **Note: If an instance recovers due to a timeout and restart, the return-to-normal sequence is slightly different: only the `Instance is ready` message appears after the restart.** **Memory Usage: High and Normal** ```json { "level": "warn", "process": "fie", "component": "readiness.Controller", "time": "2025-09-24T13:55:38.184606708Z", "message": "Memory use is above the threshold of 90%" } { "level": "info", "process": "fie", "component": "readiness.Controller", "time": "2025-09-24T13:56:33.183810287Z", "message": "Memory use is below the threshold of 90%" } ``` ## Timeouts The engine enforces a configurable **timeout** for file analysis. If a file exceeds the configured timeout: - The Spectra Core instance handling the file is terminated and restarted. - Any other analyses running on that instance are aborted. - Logs will show that the analysis was aborted and timed out. - The instance will need to restart and become ready again before it can accept new files. This usually takes a few seconds, but the exact time depends on system load and disk speed. For this reason, very short timeout values are not recommended. ```json { "level": "warn", "process": "fie", "request_id": "3dc57796-2964-4249-bff3-c98ddef747ca", "component": "scanner", "sample_size": 86271956, "sample_sha1": "0aa2f850f0e87ef84743f518a10d17e3b03395d7", "sample_type": "application/x-unix-archive", "scan_duration_ms": 60000.502133, "analyzed_files": 0, "timeout": "1m0s", "time": "2025-09-23T17:35:57.299468984+02:00", "message": "Analysis aborted due to a timeout" } { "level": "warn", "process": "fie", "request_id": "3dc57796-2964-4249-bff3-c98ddef747ca", "component": "http.api", "request_path": "/scan", "content_length": 86271956, "scan_duration_ms": 60349.621584, "sample_sha1": "0aa2f850f0e87ef84743f518a10d17e3b03395d7", "time": "2025-09-23T17:35:57.307513847+02:00", "message": "Analysis has timed out" } ``` After a timeout, the affected instance restarts. Once recovery is complete, it logs that it is ready again: ```json { "level": "info", "process": "fie", "instance_id": "core-regular-0.pwcpx", "time": "2025-09-23T17:36:07.943715309+02:00", "message": "Instance is ready" } ``` **Note: The random suffix in the `instance_id` changes after a restart.** ### Check for Hard Timeout You can look at log messages to determine if your instance has experienced a hard timeout. For example: ``` {"level":"warn","process":"fie","component":"core.process","instance_id":"core-regular-0.dmssx","time":"2025-12-01T13:53:14.665857373+01:00","message":"Hard timeout"} ``` ## Check Application Liveness ```bash curl http://:/livez ``` Returns `200 OK` if the application process is running. Use for Kubernetes liveness probes. ## Check Application Readiness ```bash curl http://:/readyz ``` Returns `200 OK` only if: - The engine has fully initialized (including license validation) and is not too busy to process samples. - Current resource utilization is within configured limits (memory, concurrency) and the system is not too busy (Spectra Core, CPU/load) to process samples. - If not ready, returns a `4xx` or `5xx` status. ## Check Application Version / License / Configuration ``` GET /status ``` To check the File Inspection Engine version, license expiration date, threat data timestamp, and configuration, use the `/status` endpoint. The `config` section contains all the [configurable options](./configuration.md) and their current values. Values containing sensitive information are redacted. The `spectra_core` object describes how analysis instances are configured and available: - **percentage_of_regular_cores**: The number of regular instances expressed as a percentage of the total CPU requests. - If no large-file pool is configured (`--number-of-large-cores=0`), these instances handle all files. - Otherwise, they handle files up to the `--large-file-threshold`. - **percentage_of_large_cores**: The number of large-file instances expressed as a percentage of the total CPU requests. - **available_regular_cores / available_large_cores**: Current availability of instances in each pool (available / total). - **total_cpus**: The total number of CPUs used for calculating percentages. If `RL_CPU_REQUEST` is set, this value comes from that variable. Otherwise, it reflects the total CPUs detected on the node. **Note: The `percentage_*` fields compare instance counts to the CPU request value. They do not represent actual CPU allocation. Each instance may use more or fewer CPUs depending on workload and system limits.** The `threat_data` object describes the threat data used by the engine: - **enabled_classifications**: The classifications enabled in the threat data, based on the combination of the `--without-malicious-threat-data` and `--paranoid-mode` options. When malicious or suspicious threat data is enabled, goodware classification is also automatically enabled. - `--without-malicious-threat-data=false + --paranoid-mode=false` -> `["malicious", "goodware"]` - `--without-malicious-threat-data=false + --paranoid-mode=true` -> `["malicious", "suspicious", "goodware"]` - `--without-malicious-threat-data=true + --paranoid-mode=false` -> `[]` (no threat data is used, files are classified based on static analysis) - `--without-malicious-threat-data=true + --paranoid-mode=true` -> `["suspicious", "goodware"]` - **fp_probability**: The false positive probability for each classification. Not shown for disabled classifications. **Example Response** ```bash curl http://:/status ``` ```json5 { "config": { "add_file_type": "disabled", "cloud_update_interval": "5m0s", "cloud_updates": true, "concurrency_limit": 20, "cpu_request": 8, "http_address": ":8000", "large_file_threshold": 10, "log_json": true, "max_decompression_factor": 1, "max_upload_file_size": 100, "number_of_large_cores": 2, "number_of_regular_cores": 4, "paranoid_mode": true, "processing_unavailable_at_memory_percent": 0, "proxy_address": "http://user:xxxxx@proxy.company.lan", "timeout": "0s", "unpacking_depth": 17, "with_threat_details": false, "without_malicious_threat_data": false }, "license": { "valid_until": "2026-03-01" }, "spectra_core": { "available_large_cores": "100% (2/2)", "available_regular_cores": "100% (4/4)", "percentage_of_large_cores": "25% (2)", "percentage_of_regular_cores": "50% (4)", "total_cpus": 8 }, "threat_data": { "enabled_classifications": [ "malicious", "suspicious", "goodware" ], "fp_probability": { "goodware": "1 in 4.221353e+87 samples", "malicious": "1 in 1.154405e+12 samples", "suspicious": "1 in 2.931565e+12 samples" } }, "version": { "application": "3.2.0", "threat_data": "2025-10-28T14:04:18Z" } } ``` --- ## Analysis Timeout Issues File analysis timeouts can occur when processing complex or large files that require extensive analysis time. Understanding the causes and solutions helps ensure successful file processing. ## Common Causes Analysis timeouts typically happen due to: - **Large file sizes** - Files approaching or exceeding the size limits for your appliance tier - **Deep nesting** - Archives containing multiple layers of compressed files - **Extensive unpacking** - Files that trigger recursive decompression operations - **Complex file structures** - Files with intricate internal structures requiring detailed parsing - **Resource constraints** - Insufficient RAM or CPU allocation for the analysis workload ## Configuration Options ### Spectra Analyze The analysis timeout can be adjusted in the appliance configuration: 1. Navigate to **Administration > Configuration** 2. Locate the analysis timeout setting 3. Increase the timeout value based on your file processing requirements 4. Save the configuration changes ### File Inspection Engine Use the `--analysis-timeout` flag to control the per-file time limit: ```bash rl-scan --analysis-timeout 300 /path/to/file ``` The timeout value is specified in seconds. ## Troubleshooting Steps If analysis timeouts persist: 1. **Increase allocated resources** - Ensure the appliance or container has sufficient RAM (32 GB+ recommended) and CPU cores 2. **Check decompression ratio limits** - Verify that recursive unpacking isn't exceeding configured limits 3. **Review file characteristics** - Examine the file structure to identify potential issues 4. **Monitor system resources** - Check if the appliance is under heavy load from concurrent analyses 5. **Adjust timeout values** - Increase timeout settings for complex file processing workflows ## Related Topics - [Platform Requirements](/General/DeploymentAndIntegration/PlatformRequirements) - Hardware specifications for different appliance tiers - [How Spectra Core analysis works](/General/AnalysisAndClassification/SpectraCoreAnalysis) - Understanding the analysis process --- ## Antivirus Result Availability When a sample is uploaded or rescanned in Spectra Intelligence, it will usually get new antivirus results **within 30 minutes**. When a sample has new antivirus results, these will available in relevant APIs, for example [TCA-0104 File analysis](/SpectraIntelligence/API/FileThreatIntel/tca-0104/). --- ## Certificate Revocation ReversingLabs maintains a certificate revocation database that is updated with each [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) release. Because the database is offline, some recently revoked certificates may not appear as revoked until the next update. Certificate Authority (CA) revocation alone is not sufficient to classify a sample as malicious. Most CAs backdate revocations to the certificate's issuance date, regardless of when or whether the certificate was abused. When additional context is available, ReversingLabs adjusts the revocation date to reflect the most appropriate point in time. If a certificate is whitelisted, this correction is not applied. ## Searching for Revoked Certificates You can find samples signed with revoked certificates using **Advanced Search** with the `tag:cert-revoked` keyword. Advanced Search is available both through the [Spectra Analyze user interface](/SpectraAnalyze/search-page/) and as the [TCA-0320 Advanced Search](/SpectraIntelligence/API/MalwareHunting/tca-0320/) API. --- ## File Classification and Risk Scoring — ReversingLabs # Classification File classification assigns a risk score (0-10) and threat verdict (malicious, suspicious, goodware, or unknown) to every analyzed file using ReversingLabs Spectra Core. The classification algorithm combines YARA rules, machine learning, heuristics, certificate validation, and file similarity matching to determine security status. YARA rules take precedence as the most authoritative signal, followed by other detection methods that contribute to the final verdict. The classification of a sample is based on a comprehensive assessment of its assigned risk factor, threat level, and trust factor; however, it can be manually or automatically overridden when necessary. Based on this evaluation, files are placed into one of the following buckets: - No threats found (unclassified) - Goodware/known - Suspicious - Malicious The classification process weighs signals from all available sources to arrive at the most accurate verdict. Some signals are considered more authoritative than others and take priority. For example, [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) YARA rules always take precedence because they are written and curated by ReversingLabs analysts. These rules provide the highest degree of accuracy, as they target specific, named threats. This does not mean that other classification methods are less important. Similarity matching, heuristics, and machine learning still contribute valuable signals and may produce additional matches. In cases where multiple detections apply, YARA rules simply serve as the deciding factor for the final classification. ## Risk score A risk score is a value representing the trustworthiness or malicious severity of a sample. Risk score is expressed as a number from 0 to 10, with 0 indicating whitelisted samples from a reputable origin, and 10 indicating the most dangerous threats. At a glance: Files with no threats found don't get assigned a risk score and are therefore **unclassified**. Values from 0 to 5 are reserved for samples classified as **goodware/known**, and take into account the source and structural metadata of the file, among other things. Since goodware samples do not have threat names associated with them, they receive a description based on their risk score. Risk scores from 6 to 10 are reserved for **suspicious** and **malicious** samples, and express their severity. They are calculated by a ReversingLabs proprietary algorithm, and based on many factors such as file origin, threat type, how frequently it occurs in the wild, YARA rules, and more. Lesser threats like adware get a risk score of 6, while ransomware and trojans always get a risk score of 10. ### Malware type and risk score In cases where multiple threats are detected and there are no other factors (such as user overrides) involved, the final classification is always the one that presents the biggest threat. If they belong to the same risk score group, malware types are prioritized in this order: | Risk score | Malware types | |------------|---------------------------------------------------------------------------------------------------------------------| | 10 | EXPLOIT > BACKDOOR > RANSOMWARE > INFOSTEALER > KEYLOGGER > WORM > VIRUS > CERTIFICATE > PHISHING > FORMAT > TROJAN | | 9 | ROOTKIT > COINMINER > ROGUE > BROWSER | | 8 | DOWNLOADER > DROPPER > DIALER > NETWORK | | 7 | SPYWARE > HYPERLINK > SPAM > MALWARE | | 6 | ADWARE > HACKTOOL > PUA > PACKED | ## Threat level and trust factor The [risk score table](#risk-score) describes the relationship between the risk score, and the threat level and trust factor used by the [File Reputation API](/SpectraIntelligence/API/FileThreatIntel/tca-0101). The main difference is that the risk score maps all classifications onto one numerical scale (0-10), while the File Reputation API uses two different scales for different classifications. ### Nomenclature The following classifications are equivalent: | File Reputation API | Spectra Analyze | Spectra Detect Worker | | ------------------- | --------------- | ------------------------ | | known | goodware | 1 (in the Worker report) | In the Worker report, the [risk score](#risk-score) is called `rca_factor`. ## Deciding sample priority The [risk score table](#risk-score) highlights that the a sample's risk score and its classification don't have a perfect correlation. This means that a sample's risk score cannot be interpreted on its own, and that the primary criterion in deciding a sample's priority is its classification. Samples classified as suspicious can be a result of heuristics, or a possible early detection. A suspicious file may be declared malicious or known at a later time if new information is received that changes its threat profile, or if the user manually modifies its status. The system always considers a malicious sample with a risk score of 6 as a higher threat than a suspicious sample with a risk score of 10, meaning that samples classified as malicious always supersede suspicious samples, regardless of the calculated risk score. The reason for this is certainty - a malicious sample is decidedly malicious, while suspicious samples need more data to confirm the detected threat. It is a constant effort by ReversingLabs to reduce the number of suspicious samples. While a suspicious sample with a risk score of 10 does deserve user attention and shouldn't be ignored, a malicious sample with a risk score of 10 should be triaged as soon as possible. ## Malware naming standard --- ## Handling False Positives # Handling False Positives A false positive occurs when a legitimate file is incorrectly classified as malicious. While ReversingLabs strives for high accuracy, false positives can occasionally happen due to the complexity of malware detection across hundreds of file formats and millions of samples. ## What You Can Do If you encounter a false positive, you have several options: ### 1. Local Classification Override On Spectra Analyze, you can immediately override the classification using the classification override feature: - Navigate to the file's Sample Details page - Use the classification override option to manually set the file as goodware - The override takes effect immediately on your appliance - All users on the same appliance will see the updated classification ### 2. Spectra Intelligence Reclassification Request Submit a reclassification request through Spectra Intelligence: - The override propagates across all appliances connected to the same Spectra Intelligence account - Other appliances in your organization will automatically receive the updated classification - This is the recommended approach for organization-wide corrections ### 3. Goodware Overrides Use Goodware Overrides to propagate trusted parent classifications to extracted child files: - If a trusted parent file (e.g., from Microsoft or another reputable vendor) contains files that trigger false positives - The parent's goodware classification can automatically override the child files - This is particularly useful for legitimate installers that may contain components flagged by heuristics ## How ReversingLabs Handles False Positive Reports If a customer reports a false positive (through Zendesk, or by contacting the Support team at support@reversinglabs.com), the first thing we do is re-scan the sample to make sure that the results are up-to-date. If the results are still malicious, our Threat Analysis team will: 1. Conduct our own research of the software and the vendor 2. Contact the AV scanners and notify them of the issue 3. Change the classification in our system (we do not wait for AVs to correct the issue) --- If the file is confirmed to be a false positive, we begin by analyzing why the incorrect classification occurred. Then we try to correct the result by making adjustments related to file relationships, certificates, AV product detection velocity (e.g. are detections being added or removed), we will re-scan and reanalyze samples, adjust/add sources and, if necessary, manually investigate the file. If these efforts do not yield a correct result, we have the ability to **manually override the classification** — but we only do so after thorough analysis confirms the file is benign. --- ## ReversingLabs malware naming standard The ReversingLabs detection string consists of three main parts separated by dots. All parts of the string will always appear (all three parts are mandatory). ``` platform-subplatform.type.familyname ``` 1. The first part of the string indicates the **platform** targeted by the malware. This string is always one of the strings listed in the [Platform string](#platform-string) table. If the platform is Archive, Audio, ByteCode, Document, Image or Script, then it has a subplatform string. Platform and subplatform strings are divided by a hyphen (`-`). The lists of available strings for Archive, Audio, ByteCode, Document, Image and Script subplatforms can be found in their respective tables. 2. The second part of the detection string describes the **malware type**. Strings that appear as malware type descriptions are listed in the [Type string](#type-string) table. 3. The third and last part of the detection string represents the malware family name, i.e. the name given to a particular malware strain. Names "Agent", "Gen", "Heur", and other similar short generic names are not allowed. Names can't be shorter than three characters, and can't contain only numbers. Special characters (apart from `-`) must be avoided as well. The `-` character is only allowed in exploit (CVE/CAN) names (for example CVE-2012-0158). #### Examples If a trojan is designed for the Windows 32-bit platform and has the family name "Adams", its detection string will look like this: ``` Win32.Trojan.Adams ``` If some backdoor malware is a PHP script with the family name "Jones", the detection string will look like this: ``` Script-PHP.Backdoor.Jones ``` Some potentially unwanted application designed for Android that has the family name "Smith" will have the following detection string: ``` Android.PUA.Smith ``` Some examples of detections with invalid family names are: ``` Win32.Dropper.Agent ByteCode-MSIL.Keylogger.Heur Script-JS.Hacktool.Gen Android.Backdoor.12345 Document-PDF.Exploit.KO Android.Spyware.1a Android.Spyware.Not-a-CVE Win32.Trojan.Blue_Banana Win32.Ransomware.Hydra:Crypt Win32.Ransomware.HDD#Cryptor ``` #### Platform string The platform string indicates the operating system that the malware is designed for. The following table contains the available strings and the operating systems for which they are used. | String | Short description | | ----------- | ------------------------------------------------------------------------------------------ | | ABAP | SAP / R3 Advanced Business Application Programming environment | | Android | Applications for Android OS | | AOL | America Online environment | | Archive | Archives. See [Archive subplatforms](#archive-subplatforms) for more information. | | Audio | Audio. See [Audio subplatforms](#audio-subplatforms) for more information. | | BeOS | Executable content for Be Inc. operating system | | Boot | Boot, MBR | | Binary | Binary native type | | ByteCode | ByteCode, platform-independent. See [ByteCode subplatforms](#bytecode-subplatforms) for more information. | | Blackberry | Applications for Blackberry OS | | Console | Executables or applications for old consoles (e.g. Nintendo, Amiga, ...) | | Document | Documents. See [Document subplatforms](#document-subplatforms) for more information. | | DOS | DOS, Windows 16 bit based OS | | EPOC | Applications for EPOC mobile OS | | Email | Emails. See [Email subplatforms](#email-subplatforms) for more information. | | Firmware | BIOS, Embedded devices (mp3 players, ...) | | FreeBSD | Executable content for 32-bit and 64-bit FreeBSD platforms | | Image | Images. See [Image subplatforms](#image-subplatforms) for more information. | | iOS | Applications for Apple iOS (iPod, iPhone, iPad…) | | Linux | Executable content for 32 and 64-bit Linux operating systems | | MacOS | Executable content for Apple Mac OS, OS X | | Menuet | Executable content for Menuet OS | | Novell | Executable content for Novell OS | | OS2 | Executable content for IBM OS/2 | | Package | Software packages. See [Package subplatforms](#package-subplatforms) for more information. | | Palm | Applications for Palm mobile OS | | Script | Scripts. See [Script subplatforms](#script-subplatforms) for more information. | | Shortcut | Shortcuts | | Solaris | Executable content for Solaris OS | | SunOS | Executable content for SunOS platform | | Symbian | Applications for Symbian OS | | Text | Text native type | | Unix | Executable content for the UNIX platform | | Video | Videos | | WebAssembly | Binary format for executable code in Web pages | | Win32 | Executable content for 32-bit Windows OS's | | Win64 | Executable content for 64-bit Windows OS's | | WinCE | Executable content for Windows Embedded Compact OS | | WinPhone | Applications for Windows Phone | ##### Archive subplatforms | String | Short description | | ---------------------------------- | ------------------------------------------------------------ | | ACE | WinAce archives | | AR | AR archives | | ARJ | ARJ (Archived by Robert Jung) archives | | BZIP2 | Bzip2 archives | | CAB | Microsoft Cabinet archives | | GZIP | GNU Zip archives | | ISO | ISO image files | | JAR | JAR (Java ARchive) archives | | LZH | LZH archives | | RAR | RAR (Roshal Archive) archives | | 7ZIP | 7-Zip archives | | SZDD | Microsoft SZDD archives | | TAR | Tar (tarball) archives | | XAR | XAR (eXtensible ARchive) archives | | ZIP | ZIP archives | | ZOO | ZOO archives | | *Other Archive identification* | All other valid [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) identifications of Archive type | ##### Audio subplatforms | String | Short description | | -------------------------------- | ---------------------------------------------------------- | | WAV | Wave Audio File Format | | *Other Audio identification* | All other valid Spectra Core identifications of Audio type | ##### ByteCode subplatforms | String | Short description | | ------ | ----------------- | | JAVA | Java bytecode | | MSIL | MSIL bytecode | | SWF | Adobe Flash | ##### Document subplatforms | String | Short description | | ----------------------------------- | ------------------------------------------------------------ | | Access | Microsoft Office Access | | CHM | Compiled HTML | | Cookie | Cookie files | | Excel | Microsoft Office Excel | | HTML | HTML documents | | Multimedia | Multimedia containers that aren't covered by other platforms (e.g. ASF) | | Office | File that affects multiple Office components | | OLE | Microsoft Object Linking and Embedding | | PDF | PDF documents | | PowerPoint | Microsoft Office PowerPoint | | Project | Microsoft Office Project | | Publisher | Microsoft Office Publisher | | RTF | RTF documents | | Visio | Microsoft Office Visio | | XML | XML and XML metafiles (ASX) | | Word | Microsoft Office Word | | *Other Document identification* | All other valid Spectra Core identifications of Document type | ##### Email subplatforms | String | Short description | | ------ | ------------------------------------- | | MIME | Multipurpose Internet Mail Extensions | | MSG | Outlook MSG file format | ##### Image subplatforms | String | Short description | | -------------------------------- | ------------------------------------------------------------ | | ANI | File format used for animated mouse cursors on Microsoft Windows | | BMP | Bitmap images | | EMF | Enhanced Metafile images | | EPS | Adobe Encapsulated PostScript images | | GIF | Graphics Interchange Format | | JPEG | JPEG images | | OTF | OpenType Font | | PNG | Portable Network Graphics | | TIFF | Tagged Image File Format | | TTF | Apple TrueType Font | | WMF | Windows Metafile images | | *Other Image identification* | All other valid Spectra Core identifications of Image type | ##### Package subplatforms | String | Short description | | ---------------------------------- | ------------------------------------------------------------ | | NuGet | NuGet packages | | DEB | Debian Linux DEB packages | | RPM | Linux RPM packages | | WindowStorePackage | Packages for distributing and installing Windows apps | | *Other Package identification* | All other valid Spectra Core identifications of Package type | ##### Script subplatforms | String | Short description | | --------------------------------- | ------------------------------------------------------------ | | ActiveX | ActiveX scripts | | AppleScript | AppleScript scripts | | ASP | ASP scripts | | AutoIt | AutoIt scripts (Windows) | | AutoLISP | AutoCAD LISP scripts | | BAT | Batch scripts | | CGI | CGI scripts | | CorelDraw | CorelDraw scripts | | Ferite | Ferite scripts | | INF | INF Script, Windows installer scripts | | INI | INI configuration file | | IRC | IRC, mIRC, pIRC/Pirch Script | | JS | Javascript, JScript | | KiXtart | KiXtart scripts | | Logo | Logo scripts | | Lua | Lua scripts | | Macro | Macro (e.g. VBA, AmiPro macros, Lotus123 macros) | | Makefile | Makefile configuration | | Matlab | Matlab scripts | | Perl | Perl scripts | | PHP | PHP scripts | | PowerShell | PowerShell scripts, Monad (MSH) | | Python | Python scripts | | Registry | Windows Registry scripts | | Ruby | Ruby scripts | | Shell | Shell scripts | | Shockwave | Shockwave scripts | | SQL | SQL scripts | | SubtitleWorkshop | SubtitleWorkshop scripts | | WinHelp | WinHelp Script | | WScript | Windows Scripting Host related scripts (can be VBScript, JScript, …) | | *Other Script identification* | All other valid Spectra Core identifications of Script type | #### Type string This string is used to describe the general type of malware. The following table contains the available strings and describes what each malware type is capable of. For a catalog of common software weaknesses that enable malware, see [CWE](https://cwe.mitre.org/) maintained by MITRE. CISA maintains advisories on actively exploited vulnerabilities at [cisa.gov/known-exploited-vulnerabilities](https://www.cisa.gov/known-exploited-vulnerabilities). | String | Description | | ----------- | ------------------------------------------------------------ | | Adware | Presents unwanted advertisements | | Backdoor | Bypasses device security and allows remote access | | Browser | Browser helper objects, toolbars, and malicious extensions | | Certificate | Classification derived from certificate data | | Coinminer | Uses system resources for cryptocurrency mining without the user's permission | | Dialer | Applications used for war-dialing and calling premium numbers | | Downloader | Downloads other malware or components | | Dropper | Drops malicious artifacts including other malware | | Exploit | Exploits for various vulnerabilities, CVE/CAN entries | | Format | Malformations of the file format. Classification derived from graylisting, validators on unpackers | | Hacktool | Software used in hacking attacks, that might also have a legitimate use | | Hyperlink | Classifications derived from extracted URLs | | Infostealer | Steals personal info, passwords, etc. | | Keylogger | Records keystrokes | | Malware | New and recently discovered malware not yet named by the research community | | Network | Networking utilities, such as tools for DoS, DDoS, etc. | | Packed | Packed applications (UPX, PECompact…) | | Phishing | Email messages (or documents) created with the aim of misleading the victim by disguising itself as a trustworthy entity into opening malicious links, disclosing personal information or opening malicious files. | | PUA | Potentially unwanted applications (hoax, joke, misleading...) | | Ransomware | Malware which encrypts files and demands money for decryption | | Rogue | Fraudulent AV installs and scareware | | Rootkit | Provides undetectable administrator access to a computer or a mobile device | | Spam | Other junk mail that does not unambiguously fall into the Phishing category, but contains unwanted or illegal content. | | Spyware | Collects personal information and spies on users | | Trojan | Allows remote access, hides in legit applications | | Virus | Self-replicating file/disk/USB infectors | | Worm | Self-propagating malware with exploit payloads | --- ## Risk score reference table --- ## Risk Tolerance [PREVIEW] To enable this preview feature, contact [ReversingLabs Support](mailto:support@reversinglabs.com). When enabled, the Risk Tolerance feature introduces the following additional classification reasons: - Classified by Auxiliary Analysis - Classified by Joe Sandbox - Classified by Network References - Classified by Cloud Sandbox (RLCS) Availability of these services depends on your appliance configuration. **Note: Cloud Sandbox (RLCS) can also be included in verdicts through a GUI option. With the Risk Tolerance feature enabled, however, RLCS results are weighted more heavily and can classify a sample as malicious even if other sources do not.** --- ## Self-Service Registration This guide describes how to set up self-service registration for Spectra Analyze using Okta as the identity provider. It can be set up with SAML or OIDC integration. ## Setting up Okta with Spectra Analyze In the Okta Administration interface, navigate to **Applications > Applications** and click **Create App Integration**. Set up either OIDC or SAML integration. ### SAML 1. Enter the required username and password attribute statements. **Info: Group attributes are only necessary if the groups functionality is intended to be used.** ![Attribute settings](../images/analyze-self-service-attribute-settings.png) 2. When the integration is configured, navigate to the **Assignments** tab and assign at least one group to the application. 3. Navigate to the **Sign On** tab and download the metadata. ![Okta Sign On tab with metadata download option](../images/analyze-sign-on.png) 4. Set up the SAML configuration in Spectra Analyze with matching claims and upload the metadata. **Info: Make sure to select the `Allow unsolicited responses from IdP` checkbox.** ![Spectra Analyze SAML configuration with claim matching settings](../images/analyze-self-service-saml-configuration.png) ## Enabling Sign Up 1. In **Security > Profile Enrollment**, create a new **Profile Enrollment Policy**. 2. Set the **Self-service registration** to `Allowed`, and select the **Email verification** checkbox. ![Profile Enrollment Policy configuration with self-service registration option](../images/analyze-self-service-profile-enrollment.png) 3. Navigate to **Manage apps** and add your application to the created policy. 4. In **Security > Authenticators**, edit the **Email authenticator**, and enable it to be used for both authentication and recovery. ![Email authenticator settings for authentication and recovery](../images/analyze-self-service-email.png) 5. In the **Enrollment** tab, add a new policy for the same group that was assigned to the application, and set the **Email** authenticator to at least `Optional`. ![Email authenticator enrollment policy configuration](../images/analyze-self-service-edit-policy.png) 6. In **Security > Authentication Policies**, click **Add a policy**. 7. Add your application to this policy, and edit the **Catch-all Rule** by setting it to deny access. 8. Add a new rule, and configure it so that the access is allowed, and the user must authenticate using Password / IdP + another factor. 9. For the **Possestion factor constraints**, set `Require user interaction`. 10. Set up the Authentication methods according to your requirements, and set it to allow at least password and email. **Frequency of password** and other factors of authentication can be configured at the bottom. ![Conditions](../images/analyze-self-service-conditions.png) **Note: To add additional conditions, such as limiting the eligible emails for sign up to a certain domain, add a custom expression in the **IF** section, such as `user.profile.email.substringAfter('@') == 'reversinglabs.com'`.** Any failed attempts will create inactive user accounts in Okta's directory. An alternative involves [inline hooks](https://help.okta.com/oie/en-us/content/topics/automation-hooks/inline-hooks-main.htm?cshid=csh-inlinehooks), making the web service require additional filtering capabilities. This approach should not result in inactive user accounts. ![Edit Rule](../images/analyze-self-service-edit-rule.png) ## Customizing the Sign Up Screen 1. In **Customizations > Other**, you can change the error message for failed sign-up attempts, and other error messages. ![Error Messages](../images/analyze-self-service-error-message.png) 2. In **Customizations > Brands**, you can configure the appearance of the screens, including the sign in screens and emails sent by Okta. The sign up form inputs are configured in **Security > Profile Enrollment** by selecting the policy used for the application. ![Profile Enrollment Form](../images/analyze-self-service-profile-enrollment-form.png) ## Allowing Sign Up with other Identity Providers In **Security > Identity Providers**, you can add a number of identity providers, including Microsoft IdP, Google IdP, and others. They can be assigned to certain applications through the **Routing rules** tab. ## User Management User management can be done by using both fetch and delete methods through the API. To use the API, you need to generate an API token. 1. In **Security > API**, navigate to the Token tab to generate a new token. ### Fetching Users Fetching all users assigned to the created application can be done via the `/api/v1/apps/:appId/users` endpoint. ``` GET /api/v1/apps/:appId/users ``` Path parameter: - `appId` - The ID of the application. - Required Query parameter: - `expand=user` - Find user accounts that did not fit the criteria for the sign up, but were still created and are inactive. - Optional #### Example An example of a request to fetch all user IDs, their usernames, and statuses of the users assigned to the application. ``` curl "https://${OKTA_INSTANCE_URL}/api/v1/apps/${APP_ID}/users?expand=user" \ --header 'Authorization: SSWS ${TOKEN_VALUE}' \ --header 'Accept: application/json' | jq '.[] | [.id, .credentials.userName, ._embedded.user.status]' ``` Once the results are fetched, the `STAGED` status indicates that the user account was created, but the user has not yet activated it because they did not match the criteria for the sign up. You can take the IDs of these users and delete them. ### Deleting Users Deleting users assigned to the created application can be done via the `/api/v1/users/:userId` endpoint. ``` DELETE /api/v1/users/:userId ``` Path parameter: - `userId` - The ID of the user to be deleted. - Required Query parameter: - `sendEmail` - If set to `true`, an email will be sent to the user notifying them of the deletion. - Optional #### Example An example of a request to delete a user with the `STAGED` status. ``` curl -X DELETE "https://${OKTA_INSTANCE_URL}/api/v1/users/${USER_ID}?sendEmail=false" \ --header 'Authorization: SSWS ${TOKEN_VALUE}' \ --header 'Accept: application/json' ``` **Info: You will need to execute the request twice because the first request deprovisions the account, while the second request deletes it.** --- ## Discussion The *Discussion* page displays the comments that have been added to a sample, either by the user who uploaded it or by other users. The appliance can also automatically add comments to a sample; for example, when a user manually changes its classification. Users can add new comments and apply some basic formatting, as well as insert links into the comment text field. Another way to access the *Discussion* page for a sample is to click the *Comments* entry in the expanded details view on the uploads page. --- ## Dynamic Analysis Results Each dynamic analysis integration adds its own section to the Sample Details page. Depending on the service, you may find: - Screenshots from the sandbox execution, with a gallery view and slideshow option. - Dropped files and other artifacts (such as PCAPs and memory strings), available for download within the service's retention period. - Behavioral and network indicators of compromise (IoCs), including URIs, IP addresses, and domains seen during execution. - History of past analyses for the same sample, with the ability to download reports for each run. - Interactive report tabs showing processes, network activity, file system changes, registry modifications, and other runtime behavior. - Links to full reports in the service's native interface. - Export options to generate HTML or PDF reports (where supported). Dynamic analysis services must be configured by the appliance administrator. For more information, see the [Analysis Service Integrations](../dynamic-analysis.md) page. --- ## Extracted Files The *Extracted Files* page allows browsing through the entire hierarchy of files extracted from a sample. Below the Extracted Files page link in the sidebar, all extracted files are broken down by their classification. The Spectra Analyze performs recursive sample analysis where each file extracted from a sample is analyzed separately. The “parent” file then inherits classification (for example, malicious) if the “child” file is deemed malicious. This is also known as **classification propagation**. There are two views for extracted files: 1. Breadcrumb view. 2. Flat view. Both views have options for filtering and exporting, while the flat view has additional filters (name, threat, and format). The **Export** menu contains options to export the whole page or just the files extracted from a sample. For the **Selected** option to become available, one or more files on the page have to be selected. To export multiple pages of results, browse pages one by one and manually export them. Data can be exported as CSV, JSON or XML. To copy only the file hashes for the desired set of extracted files, the menu contains options to copy SHA1, SHA256 and MD5 hashes to the clipboard. Hashes are delimited by a whitespace, so that they can be directly pasted into the search bar. The exported file can contain one or more of the following columns. They can be enabled in the **Export** menu, under **Show more Export options** - **Files** - the number of files extracted from a sample (if the sample is a container). - **Format** - file format of each file in the results grid. - **Name** - extracted file name - **Time** - timestamp when a file was extracted - **Threat** - detected threat name for malicious and suspicious files, formatted according to ReversingLabs Malware Naming Standard. - **Size** - indicates the size of a file. - **SHA256, MD5** - additional file hashes that can be included in the exported file. The SHA1 hash is always included by default. The actions menu (☰) on the right contains options for downloading the sample, reclassifying and reanalyzing it, downloading unpacked files, downloading the top-most container of the selected file, and previewing the selected file (described in the [File Preview / Visualizations](./file-preview.md) section). When one or more files are selected, another actions menu (☰) appears highlighted to the right of the *Size* column. The menu contains the *Apply tags* and *Subscribe/Unsubscribe* options that can be used on files individually or in bulk. --- ## File Preview / Visualization There are several ways to preview a file: - by clicking the “Preview / Visualizations” link in the persistent sample summary above the navigation sidebar - from the Preview / Visualizations menu item in the sidebar - by clicking the *View HEX* option in the actions menu (☰) for a file on the *Extracted Files* page The file preview window contains three tabs: Hex, Structure and Entropy. For samples that matched a YARA rule, the Hex tab displays a YARA Matches filter, allowing the users to see the exact parts of the file that matched a specific YARA rule. Matches can be filtered by ruleset, rule or matched value. For supported image formats (PNG, JPG, GIF), an additional “View” will be present, allowing the users to preview the image. The appliance displays a warning message before allowing the image preview. ![Sample Details page showing the File Preview of a JPEG image](images/analyze-file-preview-image.png) A similar principle applies for other supported file formats, such as text documents and some script languages, where the code is displayed and can be beautified in the “PREVIEW” tab. ![Preview tab showing a section of code](images/analyze-file-preview.png) The *Entropy* tab visualizes the amount of entropy per each section of the file. Entropy is used to express “randomness” of the data in a file, or to measure predictability of any character in the file based on preceding character distribution. It is measured in a scale of 1 to 8, where 8 indicates highest entropy (highest measure of randomness). Typical user data, such as text files, rarely exhibits true randomness. On the other hand, encrypted or compressed files have a high measure of entropy. Therefore, entropy can be used to detect encryption and compression in suspicious files. The *Structure* tab visualizes the sections of a PE/PE+ file, such as PE header, Import table, and Overlay. It is possible to click the items in the list on the right (Virtual, Physical, Import table, Resource table…). Clicking an item redirects the user to the related section of the static analysis results (on the [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) page). --- ## Sample Details Page ## Types of Sample Details pages The Sample Details page presents all the available information about a sample. ### Local For local files, the information is collected from [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) static analysis results, Spectra Intelligence, dynamic analysis, and auxiliary analysis. Administrators can configure processing settings on the appliance (“Fast”, “Normal”, “Best”). This will impact which file formats will be fully processed and how much information will be presented for them. ![An image showing the local version of the Sample Summary page.](images/local.png) ### Cloud For samples that are not local, the scope of information will depend on the information provided by Spectra Intelligence. This is usually a subset of what is available for locally available files: a section of static analysis results and Spectra Intelligence scanning results. ![An image showing the cloud version of the Sample Summary page.](images/cloud.png) ### Network Threat Intelligence For URLs, domains and IP addresses, the appliance displays a different type of [sample details page focused on Network Threat Intelligence](./network-threat-intelligence.md). Additionally, information displayed on the Sample Details page differs based on the file type and classification status of each sample. ![An image showing the network threat intelligence version of the Sample Summary page.](images/url.png) ## Accessing Sample Details pages To access the Sample Details page for a sample, click the sample name in any of the following pages: - Search > Local, Cloud and Network results - Alerts - YARA > Local and local-retro ruleset matches The page consists of a navigation sidebar on the left and the main information area on the right. The sidebar can be collapsed or expanded by clicking the Show/Hide Panel button at the top of the sidebar. At the top of the navigation sidebar, there’s a persistent short summary showing information such as file hash, predicted filename (if it exists), file size, file type and format, and the **Preview / Visualizations** link to open the [File Preview Dialog](./file-preview.md). If the predicted filename exists, it can be found right below the file hash. The right section of the page is the main information area. Its contents change depending on the section selected in the navigation sidebar. The navigation sidebar provides quick access to all parts of the analysis report. The sidebar sections are collapsed by default, unless the Sample Details page is accessed via a specific link targeting a section in the sidebar. ## Main Page Actions When any item from the *File Analysis Detail* section is selected in the sidebar, the main information area will contain the following options in the top right of the page: ### Reanalyze Opens a floating dialog where users can reanalyze the submission with static, dynamic, or Spectra Intelligence analysis services. ### Similarity Contains advanced search pivot options to search for similar and [functionally similar](../search-page.md#analyzing-files-with-rha-similarity-algorithm) samples. ### Fetch & Analyze *Only on cloud samples.* Opens a dialog with analysis options for the user to choose from. Available analysis options are Static Analysis (always enabled and cannot be modified), Threat Intelligence (Spectra Intelligence), Cloud Sandbox (Dynamic Analysis), Interactive Analysis (interactive RL Cloud Sandbox session) Auxiliary Analysis, and Integration Analysis. This option will download the sample from the Spectra Intelligence cloud and analyze it locally on the appliance. If the sample is not available for download, the button will be disabled. ### Actions Button Depending on the [type](#types-of-sample-details-pages) of page, different choices are available. [Local](#local) samples: - PDF: - **Create PDF** option exports the whole *Summary* page as a PDF file. - **Export PDF (Short)** exports a shortened version, with the current [layout](#layouts-). - Downloading samples (both extracted files and original samples) always uses ZIP archives (optionally password-protected, compatible with modern ZIP tools such as `unzip` 6.00+, Python's `zipfile`, 7-Zip, or `bsdtar`). - Sample management options: - editing classification or tags - subscribing and unsubscribing - deleting the sample PDF reports have a retention period of 30 minutes and will not reflect changes that happened after they were generated. If a sample’s classification changed after the PDF report was already created, users must wait for the retention period to expire before requesting it again or use the [PDF Report API](../API%20Documentation/pdf-report-api) endpoints to immediately generate and download an updated PDF report. **Warning: Because some PDF viewers automatically convert all strings with an `http[s]*` schema into clickable hyperlinks, it is not recommended to click any links in the generated PDF as they may lead to malicious content.** [Cloud](#cloud) samples: Only subscribing and unsubscribing is available. [URLs](#network-threat-intelligence): - Reanalyze - Download options: - Payload: scraped content (if you used *local analysis* when submitting the URL). - Screenshots and dropped files: OS artifacts taken from [Analysis Service Integrations](../dynamic-analysis#reversinglabs-cloud-sandbox). The artifacts are in a 7z archive (password: `infected`). ### Layouts ⚙ **Info: This button is available only on local samples.** Click ⚙ and, from the drop-down list, select one of the preconfigured layouts of the report summary page. Optionally, click **Manage Layouts** to edit the existing layouts or create your own. For more information, see [Layouts Editor](/SpectraAnalyze/Administration/users-personalization/layouts-editor/). ------- --- ## Network Threat Intelligence Page ![description](images/url.png) The Network Threat Intelligence sample details pages are reserved for URLs, IP addresses and domains. They can be accessed by clicking any submission in the Network tab on the Search & Submissions Page, by clicking the Network Threat Intelligence link in the Sample Summary header of samples that correlate to some network resource, or by clicking the Network Threat Intelligence link that is displayed in the search box if the search query contains a single URI. The report summary section is an overview of all information available for a specific network resource, with additional information accessible using the sidebar menu. **Info: If Spectra Intelligence is unreachable, misconfigured, or not configured, the Network Threat Intelligence page displays a corresponding error message indicating the issue. Ensure your Spectra Intelligence account is properly configured to access network threat intelligence data. For more information, see [Spectra Intelligence configuration](../../Administration/configuration-update/configuration#spectra-intelligence).** The Network Threat Intelligence sidebar menu section contains the following items: - URL Analysis Contains the top threats found on the URL, as well as historical data for that URL. This data comes from the ReversingLabs [Network Threat Intelligence API](https://docs.reversinglabs.com/SpectraIntelligence/API/NetworkThreatIntel/tca-0403/). - Domain Analysis Contains the top threats found on the domain, as well as historical data for that domain. This data comes from the ReversingLabs [Domain Threat Intelligence API](https://docs.reversinglabs.com/SpectraIntelligence/API/NetworkThreatIntel/tca-0405/). - IP Analysis Contains the top threats found on the IP address, as well as historical data for that address. This data comes from the ReversingLabs [IP Threat Intelligence API](https://docs.reversinglabs.com/SpectraIntelligence/API/NetworkThreatIntel/tca-0406/). - Screenshots & DOM The Screenshots & DOM section shows the raw Document Object Model (DOM) of the analyzed page, a list of extracted links grouped by type (link, script, image, other), and a screenshot of the page as it appeared during analysis. This allows users to review both the underlying code and the rendered view in a single place. ![description](images/url-dom-screenshot.png) In addition to network threat intelligence, users can also inspect previous ReversingLabs Cloud Sandbox dynamic analysis results, submit the network resource for (re)analysis, or interact with related files. --- ## Sample Details Summary ## Report Summary The **Report Summary** page highlights the most interesting information from the sample analysis report. It contains several sections with links to more detailed information. Some parts of the page (e.g. threat name, file format, etc.) are clickable links that, when clicked, automatically perform search queries. The information on the Summary page will be different for every sample analyzed on the appliance, depending on the file type and classification status. For more information on interpreting the Sample Summary page, see [Interpreting analysis results](../index.md#interpreting-analysis-results). ### Report Summary Header ![Report Summary page header](images/local.png) The sample summary landing page starts with a header section. It is colored according to the sample’s classification, and it contains the most important information about the analyzed file. #### Risk score and threat name These tiles show the final classification of the sample. If a sample is goodware, its risk score will be in the 0 - 5 range. For suspicious and malicious files, the risk score will be in the 6 - 10 range. Risk scores depend on the severity of the threat. Read about the [ReversingLabs classification algorithm and risk score](/General/AnalysisAndClassification/Classification/#risk-score). Threat names follow our [malware naming standard](/General/AnalysisAndClassification/MalwareNamingStandard). Clicking the name scrolls the page down to the Threat Intelligence block with detailed info on the Threat family and actor and links to advanced search and external articles. If the threat is associated with a specific actor, that actor is displayed below the threat name. #### Classification Reason The ReversingLabs classification algorithm uses a number of techniques and classifiers to detect threats. This tile shows the classifier that caused the final classification of the sample. More information about classification reasons can be found in the [Threat Classification Descriptions](./threat-classification-sources.md#threat-classification-descriptions) section. The list of classification reasons can be expanded by enabling the [Risk Tolerance](../Appendix/risktolerance.md) preview feature. When Risk Tolerance is enabled, a header indicator will display **"Risk Tolerance: Low"** to inform users when classification results are influenced by Risk Tolerance. #### Detections Shows the number of Community Threat Detections and YARA matches for the sample. Click the tiles for more information. #### Community Threat Detections The number of community threat detections for this sample. #### YARA Matches The number of YARA rulesets that matched this sample. Visit the [YARA](../yara.md) chapter for more information on using YARA for threat hunting on Spectra Analyze. --------- Below the header, the page displays a list of available or configurable analysis methods for the sample. Methods can be clicked to open a menu with more details or analysis options. The method responsible for the sample's classification is highlighted in the final classification color and labeled as (decisive). Not all analysis methods can be decisive for the final sample classification. These are indicated by a (i) tooltip icon saying "Does not impact final classification." Analysis methods that are configured but not yet performed display an Analyze button, while unconfigured methods offer a link to their configuration page. ### Customizing the Report Summary page The remainder of the report summary page contains **data blocks**. Each data block contains a shortened report for a specific subject, such as sample attribution and threat intelligence, relationship graphs, or static analysis insights. The layout of data blocks is [configurable](/SpectraAnalyze/Administration/users-personalization/layouts-editor/). When you hover over the title of a data block, a short explanation is presented in the tooltip. ## Sample Details The **Sample Details** report section is broken into multiple tabs containing more information about the sample. Tab sections containing a large amount of entries will be paginated, and can be navigated using the arrow buttons in the top right corner of the section. ### Sample Details (tab) **Tags and Uploads** - lists a sample’s System and User Tags. Clicking the *Add* link opens a dialog where it is possible to add tags to the sample. Each tag is clickable and, when clicked, automatically performs a search query for samples based on the selected tag. **Capabilities** are displayed as a horizontal bar with icons, but only for supported file formats. The same information can be found in the *Static Analysis > FileType > Capabilities* section. **Hash values** for the sample. Hovering over any of the hashes displays a button to copy the hash to clipboard. SHA1 and SHA256 are shown by default, and the section can be expanded to show the remaining hashes (MD5, IMPHASH, SHA512, SSDEEP, TLSH). Hovering over hash values displays a tooltip button that can be clicked to copy the hash to the clipboard. **Malware Description / Sample Description** shows a short textual summary (“Storyteller”) that describes the sample’s properties in a user-friendly language. The Storyteller can contain links provided by [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis). Each link leads to the Search & Submissions page, automatically performing a search query for samples based on the selected IOC. Links are also part of the exported PDF report (except those leading outside of the Spectra Analyze instance). **Show AI Summary** button provides an LLM-generated overview covering the sample’s behavior, classification, and an executive summary. This feature is disabled by default. Contact [ReversingLabs Support](mailto:support@reversinglabs.com) to enable it. In addition to the Storyteller, this section contains the same malware family/type description from the Threat Intelligence Cards on the Advanced Search results page. Clicking the See Full Details link in the section header opens the Advanced Search results page for the sample’s threat name. If a sample is not malware, this section is titled Sample Description, and does not contain threat intelligence information about malware type or family. Storyteller text is present in all cases. **Comments** This section displays comments that have been added to a sample, either by the user who uploaded it or by other users. The appliance can also automatically add comments to a sample; for example, when a user manually changes its classification. ### How We Caught This **Prevalence** The graphs in the *Prevalence* section track the history of malware detections for the sample. The first graph, *Threat Reputation Scans*, shows the percentage of Threat Reputation scanners that have detected the malware over time. The *Malware Prevalence* graph shows how many overall samples have been detected as belonging to the malware family over the last N months, with the total number of unique samples shown underneath it. Hovering over the dots in the graph displays the exact dates and the amount of detections for each date. The *Total* amount displayed below the graph refers to the current month. The *Uploads* list shows the users who have uploaded the sample, and the number of times the sample has been uploaded. Clicking the number redirects to a search results page with all upload records of this particular sample. The next part of the *Summary* page contains information about malware prevalence and the results of the RHA File Similarity ([RHA File Similarity](../search-page.md#analyzing-files-with-rha-similarity-algorithm)) analysis, extracted files and the timeline. **File classifications** - displayed for samples that have been classified as suspicious or malicious. The *Threat Name* information indicates which threat names and classification statuses the sample has received from which classifier. The *File* information indicates the files within the sample that have been classified with each status and threat name. Clicking a link under *File* opens the Sample Details page for the selected file. ![Sample Details Summary showing the prevalence graph and the file similarity chart](images/analyze-sample-details-summary3.png) **Extracted Files** The *Extracted Files* section displays the information about the number and type of file(s) extracted from the sample. If a sample has no extracted files, this section is not displayed at all. If there have been issues while extracting the file, this is indicated by “Failed” and “Partial” status labels on the right side. **Additional sections** Additional sections may be present on the *Summary* page depending on the metadata extracted from the sample during analysis: - **Files with indicators** - **Files with signatures** - **Files with descriptions** - **YARA matches** - displayed if the sample matches any of the YARA rules on the appliance. - **Network references** - samples analyzed by dynamic analysis services can have a *Network connections* section. Clicking any link in the *File* column redirects to the *Interesting strings* section of the file associated with that network reference. ![Network References section of the Sample Details Summary page](images/analyze-uri-stats-network-references.png) - **Embedded scripts** - offers quick access to the contents of scripts found in the sample. The “eye” icon next to each script opens the floating [File Preview dialog](./file-preview.md). Clicking any of the links (embedded script names) in the *File* column opens the Sample Details page for that script. ![Embedded Scripts section of the Sample Details Summary page](images/analyze-sample-details-embedded-scripts.png) - **Malware configurations** - samples with detected malware configurations will have a section with URI stats. Clicking any of the URIs opens the search results page for that URI. ![Malware Configurations section of the Sample Details Summary page](images/analyze-uri-stats-malware-configurations.png) ### Analysis Results **ReversingLabs Analysis** is a collapsible table showing an overview of sample processing activities and results. The table records and displays the last time when the sample was (re)analyzed by ReversingLabs services - Static Analysis (Spectra Core), Threat Intelligence (Spectra Intelligence) and the ReversingLabs Cloud Sandbox (dynamic analysis). The method that resulted in the final classification of the sample will be highlighted in the appropriate classification color and marked as the **Decisive Classification Reason**. This table is collapsed by default, unless processing is still taking place, the sample is missing Spectra Intelligence results, or there is a special point of interest like the sample classification coming from a Local / Spectra Intelligence user override, an extracted file or from goodware overrides. When applicable, these special/additional classification reasons will be highlighted in a colored bar at the top of the table. Depending on the current status of the sample and the last analysis time, the table will offer useful suggestions, indicators and possible error messages. For example, it will display a message if Spectra Core was updated to a newer version since the last analysis, if the sample might have outdated Spectra Intelligence classification data, or if the file type is not supported by the Cloud Sandbox. The far right of the column contains context-dependent buttons for reanalyzing the sample, uploading it to the cloud, or accessing the Spectra Intelligence configuration page. Sample classification can be manually changed by clicking the **OVERRIDE** button in the top right corner of the table. See [Setting Custom Classifications](./threat-classification-sources.md#setting-custom-classification) for more information. If the appliance detects a possible discrepancy between the Spectra Core and Spectra Intelligence classification results, it will automatically submit the sample for reanalysis. This functionality is available only when the *Enable ReversingLabs File Reputation* option is active in **Administration ‣ Configuration ‣ Spectra Detect Processing Settings**. The classification will not necessarily change after reanalysis. Keep in mind that samples previously processed by Spectra Intelligence will still retain their analysis result in the table even after Spectra Intelligence configuration is disabled on the appliance **Integrations Analysis** is a collapsible table showing an overview of dynamic analysis services and results. Items in this section do not affect the final classification of the sample, but can be useful as additional sources of metadata. The table records and displays the last time when the sample was (re)analyzed by integrated dynamic analysis services, if any are configured. The far right of each table row contains context-dependent buttons for reanalyzing the sample or, if the service is not configured, accessing the configuration page for the respective service. **Classification Timeline** The *Timeline* provides a historical overview of activities performed on the sample. It records the dates when the sample was first seen by Spectra Intelligence and on the local appliance, when it was classified by Spectra Core, (re)analyzed by static or dynamic analysis services, and more. ### Attribution Data The Attribution Data page provides comprehensive threat intelligence about the malware family associated with the analyzed sample. This page offers deeper insights into the threat's origins, behavior, and related indicators. **Important: The information on this page is predominantly LLM-generated, indicated by the star icon next to the section title. Review the information for accuracy before relying on it.** ![Attribution data page](images/attribution-data.png) The page is organized into several sections: #### Malware Family Core identification details including threat name, aliases, first/last seen dates, current version, packer information, targeted operating systems, and industries. **Detection Statistics** - Weekly ranking, total detected samples, and week-over-week detection trends. **Top File Types** - Distribution of file formats commonly used by the malware family (e.g., PE executables, DLLs, archives, Office documents). **Malware Family Prevalence** - timeline graph showing detection activity over the past six months. **Threat Description** - A detailed narrative explaining the malware's history, capabilities, infection vectors, persistence mechanisms, and communication protocols. **References** - External sources and research publications for further reading. #### Sample Attribution This LLM-generated section maps the sample to known threat actors and campaigns: **Actor Name** - The threat actor or group attributed to the malware. **Campaigns** - Known malicious campaigns associated with the sample. **Victims** - Targeted entities or sectors. **Vulnerabilities** - CVEs or weaknesses exploited by the malware. **Sources** - Intelligence sources supporting the attribution. #### Actor Insights - Actor description and operational history - Aliases used across different threat intelligence vendors - Country of origin (if known) - Group type (e.g., cybercrime group, APT) - Known tactics employed by the actor #### Actor Related IOCs A table listing indicators of compromise (IOCs) linked to the actor, including IP addresses, domains, and other artifacts. Each entry shows the indicator name and its type. #### Actor Related Campaigns A table of campaigns attributed to the actor. Each campaign can be expanded to reveal additional details: - **Date** - When the campaign was active. - **Techniques** - Tactics and techniques used during the campaign. - **IOCs** - Associated indicators including hashes, URLs, IPs, and domains. #### Actor Related TTPs A table of tactics, techniques, and procedures (TTPs) associated with the actor. Each entry includes: - **Name** - The technique or tool identifier. - **Description** - How the actor employs the technique. #### Related Vulnerabilities A list of vulnerabilities exploited by the actor. Each entry shows the CVE identifier and a brief description of how it was used. #### Referenced Actors Connections to other threat groups. #### Sources Intelligence reports and documentation used to generate the attribution data. ### Network References The network references tab contains all URLs, IP addresses and domains found *in relation* to the sample. This includes network locations that the file attempted to visit during a dynamic analysis run, IP addresses found inside the sample during static analysis, as well as places where the URL was found (sources). ### MITRE ATT&CK **MITRE ATT&CK** is a section mapping indicators detected by Spectra Core to MITRE threat IDs. This section can be displayed for all samples regardless of their classification status (malicious / suspicious / known / no threats found), as long as they have indicators that can be appropriately mapped to the ATT&CK framework. Samples without indicators will not have this section on their Sample Details page at all. The section lists MITRE tactics in the table header. MITRE techniques are grouped under each tactic. Every technique can be clicked to show Spectra Core indicators mapped to it (which can then be clicked to run an Advanced Search for samples with those same indicators). The table can be further filtered to show all techniques or just the detected ones, and technique IDs can be either displayed or hidden. --- ## Sources The *Sources* page displays different types of sources for the selected sample. Sources are categorized into four tabs, each tab showing relevant information about the origin of the sample. The Spectra Intelligence tab shows a list of all locations from which the sample was retrieved, as well as time and date of retrieval. If available, it also shows additional information about the sample, such as versions or descriptions. The Network References tab shows a list of all locations from which the sample was retrieved, and the Relationships tab shows the top-level containers and direct parents that contain the relevant sample. ![Sample Details page with visible Sources](images/analyze-sources.png) --- ## Spectra Core - Static Analysis Results The *[Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis)* page visualizes the static analysis report for every sample. The information is organized into sections which can be expanded using the accordion menus. The amount of information and the available sections vary based on file type. ## File information and statistics The *Info* section displays basic information such as file type, predicted filename (if it exists), file size, and entropy; file validation; the set of hashes computed for the sample; as well as statistics about other files contained within the original sample. If there are any errors or warnings detected during sample analysis, they are listed as individual entries in this section. ## File type-specific information This section contains detailed information about the sample based on the metadata extracted by Spectra Core. As the name implies, the information in this section depends on the file type of the sample. In the example of an Application (Portable Executable), the following information may be included: ### Identity Application identity takes whitelisting and certificates into consideration and labels applications with their correct software versions. It removes the ambiguity left by source and certificate whitelisting as it can pinpoint the exact software release. Furthermore, it can identify the correct identity for software that isn’t typically signed or published on a reputable source. ### Capabilities Capabilities are actions the sample can perform without executing an application. Insight into capabilities is obtained exclusively through static analysis of the sample. **Tip: To find samples that exhibit specific capabilities, use the **tag** keyword in [Advanced Search](../advanced-search.md). For example, *tag:capability-cryptography* will find samples that can encrypt data. Consult the [full list of supported tags](../advanced-search.md#supported-tags).** ### Analysis Shows the security grade and detected security issues for the analyzed sample. Depending on the security issues, the sample can get one of the following grades: - Grade A. Best security grade. The application follows the latest standards and policies. - Grade B. Good security grade. The application has sufficient security mechanisms implemented, but does not have all the latest features enabled. - Grade C. Minor security issues detected. The application has some security mechanisms implemented, but is not considered safe to use in all environments. - Grade D. Major security issues detected. The application should only be executed in secured environments. - Grade E. Major security issues detected. The application should only be executed in highly secured environments. - Grade F. Major security issues detected. Consider the application unsafe to run. ### DOS Header Historical header that serves as a pointer to the *File Header* through *e_lfanew* entry. ### File Header Important entries in the *File Header* include: - Machine - target architecture for which the PE file was compiled - Time Date Stamp - date when the PE file was compiled - Characteristics – PE file attributes - Number of Sections, Number of Symbols ![Sample Details page with visible File Header](images/analyze-sample-details-header.png) ### Optional Header Describes elements such as import and export directories that make it possible to locate and link DLL libraries. Other entries provide structural information about the layout of the file, such as the alignment of its sections. ### Sections Describes each of the sections making up the file. Sections can contain code, initialized and uninitialized data, resources and any other data. The following information may be of particular interest to malware analysts: - Flags show whether the section contains executable code, can be read from, written to or has other properties - Hashes can be used to correlate with other files - Entropy can show if a section is encrypted or compressed **Note: **Physical Size** does not always match the raw size value in the PE header section. It is derived from actual loader behavior and the resulting bytes of the file as loaded into memory.** Section hashes are calculated using the Physical Size, which may differ from hashes computed with the raw size. ### Imports Contains an array of import directory entries; one entry for each DLL to which the file refers. Every entry can be expanded to reveal the list of symbols that are being imported. ### Resources Indicates what resources the file contains, together with all the details about them (such as type, language, and whole resource data hashes). The language can be an indicator of the machine locale settings used by the person who developed and/or compiled the file. Hashes can be used to look up and correlate which files contain the same resources. Additional information can include **version info, dynamic libraries, symbols, segments**, and more, depending on the file type. If the sample is an email message, the sections can include information about from, sender, and reply-to email addresses, email message subject and headers, as well as attachments (if there are any). If the sample is an image, EXIF metadata will be extracted and included in the sections (for example, camera make and model). For mobile applications, the sections can include information about the application package, activities, services, receivers, and permissions. Currently supported mobile platforms are iOS, Android, and Windows Phone. ### Software Packages Relevant information for files recognized as software packages, which are archive files containing an assortment of individual files or resources and related metadata (such as name, vendor, version number, version number) that work together to provide users with a particular functionality. For the full list of supported package formats, refer to [this article](https://docs.secure.software/concepts/language-coverage). ## Signatures The *Signatures* section contains tabbed information about signatures, digital certificates and their validation states reported by the Spectra Core engine, including the certificate trust chain with signer and counter-signer details. The chain of trust starts with a certificate authority and ends with the signer. Clicking any of the individual elements in the signer/counter signer chains will show more detailed information below. In case there are multiple signatures found, results will be paginated. ![Sample Details page showing a Certificate Trust Chain of a sample](images/analyze-sample-details-certificates.png) Spectra Core supports classifying samples based on digital certificate blacklists and whitelists. By default, it provides more than 300 CA (Certificate Authority) certificates in its certificate store. The certificate store is a set of trusted CA certificates imported from sources such as Microsoft Windows, Mozilla Firefox, and Apple, which are included in Spectra Core and, by extension, in the Spectra Analyze system. Samples classified on the basis of their certificates receive the “Classified by Digital Certificate” [threat description](./threat-classification-sources.md#threat-classification-descriptions). Spectra Analyze also provides information about certificate status. Check the **Sample Details > Static Analysis > Info > Validation** section to see how Spectra Core validated the certificate(s) for a sample. | Certificate Status | Detailed Status Description | | --------------------------- | ------------------------------------------------------------ | | **Valid certificate** | Any certificate with an intact digital certificate chain that confirms the integrity of the signed file. The hash within Signer Info matches the hash of the file contents. | | **Invalid certificate** | Any certificate with an intact digital certificate chain, but for which the certificate chain validation failed due to other reasons (e.g. because of attribute checks). Without a valid digital certificate chain, the integrity of the signed file cannot be validated. | | **Bad checksum** | The integrity of the signed file could not be verified, because the hash within Signer Info does not match the hash of the file contents. | | **Bad signature** | Any certificate with an intact digital certificate chain, but for which the signature validation failed. Without a valid signature, the integrity of the signed file cannot be validated. | | **Malformed certificate** | Any certificate that does not have an intact digital certificate chain. The digital certificate is corrupted or incomplete, but that doesn’t mean the file is also corrupted. Without a valid digital certificate chain, the integrity of the signed file cannot be validated. | | **Self-signed certificate** | A self-signed certificate is a certificate that is signed by the same entity whose identity it certifies. In other words, this is a certificate that is used to sign a file, but is its own CA (certificate authority), and doesn’t have a CA that issued it. If CA information is present, but not found within the Spectra Core certificate store, the CA will be considered plausible and files signed with it will be declared valid (i.e., they will not be considered as self-signed). | | **Impersonation attempt** | Any self-signed certificate is a candidate for an impersonation check. Impersonation means that the signer is trying to misrepresent itself as a trusted party, where “trusted party” is defined by the certificate whitelist. Any self-signed certificate that matches the common name of another certificate on the Spectra Core whitelist is marked as an impersonation attempt. | | **Expired certificate** | Any certificate with signing time information is checked for expiration. When the time on the local machine indicates that the certificate has passed its “valid to” date and time, the certificate is considered expired. The “Expired” certificate status is merely informative, and expired certificates cannot influence certificate classification. | | **Untrusted certificate** | Any valid certificate for which the digital certificate chain cannot be validated against a trusted CA. Untrusted certificates are valid certificates, but they cannot be whitelisted because their chain does not terminate with a CA in the Spectra Core certificate store. | **Tip: To find samples by their certificate status, use the **tag** keyword in [Advanced Search](../advanced-search.md). For example, *tag:cert-invalid* will find samples signed with invalid certificates. Consult the [full list of supported tags](../advanced-search.md#supported-tags).** ## Indicators Indicators are extracted by Spectra Core during static analysis and displayed as human-readable descriptions of sample behavior. There are many indicators that are common in regular applications, like opening files, writing to files, and so on. The full list of indicator IDs and their descriptions can be found [here](/General/SpectraCore/indicators/). The *Indicators* section shows what a sample is capable of doing, with more significant indicators listed first. If static analysis indicates that an application contains an encrypted executable or that it is capable of accessing passwords, those indicators will get much higher priority than if the application can just open files. The former indicators are more important in this context and not common at all in legitimate applications. Therefore, they will be listed before other indicators. ![img](images/analyze-sample-details-indicators.png) Spectra Analyze displays special icons next to indicators that contributed to the final classification by a Machine Learning model. This applies only when Machine Learning is among the engines that classified the file, and is limited to Worm, Ransomware, Keylogger, and Backdoor (RAT) malware types. This indicator is not be displayed if classification is propagated, or if Machine Learning is not on the list of engines used to classify the sample. **Tip: To find samples with specific indicators, use the **tag** keyword in [Advanced Search](../advanced-search.md). For example, *tag:indicator-macro* will find samples that contain or execute macros. Consult the [full list of supported tags](../advanced-search.md#supported-tags).** ## ATT&CK The ATT&CK section maps indicators detected by Spectra Core to MITRE threat IDs. This section can be displayed for all samples regardless of their classification status (malicious / suspicious / known / no threats found) as long as they have indicators that can be appropriately mapped to the ATT&CK framework. Samples without indicators will not have this section on their Sample Details page at all. MITRE tactics are listed as table columns, and MITRE techniques are grouped under each tactic. Every technique can be clicked to show Spectra Core indicators mapped to it. The same technique can be listed under multiple different tactics. The mapping is limited to indicators that Spectra Core can detect with static analysis, so it does not cover the full range of MITRE tactics and techniques, but only a subset of it. Buttons above the table can be used to filter Techniques to only those that were triggered, and to show or hide technique IDs. ## Classification If the sample has been classified, this section shows its status and a list of scanners that determined the classification. If the sample has been classified by a YARA rule, this section contains the relevant YARA ruleset metadata. More information on sample classification can be found on the [Sample Details Summary](./sample-details-summary.md) page. To find out more about classification methods and reasons, consult the [Threat Classification Descriptions](./threat-classification-sources.md#threat-classification-descriptions) chapter. ## Protection Shows the protection features with which this file was compiled, and other protection mechanisms that were detected while analyzing the file (such as cryptographic or compression algorithms). ## Strings and Interesting strings Spectra Analyze extracts all strings from samples and separates *Interesting strings* into their own section. Strings are considered interesting if they contain information related to network resources and addresses (for example IP addresses, HTTP, HTTPS, FTP or SSH). Interesting strings are usually found in binary files, documents and text files. If available, reputation statistics are displayed next to URIs in the *Interesting strings* section. In both sections related to strings, results can be searched with regular expression patterns and filtered by string length for more dynamic hunting. Additionally, Interesting Strings can be filtered by category (all, http, https) and by classification (all, malicious, suspicious, goodware, no threats found). Strings can be filtered by their origin. - CARVED - String is generically extracted from the file. - TABLE - String is found within a file format specific strings table. - DEEP - String is found within a compressed part of the file or a non-offset accessible location. Any of these strings can be deemed human readable by a ML model. This filter is available as a separate button. ![Sample Details showing string filtering options](images/analyze-filter-strings.png) Very long strings can exceed the display length and appear as if they are cut off. To see the entire string, click the **Show more** button. ## Tags When Spectra Core analyses a file, it assigns it different [tags](../tags.md) which help when searching for specific capabilities. ## Email If the analyzed file is an email, the structure extracted by Spectra Core will be presented on this page. This includes email headers (subject, sender, date...). The content of the email itself is available on the [preview page](file-preview.md). The attachments are available in the [extracted files](extracted-files.md) section. --- ## Threat Classification Sources In addition to classification from the [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) engine included in the Spectra Analyze appliance, the threat classification for samples comes from either the Spectra Intelligence service or an on-premises T1000 File Reputation Appliance. Appliance administrators should configure access to the desired reputation service (Spectra Intelligence or T1000) via the [System Configuration](/SpectraAnalyze/Administration/configuration-update/configuration/) page. **Only one of those services should be configured, not both. If both are configured, the T1000 service will be used by default.** If the appliance is not connected to the Spectra Intelligence or to the T1000 appliance, then no classification or AV scan information (malicious, suspicious, known) will be present. However, classifications will still come from Spectra Core (RHA1, signatures and so forth), which is locally installed on the appliance. When either the Spectra Intelligence or the T1000 reputation service is properly configured, the sample threat status will incorporate data from Spectra Core and from the configured reputation service. ## Threat Classification Descriptions The following is a list of all possible classification reasons a sample can have. The classification reason specifies which technology detected this threat. It is visible on the [landing page of the Sample Summary page](./sample-details-summary.md) and in the expanded row on the Search & Submissions page. Files will usually have multiple detections from more than one classifier, but the classification reason tile on the Sample Summary page always shows the one that produced the final classification. **Note: The list of classification reasons can be expanded by enabling the [Risk Tolerance](../Appendix/risktolerance.md) preview feature.** ### **Classified by: Threat Reputation** The sample was classified by the Threat Reputation scanner. ### **Classified by: Certificate Validation** Digital signatures include a file integrity validation hash. Validating digital certificates is a multi-step process. Valid certificates have a properly formed digital certificate chain and pass file hash integrity validation. Spectra Core detects signed file tampering and is capable of detecting signer impersonation, certificate malformation and content modification. Failing to comply with any of these checks will classify the file as at least suspicious. The displayed threat name will reflect the detected type of the tampering attempt. When a self-signed certificate is trying to misrepresent itself and emulates a trusted certificate, the displayed threat name will be `{Platform}.Certificate.Impersonation`. On the other hand, when a file fails integrity validation, the threat name can appear as `{Platform}.Certificate.Invalid` or `{Platform}.Certificate.Malformed`. In case of valid signing time, with signature that is created after signing certificate is already expired or revoked by Certificate Authority, threat name will be `{Platform}.Certificate.SignedAfterExpiration` and `{Platform}.Certificate.SignedAfterRevocation` respectively. ### **Classified by: Threat Signature** Rules, Indicators, Classifications and Capabilities (RICC) is an offline database that applies static analysis rules to analyzed content. Part of its responsibility is to classify files based on signatures and unique metadata properties found only in malicious files. Two such classification technologies are deployed through RICC. Byte Pattern Matches as signatures that detect known threats, and Malware Artifacts Classifier that looks at the metadata for malware clues. Both of these technologies correlate the detection to a named threat. In terms of classification, they are the most specific detection technologies within the engine, and are reserved to be used only for precise threat detections. ### **Classified by: Format Structure** The sample was classified based on the format structure of the file determined by Spectra Core. Spectra Core contains a large collection of file format parsers. Some file formats are used exclusively by malware, while other, “legit” file formats, can be structured in a suspicious manner. Samples containing those file formats receive this classification. Unlike file or threat reputation results, this classification method is not constantly updated. New information and signatures for this method can only be obtained by updating the entire appliance to a new release. ### **Classified by: Email Contents** Email messages are stored in structured file formats. This encapsulation includes email headers, message body and a number of attachments. Any of these components can be malicious and therefore needs to be inspected. Email headers are checked for identity misrepresentation that relates to phishing and BEC attacks. Message bodies are inspected for URLs that could lead to phishing and malware downloads. Attachments are decomposed through static analysis in search for malicious code. Additionally, any attached file is also inspected for embedded URLs that themselves are checked for malicious intent. When this technology detects phishing, it will name the threat as `Email.Phishing.{ServiceName}`. The following services can be identified: Adobe, Amazon, AmericanExpress, Apple, BankOfAmerica, ChaseBank, DocuSign, Dropbox, Ebay, Facebook, Google, LinkedIn, Microsoft, Netflix, PayPal, Twitter and WhatsApp. If the email was detected as malicious due to embedded URL, the threat name can appear as `Email.Hyperlink.Homoglyph` ### **Classified by: Exploit Signature** During engine analysis, parsed format structure is validated and any departures from specification are reported. Detected malformations are automatically mapped back to exploits that are known to abuse format parsing bugs. Exploit detectors are a special kind of signature detection. They are implemented individually for each supported format, and are made to detect known exploits. Exploit detection is available for images, documents, archives and mobile application package formats. When an exploit is detected within an image format, the reported threat name can be `{Platform}.Exploit.CVE-{ID}`. ### **Classified by: File Contents** During automated file extraction, the supported formats are decomposed recursively. Unexpected format combinations can be discovered during extraction. For example, documents and multimedia files should never embed executable files. If such unusual format combinations are discovered, the engine will declare those files as suspicious with the following threat name: `{Platform}.Format.Content`. ### **Classified by: Explainable Machine Learning** Machine learning is a predictive detection technology. Explainable Machine Learning, a concept unique to ReversingLabs, bases its classification on the principles of expandability, transparency and relevancy. Based solely on human readable indicators, machine learning models detect specific threat types and can differentiate between threats and benign files. When the machine learning model predicts that a threat type falls into a recognized category, it will name the threat as `Win[32|64].{ThreatType}.Heuristic`. However, if the model is certain that the file is a threat, but can’t place it into a threat category, it will name the threat as `Win[32|64].Malware.Heuristic`. Machine learning models are made to detect Windows executable and fileless malware types. ### **Classified by: File Similarity** ReversingLabs Hashing Algorithm (RHA1) is a proprietary functional file similarity algorithm. It is primarily designed for executable formats, and as such it is specifically implemented for each supported format. RHA1 converts functional file features, both the code and its layout, to four precision level hashes. Each precision level represents a 25% increase in similarity between files that share the same hash at the same precision level. Lowest precision is 25% and highest is a 100%. Spectra Core comes with an offline database of blacklisted RHA1 hashes. This technology is capable of detecting polymorphic threats and their variants. Even though threats are detected based on similarity, they are still named after the threat the file is most similar to. ### **Classified by: Embedded HyperLink** Many file formats enable active linking to content hosted on remote servers. These are commonly referred to as hyperlinks or uniform resource locators (URL). Since the active content is on a remote server, it can change at any time. However, some URLs themselves do contain information that helps to infer the content type to which they are pointing to. With static analysis, Spectra Core can detect various kinds of deceptive links without visiting the content targeted by the URL. Attacker techniques such as typosquatting, domain spoofing, and homoglyphs are detected for more than 5000 popular websites. In addition to deceptive links, the solution includes an offline database of blacklisted domains and known malicious URL patterns. When the engine finds an embedded link that points to a blacklisted domain, it will name the threat as `{Platform}.Hyperlink.Blacklisted`. ### **Classified by: Goodware Override** Any files extracted from a parent and whitelisted by a high trust certificate, source or user override can no longer be classified as malicious or suspicious. Goodware Overrides propagate from parents to children. The goodware overrides will not apply if the extracted file is found by itself (for example, if an extracted file is rescanned without its container). ### **Classified by: Extracted Files** The classification of certain samples originates from children samples extracted during analysis. It propagates from children to the parent, for example from a malicious executable file to the zip archive where it originated from. If this is the case, the description beneath the classification will highlight that it was based on an extracted file. ### **Classified by: YARA Rule** The sample was classified based on YARA rules provided by ReversingLabs or by the user. ### **Classified by: NextGen Antivirus** Sample was classified by an antivirus engine considered to be next-generation, such as those that use machine learning. ### **Classified by: File Reputation** The sample was classified by Threat Intelligence Database. It can be obtained from a trusted source, or it was unpacked from the file originating from a trusted source. ### **Classified by: User Classification** The sample has been manually classified locally or in the Spectra Intelligence cloud, overriding any classification from ReversingLabs. Local classifications only apply to the Spectra Analyze instance they are set on, while cloud overrides get synchronized to other ReversingLabs appliances using the same Spectra Intelligence account and to other cloud accounts with the matching company username segment (u/**company**/user) ### **Classified by: Spectra Core Classification** The sample was classified by the Spectra Core static analysis engine. ### **Classified by: Certificate Reputation** Applications, archives, documents and software packages can all be digitally signed. These signatures guarantee integrity and certify the origin of the content they are signing. Spectra Core comes with a customizable list of signers, or identities, that own recognized certificates. These identities can be added to either the Spectra Core certificate blacklist or whitelist. The former will declare signed content as malicious, while the latter will classify analyzed content as goodware. When a file is declared to be malicious due to a blacklisted certificate, the threat name will be displayed as `{Platform}.Certificate.Blacklisted` ### **Classified by: Sandbox** The sample was classified by the ReversingLabs Cloud Sandbox. ## Setting Custom Classification Users can manually override a sample’s classification: - by selecting the **Set classification** option in the actions menu (☰) in the [Results List](../search-page.md#results-list) list, or in the *ACTIONS* menu on the [Sample Details](./sample-details-summary.md) page - by selecting the **OVERRIDE** option in the *ReversingLabs Analysis Table* on the Sample Details page Selecting either of those options opens the *Set classification* dialog. It consists of two tabs: **Spectra Intelligence** and **Local**. They are identical in design, but behave differently. Local classification overrides persist only on the current Spectra Analyze appliance, they are not propagated to any other service or Spectra Analyze instance. Spectra Intelligence overrides get synchronized to other ReversingLabs appliances using the same Spectra Intelligence account, but also to other Spectra Intelligence accounts with the matching company username segment (u/**company**/user). Samples that have both the local and the Spectra Intelligence override will be classified according to the local override. Users can change the sample classification by modifying the threat status and trust factor values of the sample. Clicking **Save** in the dialog applies the new classification to the sample. ![Set Classification dialog with options to modify the risk score](images/analyze-set-classification.png) For every sample classified in this way, manual classification changes are recorded in the **Sample Details > Timeline** section. Any user of the current Spectra Analyze instance can manually change the classification of any file submitted to the appliance, regardless of who submitted the file. For both local and Spectra Intelligence overrides, users who wish to closely keep track of classification changes to their important files should [create an alert subscription](../alerts.md#creating-a-new-alert-subscription) to get notified as soon as there is a change (by another user or by any of the ReversingLabs classification methods). --- ## Threat Intelligence (Spectra Intelligence) The *Cloud Threat Intelligence > Community Threat Detections* page contains statistics about malware detection for the sample based on the results of Threat Reputation scanning in the ReversingLabs Spectra Intelligence system. The information is obtained from a number of AV engines and organized by the time and date of each AV scan. The upper half of the page contains a graph that shows how malware detections for the sample changed over time. More specifically, the graph indicates changes in the maximum detection rate for the sample. The graph always displays the same time range regardless of the currently selected scan date. The lower half of the page contains a list of all AV vendors used to scan the sample for each recorded scan date. If an AV engine produced a detection, the detected threat name is listed next to the AV vendor name. Every result on this page is clickable and, when clicked, automatically performs a search query. Users can navigate between recorded scan dates by clicking the arrows above the list of AV vendors, and compare the differences between each scan. The number of AV engines used can differ between recorded scan dates, depending on which vendors/engines were available at the time of each scan. Text formatting in the list of AV vendors also carries important information. - If the name of an AV vendor is greyed out, that indicates their AV engine(s) did not produce a detection for the sample. This can happen for a variety of reasons (e.g. the AV engine does not support the file format of the sample, or it did not have the relevant signature at the time of the scan). In case of goodware, this is the intended behavior. - If there is a number next to the AV vendor name, that indicates how many AV engines of that vendor were used to scan the sample. - If an AV vendor name is displayed above the black line in the list, that indicates their AV engines are considered next-generation (such as those that use machine learning). - If a detection name is red, that indicates it is a new detection for the AV vendor. This means the AV vendor did not produce a detection for the sample in any of the previous scans. - If a detection name is red with another name crossed-out next to it, that indicates the detection name has changed in the currently selected scan. This means the AV vendor previously produced a different detection for the sample. The crossed-out detection name is the old name, and the red detection name is new. - If a detection name is crossed-out, but there is no other name next to it, that indicates the AV vendor is no longer producing detections for that particular threat. Some AV vendors indicate that a threat has been detected in a sample, but they do not expose the full threat name. Their detection still influences the final classification of the sample, and in some cases affects the final threat name assigned to the sample. **Tip: The information from the Spectra Intelligence section can be used with [Advanced Search](../advanced-search.md) keywords for targeted malware hunting. Use the **av-detection** keyword with any of the detected threat names (*av-detection: Adware/Genieo!OSX*), or specific AV vendor keywords with the threat name detected by a particular vendor (*av-[vendor]/Clovis-A*). To find samples by the number of Community Threat Detections, use the **antivirus** keyword. Consult the [full list of supported search keywords](../advanced-search.md#supported-search-keywords).** ### File Similarity The *Cloud Threat Intelligence > File Similarity* section of the sidebar displays the amount of [functionally similar samples](../search-page.md#analyzing-files-with-rha-similarity-algorithm), grouped by their threat status. Clicking the links in the File Similarity section redirects to the list of similar samples, where it is possible to download them to the appliance as described in the [Downloading Files from Spectra Intelligence](../search-page.md#downloading-files-from-spectra-intelligence) section. --- ## Advanced Search ## Introduction The Advanced Search feature introduces rich metadata search capabilities on the ReversingLabs Spectra Analyze appliance, makes it easier to search across large data sets (both locally and in ReversingLabs Spectra Intelligence), and enables faster, more powerful malware discovery with increased coverage. With 100+ keywords, 30+ anti-virus vendors, 130+ sample types and subtypes and 280+ tags, Advanced Search makes it possible to build more than 500 unique search queries using Boolean operators and keyword auto-completion. Users can create targeted, multi-conditional queries and combine search criteria using logical operators to quickly identify potential threats. **The Advanced Search feature can be used to perform local searches without a Spectra Intelligence account. Using Advanced Search to retrieve Spectra Intelligence results is available to customers at additional cost. For more information, contact ReversingLabs Sales Support (insidesales@reversinglabs.com).** **Important notes about the Advanced Search feature** 1. **Different search queries return results at different speeds** - for some combinations of keywords and operators, it can take longer to load the results. To ensure quicker response times for long and complex queries, returned results may contain fewer samples than are available in the database; i.e., the service will only return the latest matches found within a reasonable timeframe. **Important: To improve search query responsiveness and performance, Cloud results prioritize *First Seen* within the last month by default. However, this may result in zero results if users specify time ranges outside this time frame. In such cases, the results page provides links to expand the search results. If the query returns some results but there are more in the previous months, clicking the link next to the query summary under the drop-down menu filters broadens the search to encompass a wider time range. Alternatively, users can set the provided drop-down filters to the desired expanded time range.** 1. **Local-only keywords will not work on the Cloud tab, as local-only keywords cannot be used to search for samples in the Spectra Intelligence cloud. Only actual file submissions will be returned as results.** Local-only keywords are: `filecount`, `tag-user`, `submission-user`, `submission-time` and `processing-status`. To perform Spectra Intelligence searches or search for extracted files, remove any local keywords from the query. 2. **The maximum length of a single search query is 1024 characters.** Queries longer than 1024 characters cannot be shared or added to Favorites. Attempting to submit queries longer than 1024 characters will result in an error. This does not apply to Bulk hash search queries. 3. **The maximum amount of Cloud results that can be returned for a search query is 100 000.** Although there may be more samples matching the query in the Spectra Intelligence cloud, the Spectra Analyze will only allow browsing through 100 000 of them. 4. **Currently it is only possible to export a single page of search results.** To export all results from the list, the user would have to browse pages one by one and manually export them. It is possible to adjust the amount of results displayed per page in the navigation bar, thus increasing or decreasing the number of results that will appear in the exported CSV file. 5. **The \*Fetch & Analyze\* option for Cloud results is currently limited to downloading 100 samples at a time, with a daily limit of 10 000 samples in total.** Samples that already exist on Spectra Analyze will not be downloaded again. It is not possible to fetch and analyze all samples in the Cloud results list at once. 6. **Large volumes of data indexed for Advanced Search in the Spectra Intelligence cloud are constantly updated in order to return the most relevant information.** During synchronization of various Spectra Intelligence services, searching for samples the cloud may return inconsistent or incorrect results in some cases. The data is updated multiple times per hour. This can cause discrepancies between the results offered on the Local and Public (Spectra Intelligence) results tabs. ## How to Write Search Queries **Local-only keywords will not work on the Cloud tab, as local-only keywords cannot be used to search for samples in the Spectra Intelligence cloud. Only actual file submissions will be returned as results.** Local-only keywords are: `filecount`, `tag-user`, `submission-user`, `submission-time` and `processing-status`. To perform Spectra Intelligence searches or search for extracted files, remove any local keywords from the query. Local-only keywords, when added using the drop-down menus, will not be shown in the Advanced Search box as part of the query, but they will still be applied to the results, saved to the *Recent queries* list, and shared using the *Share query* button. To create a search query, start typing into the *Advanced search* box. The pull-down list with all matching search keywords or their predefined values will open. The keywords are listed alphabetically. Every search query must contain at least one keyword and one value. Search queries are built according to the following formula. ``` keyword:value OPERATOR keyword2:value OPERATOR keyword3:[value1,value2,...] ``` The values for a keyword can be typed in manually, or if the keyword supports it, selected from the pull-down list. Selecting a keyword that supports predefined values (for example, *classification*, *riskscore*) displays all those values in the pull-down list. Selecting a keyword that supports date and time ranges (such as *lastseen* or *firstseen*) displays the date picker. To add a custom range to the search box, select “Custom” in the date picker and click the *Apply* button. Keywords have short usage examples in the pull-down list. For a detailed overview of supported keywords and their features, refer to the [Supported Search Keywords section](./advanced-search.md#supported-search-keywords). Some keywords have aliases - additional forms that can be used to search for the same values. Aliases are indicated in the Supported Search Keywords section in parentheses next to keyword names, and in the interface as illustrated in the screenshot below. ![Pull-down list of search keywords with aliases highlighted](./images/analyze-search-aliases-help.png) To run a search query, click the *Search* button in the search box, or press Enter. ------ The following is an example of a basic search query that returns all samples classified as suspicious: ``` classification:suspicious ``` What can and cannot be included in a search query depends on the values and operators supported by the keyword, as well as on the restricted words and characters. The maximum length of a single search query that can be entered into the *Advanced search* box is 1024 characters. ### Restricted Words and Characters All restricted words and characters should be escaped with double quotation marks in the search query. Example: a query contains one of the restricted characters **[**, **]**, **(**, **)**, **:** ``` pdb:"C:\Windows*" ``` Example: a query contains one of the restricted words (**AND**, **OR**, **NOT**) ``` cert-subject-name:"AND" ``` If the search query contains spaces, use double quotation marks around it. ``` cert-subject-org:"microsoft corporation" ``` ### Searching for Exact Matches For more precise results, use quotation marks in search queries, especially when looking for a specific string. The underscore character ( **_** ) is treated as a delimiter. Phrases containing the underscore should be enclosed in quotation marks to get exact matches. For example, searching for `pe-function:"Py_Initialize"` returns results that match the exact phrase, including the underscore character. Searching for `pe-function:Py_Initialize` returns results that match either “Py” or “Initialize”, or both. ### Using Wildcards for Partial Matching Some search keywords support partial matching with wildcard symbols. The `*` symbol matches any sequence of characters. The `?` symbol matches any single character. Example: this query returns all samples that have the string “emo” anywhere in their threat name (such as Wemosis, Remora, Temonde). ``` av-detection: *emo* ``` Example: this query returns all samples with the threat name “Emotet” and any other variant where the first letter T is replaced by any other character (such as Emonet, Emoret). ``` av-detection: emo?et ``` ### Searching for a Range and Greater/Less-Than Values For keywords that support searching for a range of values, the formula looks like this. ``` keyword:[value1 TO value2] size:[50000 TO 70000] ``` To search for greater/less-than values, create an open-ended range using the wildcard symbol `*` ``` keyword:[value TO *] - for greater-than values keyword:[* TO value] - for less-than values ``` This example returns all samples that have a trust factor lower than and equal to 4. ``` trustfactor:[* TO 4] ``` ### Searching for a List of Values To search for any of the values in a list, the following formula is used. ``` keyword:[value1, value2, value3] ``` The values must be comma-separated. ``` classification:[suspicious, unknown] av-detection:[emotet,wannacry] sha1:[91b21fffe934d856c43e35a388c78fccce7471ea,4e8c5b9fc9a6650f541fa0dbe456731309a429e4, 66720a660761e9b3b9b071ba4c16d6ab69c442bb] ``` ### Creating Multi-keyword Search Queries Search operators and parentheses can be used to combine multiple keywords and create advanced search queries. The following search operators are supported: **AND**, **OR**, **NOT** If an operator is not provided, **AND** is used as the default. Operators are case-insensitive, so the following queries all return the same results. ``` firstseen:2018-01-01T00:00:00Z AND classification:malicious firstseen:2018-01-01T00:00:00Z and classification:malicious firstseen:2018-01-01T00:00:00Z classification:malicious ``` The **NOT** operator excludes search results that match the search criteria. In the following example, malicious and suspicious files will be excluded from the results: ``` av-detection:*linux* NOT classification:[malicious, suspicious] ``` The **OR** operator can be used to look for any of the values supported by a single keyword: ``` classification:suspicious OR classification: malicious ``` It can also be used to look for any of the different keywords and their values: ``` pdb:JigsawRansomware.pdb OR uri:"http://btc.blockr.io/api/v1/" ``` The **OR** operator cannot be used instead of a comma when searching for a list of values. The following example is **not a valid query**: ``` av-detection:[emotet OR wannacry] ``` **Parentheses** can be used to combine keywords. The following two queries show how to format the same request using square brackets versus parentheses: ``` firstseen:2018-01-01T00:00:00Z av-detection:[trojan,wannacry] firstseen:2018-01-01T00:00:00Z (av-detection:trojan OR av-detection:wannacry) ``` Apart from using parentheses with the same keyword, they can be used to combine multiple different keywords, operators, and even a range: ``` firstseen:2018-01-01T00:00:00Z (av-detection:trojan AND type:binary NOT positives:[* TO 3]) ``` ### Saving and Sharing Search Queries There are several ways to save search queries on the Spectra Analyze appliance. 1. Search queries can be saved as Favorites on the Spectra Analyze appliance itself. Run any query and click the star button right of the search box to save it. The query will be listed under *Favorites* in the [Suggestions menu](./search-page.md#show-suggestions). It can be modified to include other search keywords and parameters, or removed from the appliance at any time. The maximum of 20 search queries can be saved in this way. 2. Search queries can be saved using the built-in bookmarking functionality of the web browser. Run any query and bookmark the results page. In this case, any active filtering parameters (such as sorting and number of results per page) are also preserved in the bookmarked URL. A search query saved in this way will only work on the Spectra Analyze instance specified in the bookmarked URL. Similarly, search queries can be shared in several ways: 1. by using the *Share query* option on the Spectra Analyze appliance. Type in any query and click the *Share* button right of the search box. The *Share Query* dialog opens, where recipient email addresses have to be entered. Clicking the *Share* button in the dialog will send the selected query to provided email addresses. The email *Subject* field will contain the username of the Spectra Analyze user who shared the query. 2. by copying the URL of the search results page from the address bar of the browser, and sending it manually via email or other communication channel. A search query shared in this way will only work if the recipient can log into the same Spectra Analyze instance from which the query was sent. 3. by copying a favorite query to the clipboard (hover over the query in the *Favorites* list and select the *Copy* option from the triple-dot menu), then sharing it manually via email or other communication channel. ### Non-keyword Queries Advanced search queries can be quickly built without using keywords. Non-keyword search is available only for a particular subset of indicators of compromise: > - SHA1, SHA256 and MD5 hashes > - URLs > - IP addresses > - domains > - emails #### Non-keyword Search Queries Non-keyword searches can be performed as standalone queries containing one or more non-keyword values, or be combined with traditional keyword searches. Email and IP (IPv4, IPv6) non-keyword queries support wildcard matching. If a list of non-keyword search values contains invalid entries, search will respond with the message “Unrecognized nonkeyword argument” and return the first invalid non-keyword. In cases where the query contains only hashes, the response returns “Invalid value for hashes field”. Using commas between non-keyword search values will result in an invalid query. Searching for strings containing commas and other special characters is supported by using quotation marks. For example, IPV6 addresses or URLs containing colons, commas, or brackets must be enclosed in quotation marks: ``` "2001:0db8:85a3:0000:0000:8a2e:0370:7334" ``` ``` "http://www.evildomain.com/gate.php?13,35869" ``` **Single non-keyword search** This can be any one of the IOCs listed above. Example: SHA1 ``` 0000038704cb5f0e1bd87d6a75e904529af0d6ac ``` **Multiple non-keyword search** To combine multiple non-keyword search values, separate them by space. The whole query will be enclosed in brackets and the spaces will be interpreted as the operator OR. Other operators (AND/NOT) can be explicitly provided to build more complex queries. Example: IPV4, IPV6 and domain ``` 127.0.0.1 "2620:119:35::35" google.com ``` Example: Hashes only ``` 0000038704cb5f0e1bd87d6a75e904529af0d6ac 2abcd3fb8b7761526d177ab007c40e74 4dea2daa9a41dd6c4cb172eb6d8d8a1d1811360e21c5fa0c8ce2e20fd6903041 ``` **Non-keyword with keyword** When combining non-keyword search values with keywords, consecutive non-keyword values will be enclosed in brackets and the spaces between them will be interpreted as the operator OR. Spaces between non-keyword search values and keywords will be interpreted using the operator AND, meaning that the order of keywords and non-keyword values in the query is important. Example: Samples containing the provided URL that are classified as goodware ``` "https://hope-bd.com/googledocs.php" class:goodware ``` **Combining queries with the NOT operator** The **NOT** operator excludes search results that match the defined criteria. Example: Query using the operator NOT > ``` > NOT *@mockmail.com "https://hope-bd.com/googledocs.php" AND NOT 0000038704cb5f0e1bd87d6a75e904529af0d6ac class:MALICIOUS > ``` #### Non-keyword Search Examples | Query Type | Example | Syntax | Outcome | | ----------------------------------------------------------- | ------------------------------------------------------------ | -------------------------- | --------------------------------------------------- | | Single non-keyword | 0000038704cb5f0e1bd87d6a75e904529af0d6ac | NK | NK | | Non-keyword search values combined with keywords | “https://hope-bd.com/googledocs.php” class:goodware | NK K | NK **AND** K | | Multiple non-keyword values (hashes only) | 0000038[…]af0d6ac 2abcd3[…]7c40e74 4dea2da[…]6903041 | NK NK NK NK | (NK **OR** NK **OR** NK **OR** NK) | | Multiple non-keyword values | 127.0.0.1 “2620:119:35::35” google.com | NK NK NK NK | (NK **OR** NK **OR** NK **OR** NK) | | Multiple non-keyword values with an AND operator | [mock@mockmail.com](mailto:mock@mockmail.com) 127.0.*.1 AND google.com “https://hope-bd.com/googledocs.php” | NK NK AND NK NK | (NK **OR** NK) **AND** (NK **OR** NK) | | Multiple keywords combined with multiple non-keyword values | class:MALICIOUS mock@mockmail.com google.com firstseen:2018-04-05T21:11:47Z | K NK NK K | K **AND** (NK **OR** NK) **AND** K | | Combining queries with the NOT operator | NOT [*@mockmail.com](mailto:*@mockmail.com) “https://hope-bd.com/googledocs.php” AND NOT 0000038[…]af0d6ac class:MALICIOUS | NOT NK NK AND NOT NK AND K | (**NOT** NK **OR** NK) **AND** **NOT** NK **AND** K | Note The final, transformed queries will be returned in the *Advanced search* box and added to the *Recent queries* list. They can be saved as favorites by clicking the star button to the right of the search box. ## Supported Search Keywords ### User-friendly modifiers Some keywords support **modifiers** that serve as shorthand notation for search expressions. #### Numbers Keywords that accept numbers as values also accept a trailing plus or minus sign. For example: - `5+` (five or more) - `42-` (fourty-two or less) Exceptions: - Spaces are not allowed. - Prefixes are also not allowed (only trailing plus or minus). - Modifiers can't be used in range queries. For example, `[3+ to 5-]` is invalid.
List of keywords that accept numbers - `av-count` - `av-threatlevel` - `document-pages` - `elf-section-count` - `filecount` - `macho-section-count` - `macho-segment-count` - `pe-section-count` - `riskscore` - `size` - `submissions` - `threatlevel`
#### Dates Keywords that accept dates also accept period abbreviations and trailing plus/minus. Accepted abbreviations: - `h` for hours - `d` for days - `w` for weeks - `m` for months - `y` for years The trailing plus or minus sign behaves just like for numbers. For example: - `2023-04-11T08:10:00+` (on April 11 2023 at 08:10, or after that time) - `3d+` (three days or more) - `1w-` (one week or less) Exceptions: - Spaces are not allowed. - Prefixes are also not allowed (only trailing plus or minus). - Modifiers can't be used in range queries. For example, `[3d to 5w]` is invalid.
List of keywords that accept dates - `firstseen` - `lastanalysis` - `lastseen` - `pe-timestamp` - `signer-valid-from` - `signer-valid-to` - `submission-time` - `taggant-valid-from` - `taggant-valid-to`
#### Sizes Keywords that accept sizes also accept unit abbreviations (KB, MB, GB...) and trailing plus/minus. The abbreviations are case-insensitive. If an abbreviation is not specified, the expression is evaluated in bytes. [Byte multiples](https://en.wikipedia.org/wiki/Byte#Multiple-byte_units) are supported in both decimal (kilo-, mega-, giga-...) and binary (kibi-, mebi-, gibi-...) form. For example: - `5MB+` (five megabytes or larger) - `13kib-` (thirteen kibibytes or smaller) Exceptions: - Spaces are not allowed. - Prefixes are also not allowed (only trailing plus or minus). - Modifiers can't be used in range queries if they contain a trailing plus or minus sign. For example, `[3kB+ to 5MB]` is invalid. However, if you use them without a trailing plus/minus, they can be used in a range query. For example, `[3kb TO 5mb]` is allowed. The only keyword that accepts a size is `size`. ### Group keywords When using group keywords, the provided search query will be used with all single keywords in the group's respective list. Refer to the single keyword descriptions for more information. Keyword aliases are enclosed in parentheses. | `certificate` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `cert-issuer-name` `cert-issuer-org` `cert-issuer-unit` `cert-subject-name` `cert-subject-org` `cert-subject-unit` | | Examples | Case-insensitive wildcard matching is supported.Wildcard: `certificate:*micr*` | | `certificate-country` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `cert-issuer-country` `cert-subject-country` | | Examples | Case-insensitive wildcard matching is supported.List (any of the values): `certificate-country:[HR, US]` | | `document` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `document-author` `document-subject` `document-title` `document-description` | | Examples | Case-insensitive wildcard matching is supported.List (any of the values): `document:[adobe, microsoft, *confidencial]`Wildcard: `document:*soft` | | `mutex` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `mutex-config` `mutex-dynamic` | | Examples | The keyword is case-sensitive and doesn't accept wildcards.Exact: `mutex:111c`List (any of the values): `mutex:[111c, 2124]` | | `ipv4` (`ip`) | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `ipv4-static` `ipv4-dynamic` | | Examples | Wildcard matching supported.Wildcard: `ipv4:192.*`List (any of the values): `ipv4:[1.0.0.0,1.0.2.1]` | | `ipv6` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `ipv6-static` (IPv6 address strings detected by ReversingLabs Dynamic Services) | | Examples | If the address contains colons or brackets, enclose it in quotation marks.Wildcard matching supported.Wildcard: `ipv6:c*`Exact: `ipv6:"2002::/16"`List (any of the values): `ipv6:["2001:db8*", "3731:54:"]` | | `section` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `pe-section-name` `elf-section-name` `macho-section-name` | | Examples | Case-insensitive wildcard matching is supported.Wildcard: `section:*data`List (Any of the values): `section:[.ndata, bss]` | | `segment` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `macho-segment` `macho-segment-name` `elf-segment-sha1` | | Examples | Case-insensitive wildcard matching is supported.Wildcard: `segment:page*`List (any of the values): `segment:[pagezero, text]` | | `software` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `software-package` `software-description` `software-author` | | Examples | The keyword does not accept wildcards.Exact: `software:"James Newton-King"`List (any of the values): `software:[Microsoft, "This package consists of multiple activities that simplify the processes in Excel."]` | | `upload-data` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `upload-source` `upload-source-tag` | | Examples | Examples:`upload-data:myapp`, `upload-data:myemail*` | | `uri` | Group keyword | |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Includes | `uri-source` `uri-static` `uri-config` `uri-dynamic` | | Examples | Case-insensitive wildcard matching is supported. (`uri*` keywords don't support IP addresses. For that, use `ip*` keywords.)Wildcard: `uri:mozilla.org*`List (any of the values): `uri:[\*.tor,*.onion,*.exit]` | ### Single keywords | `actor` | | |-------------|---------------------------------------------------------------------------------------------------------| | Description | Search for files associated with a specific threat actor. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `actor:*APT*`List (any of the values): `actor:[Hellsing, Cycldek]` | | `android-app-name` | | |--------------------|----------------------------------------------------------------------------------------------------------------------------| | Description | Search for Android applications by their process name. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `android-app-name:*SkypeApplication*`List (any of the values): `android-app-name:[MainApp, *alt.ywuajgf*]` | | `android-features` | | |--------------------|----------------------------------------------------------------------------------------------------------------------| | Description | Search for Android applications by their features. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `android-features:*hardware.camera*`List (any of the values): `android-features:[camera, telephony]` | | `android-import` | | |------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------| | Description | Search for Android applications by one or more shared libraries that the applications are linked against. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `android-import:org.apache.http.legacy*`List (any of the values): `android-import:[sec_fe?ture, *google*]` | | `android-package` | | |-------------------|--------------------------------------------------------------------------------------------------------------------------------------| | Description | Search for Android applications by their package name. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `android-package:*com.picklieapps.player*`List (any of the values): `android-package:[*ruckygames*, *skype.raider*]` | | `android-permission` || |----------------------|------| | Description | Search for Android applications by their permissions. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `android-permission:*WRITE_SETTINGS*`List (any of the values): `android-permission:[*storage*, *disable_keyguard*]` | | `appid-company-name` (`appid-author`) || |-------------|------| | Description | Search for applications and libraries by their publisher. Case-insensitive wildcard matching is supported. | | Examples | Exact: `appid-company-name:"Mozilla Foundation"`List (any of the values): `appid-company-name:["Mozilla Foundation", "Microsoft Corporation"]` | | `appid-description` || |-------------|------| | Description | Search for applications and libraries by their description. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `appid-description:"*Firefox Plugin Hang UI*"*` | | `appid-product-name` | | |-------------|------| | Description | Search for files with a matching product name. Case-insensitive wildcard matching is supported. | | Examples | Exact: `appid-product-name:"Mozilla Firefox Plugin Hang UI"`List (any of the values): `appid-product-name:["Mozilla Firefox Plugin Hang UI", "Mozilla Firefox Helper"]` | | `appid-product-type` (`appid-category`) | | |-------------|------| | Description | Search for applications and libraries by their type. Case-insensitive wildcard matching is supported. | | Examples | Exact: `appid-product-type:browser`List (any of the values): `appid-product-type:[browser, development]` | | `attack-tactic` | | |-------------|------| | Description | Search for files that use a specific Mitre ATT&CK tactic. The keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `attack-tactic:TA0007`List (any of the values): `attack-tactic:[TA0007, TA0005]` | | `attack-technique` | | |-------------|------| | Description | Search for files that use a specific Mitre ATT&CK technique. The keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `attack-technique:T1222`List (any of the values): `attack-technique:[T1222, T1112]` | | `av-count` (`positives`, `p`, `antivirus`) | | |-------------|------| | Description | The number of antivirus scanners that have detected a sample as malicious. Currently supports any integer from 0 to 46 (46 being the number of active AV scanners). | | Examples | Exact: `av-count:5`Range: `positives:[10 TO 20]`Greater than 5: `positives:[5 TO *]`List (any of the values): `av-count:[5,3]` | | `av-detection` (`engines`) | | |-------------|------| | Description | Detection string generated by the antivirus engines. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `av-detection:micro*`List (any of the values): `av-detection:[W32.Duqu, *Vitro]` | | `av-` (``) | | |-------------|------| | Description | Search for all samples or samples of specific malware detected by a selected antivirus vendor. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `av-[vendor]:*wannacry*`List (any of the values): `[vendor]:[win32, emotet]` | | `available` (`in`, `shareable`) | | |-------------|------| | Description | Indicates whether a sample is available for download from the cloud. The only supported values are true and false (case-insensitive). | | Examples | `available:TRUE``in: false` | | `browser-package` | | |-------------|------| | Description | Search for web browser extensions by their package name. Supported package formats: Chrome, Safari, Firefox. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `browser-package:*Click2Save*`List (any of the values): `browser-package:[*priiceechOp*, *iCalc*]` | | `cert-issuer-country` | | |-------------|------| | Description | Search for files by the country code in the country name property field of the issuer of the certificate used to sign the file. Case-insensitive wildcard matching is supported. | | Examples | Exact: `cert-issuer-country: US`List (any of the values): `cert-issuer-country:[Z?,G*]` | | `cert-issuer-name` | | |-------------|------| | Description | Search for files by the name of the certificate authority (CA). Case-insensitive wildcard matching is supported. | | Examples | Exact: `cert-issuer-name: COMODO`List (any of the values): `cert-issuer-name:[microsoft,*VeriSign*]` | | `cert-issuer-org` | | |-------------|------| | Description | Search for files by the organization name of the certificate issuer. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `cert-issuer-org:*authority`List (any of the values): `cert-issuer-org:[verisign, microsoft]` | | `cert-issuer-unit` | | |-------------|------| | Description | Search for files by the organizational unit name of the issuer unit of the certificate authority (CA). Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `cert-issuer-unit:*root*` List (any of the values): `cert-issuer-unit:["trust network", *root*]` | | `cert-serial` | | |-------------|------| | Description | Search for a file by the serial number of the file certificate provided by the CA that issued the certificate. The keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `cert-serial:6101CF3E00000000000F`List (any of the values): `cert-serial:[,]` | | `cert-subject-country` | | |-------------|------| | Description | Search for files by the country code in the country name property field of the subject to which the certificate has been issued. Case-insensitive wildcard matching is supported. | | Examples | Exact: `cert-subject-country:DE`List (any of the values): `cert-subject-country:[US, B*]` | | `cert-subject-name` | | |-------------|------| | Description | Search for files by the name of the organization/system to which the certificate has been issued. Case-insensitive wildcard matching is supported. | | Examples | Exact: `cert-subject-name:Piriform`List (any of the values): `cert-subject-name:[cinectic*, google]` | | `cert-subject-org` | | |-------------|------| | Description | Search for files by the organization name of the certificate authority organization (CA). Case-insensitive wildcard matching is supported. | | Examples | Exact: `cert-subject-org:apple`List (any of the values): `cert-subject-org:[apple, Microsoft]` | | `cert-subject-unit` | | |-------------|------| | Description | Search for files by the organizational unit name inside the organization to which the certificate has been issued. Case-insensitive wildcard matching is supported. | | Examples | Exact: `cert-subject-unit:"Developer Relations"`List (any of the values):`cert-subject-unit:[Developer*, "Trust Network"]` | | `cert-thumbprint` | | |-------------|------| | Description | Search for files by their unique certificate thumbprint. A thumbprint of a file certificate is a hash value (SHA256). The keyword doesn't accept wildcards. | | Examples | Exact: `cert-thumbprint:277D42[...]2A17DD`List (any of the values): `cert-thumbprint:[, ]` | | `classification` (`class`) | | |-------------|------| | Description | Search for files by their Malware Presence status designation. Accepted values: malicious, known, suspicious, unknown (case-insensitive). | | Examples | Exact: `classification:malicious`List (any of the values): `classification:[KNOWN, suspicious]` | | `dex-class-name` | | |-------------|------| | Description | Search for DEX files by the names of classes they contain. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `dex-class-name:android.content.DialogInterface.On*`List (any of the values): `dex-class-name:[android.content.DialogInterface.On*, android.support.v4.*]` | | `dex-method-name` | | |-------------|------| | Description | Search for DEX files by method names their classes call to perform an action. Method names are indexed regardless of their visibility, meaning both public and private methods are searchable. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `dex-method-name:unregisterCallB*`List (any of the values): `dex-method-name:[getLocation, invok*]` | | `document-author` | | |-------------|------| | Description | Search for files by the contents of their document author metadata property. Case-insensitive wildcard matching is supported. | | Examples | List (any of the values): `document-author:[adobe, microsoft]`Wildcard: `document-author:*soft` | | `document-description` (`doc-description`) | | |-------------|------| | Description | Search for files by the document description field, as provided by the document author. Case-insensitive wildcard matching is supported. | | Examples | List (any of the values): `document-description:["Carta personal", *confidencial]`Wildcard: `document-description:*Math*` | | `document-pages` (`doc-pages`) | | |-------------|------| | Description | Search for files by their number of pages. In case of spreadsheet documents, this number represents the number of sheets. The keyword accepts only integer values. | | Examples | Exact: `document-pages:73`Range: `document-pages:[4 TO 20]`More than 4: `document-pages:[4 TO *]` | | `document-subject` | | |-------------|------| | Description | Search for files by the contents of their document subject metadata property. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `document-subject:*search`List (any of the values): `document-subject:[free, download]` | | `document-title` | | |-------------|------| | Description | Search for files by the contents of their document title metadata property. Case-insensitive wildcard matching is supported. | | Examples | Exact: `document-title:"Powered by"`List (any of the values): `document-title:[*free*, README]` | | `document-version` | | |-------------|------| | Description | Search for files by the contents of their document version metadata property. Wildcard matching is supported. | | Examples | Wildcard: `document-version:1.1*`List (any of the values): `document-version:[1.7, 2.*]` | | `domain` | | |-------------|------| | Description | Search for files by any associated domain. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `domain:mozilla.org*`List (any of the values): `domain:[*.tor,google.com,*.exit]` | | `dotnet-assembly` | | |-------------|------| | Description | Search for .NET files by assemblies they reference. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `dotnet-assembly:*mscorlib*`List (any of the values): `dotnet-assembly:[*iJnJWYUQA*, "NanoCore Client"]` | | `dotnet-method-name` | | |-------------|------| | Description | Search for .NET files by method names their classes call to perform an action. Method names are indexed regardless of their visibility, meaning both public and private methods are searchable. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `dotnet-method-name:get_Url`List (any of the values): `dotnet-method-name:[?oadCompl*, *HoldEnd]` | | `dotnet-module-id` | | |-------------|------| | Description | Search for .NET files by IDs of modules they contain. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `dotnet-module-id:*20DEC3DA-523F*`List (any of the values): `dotnet-module-id:[*9249F5D0-1821*, *E133ACC7-60C9*]` | | `dotnet-module-name` | | |-------------|------| | Description | Search for .NET files by names of modules they contain. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `dotnet-module-name:*TeSt.exe*`List (any of the values): `dotnet-module-name:[Posh.exe, adobe.exe]` | | `dotnet-pinvoke-function` | | |-------------|------| | Description | Search for .NET files by pinvoke functions. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `dotnet-pinvoke-function:EncodePointer*`List (any of the values): `dotnet-pinvoke-function:["EncodePointer", "DecodePointer"]` | | `dotnet-pinvoke-import` | | |-------------|------| | Description | Search for .NET files by pinvoke imports. Case-insensitive wildcard matching is supported. | | Examples | Exact: `dotnet-pinvoke-import:kernel32.dll`List (any of the values): `dotnet-pinvoke-import:["kernel32.dll", "user32.dll"]` | | `dotnet-resource` | | |-------------|------| | Description | Search for .NET files by resources they contain. Case-insensitive wildcard matching is supported. | | Examples | Exact: `dotnet-resource:"Hidden Tear"`List (any of the values): `dotnet-resource:[*Orcus*, *Clientloaderform*]` | | `dotnet-type-name` | | |-------------|------| | Description | Search for .NET files by type names found in them. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `dotnet-type-name:Form1*`List (any of the values): `dotnet-type-name:[Form1*, NetscapeRevocationUrl]` | | `elf-section-count` | | |-------------|------| | Description | Search for ELF files by the amount of sections they contain. The keyword accepts only integer values. | | Examples | Exact: `elf-section-count:5`Range: `elf-section-count:[5 TO 15]`More than 5: `elf-section-count:[5 TO *]` | | `elf-section-name` | | |-------------|------| | Description | Search for ELF files by names of the sections they contain. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `elf-section-name:*data`List (any of the values): `elf-section-name:[.rodata, .ndata, .bss]` | | `elf-segment-sha1` (`elf-segment-hash`) | | |-------------|------| | Description | Search for files by the SHA1 hash of their ELF segment. The keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `elf-segment-sha1:116e279b55b58e5b9619aac80a8e85bfa9c839fc` | | `email-from` | | |-------------|------| | Description | Search for files by the sender of an email associated to a file. Includes "from", "reply-to" and "sender" fields. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `email-from:*@kiski.net`List (any of the values): `email-from:[*@domain.com, *@orbitz.com]` | | `email-static` (`email`) | | |-------------|------| | Description | Search for files by associated email address(es) detected by [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis). Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `email-static:*@Compartir.es`List (any of the values): `email-static:[*@gmail.com, *@hotmail.com]` | | `email-subject` | | |-------------|------| | Description | Search for files by the subject of an email associated to a file. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `email-subject:*HackTool`List (any of the values): `email-subject:[Invitation*, *Nova*]` | | `email-to` | | |-------------|------| | Description | Search for files by the receiver of an email associated to a file, specified in the "to" field. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `email-to:*@netnook.com`List (any of the values): `email-to:[*@dekalb.net, *@rogers.com]` | | `email-x-key` | | |-------------|------| | Description | Search for files with non-standard header fields, called X-extensions. Security vendors use X-extensions to annotate emails that have been scanned using their product. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `email-x-key:*MDRemoteIP`List (any of the values): `email-x-key:[*Indiv, *Markup]` | | `email-x-value` | | |-------------|------| | Description | Search for files by values stored in non-standard (X-extension) header fields. Case-insensitive wildcard matching is supported. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `email-x-value:?HAILAND`List (any of the values): `email-x-value:[Produced*, BHUTAN]` | | `exif` | | |-------------|------| | Description | Search for multimedia files by the contents of their EXIF metadata fields. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `exif:Picasa*`List (any of the values): `exif:["Paint.NET v3.5.8", Picasa*]` | | `exploit` | | |-------------|------| | Description | Search for samples that are exploiting a specific vulnerability, identified either by ReversingLabs or by antivirus scanners. | | Examples | Examples Wildcard: `exploit:cve-2024-*`*List (any of the values): `exploit:["CVE-2014-0114", "CVE-2018-15982"]` | | `filecount` | | |-------------|------| | Description | Search for a file by the number of unpacked files it contains (if it's a container). Accepts any integer number. **Note: this keyword currently returns only Local samples as results.** | | Examples | Exact: `filecount:25`Range: `filecount:[3 TO 10]`More than 20: `filecount:[20 TO *]` | | `filename` (`name`) | | |-------------|------| | Description | Search for a file by its full or partial file name, predicted file name (generated by Spectra Core for samples without a file name), or file extension. Case-insensitive wildcard matching is supported. | | Examples | Exact: `filename:notepad.exe`List (any of the values): `filename:[*.PDF, *.epub]` | | `firstseen` (`fs`) | | |-------------|------| | Description | Time when a file was first analyzed by Spectra Intelligence. Supported time format is UTC timestamp. | | Examples | Exact: `fs:2018-04-03T12:58:27Z`Range (time period):`firstseen:[2017-12-01T11:36:59Z TO 2018-03-06T11:36:59Z]` | | `hashes` | | |-------------|------| | Description | Allows mixing different types of hashes in one search query, without the need to explicitly name the hash type or to group hashes by type. All hash types (MD5, SHA1, SHA256) can be used with this keyword. The maximum length of a single query is 1024 characters. The keyword is case-sensitive and doesn't support wildcards. | | Examples | Exact: `hashes: `List (any of the values): `hashes:[, , , , ]` | | `imphash` | | |-------------|------| | Description | Hash based on library/API names and their specific order within the executable. Used to find similar PE files. The keyword doesn't support wildcards. | | Examples | Exact: `imphash:f34d5f2d4577ed6d9ceec516c1f5a744`List (any of the values): `imphash [, ]` | | `indicators` | | |-------------|------| | Description | Search for files by their static analysis behaviors. The keyword is case-sensitive and doesn't accept wildcards. The full list of indicator IDs and their descriptions can be found [here](/General/SpectraCore/indicators/). | | Examples | Exact: `indicators:"2150"`List (any of the values): `indicators:["2150", "2102"]` | | `ios-app-name` | | |-------------|------| | Description | Search for iOS applications by their name. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `ios-app-name:FruitNinja*`List (any of the values): `ios-app-name:[FruitNinja*, *facebook*]` | | `ios-author` | | |-------------|------| | Description | Search for iOS applications by their author name. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `ios-author:*halfbrick*`List (any of the values): `ios-author:[*halfbrick*, Apple*]` | | `ios-package` | | |-------------|------| | Description | Search for iOS applications by their package name. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `ios-package:*FruitNinja*`List (any of the values): `ios-package:[*FruitNinja*, *facebook*]` | | `ipv4-dynamic` | | |-------------|------| | Description | Search for files by IPv4 address strings detected by ReversingLabs Dynamic Services. Wildcard matching supported. | | Examples | Wildcard: `ipv4-dynamic:192.*`List (any of the values): `ipv4-dynamic:[1.0.0.0,1.0.2.1]` | | `ipv4-static` | | |-------------|------| | Description | Search for files by IPv4 address strings detected by Spectra Core analysis. Wildcard matching supported. | | Examples | Wildcard: `ipv4-static:192.*`List (any of the values): `ipv4-static:[1.0.0.0,1.0.2.1]` | | `ipv6-static` | | |-------------|------| | Description | Search for files by IPv6 address strings detected by Spectra Core analysis. If the address contains colons or brackets, enclose it in quotation marks. Wildcard matching supported. | | Examples | Wildcard: `ipv6-static:c*`Exact: `ipv6-static:"2002::/16"`List (any of the values): `ipv6-static:["2001:db8*", "3731:54:"]` | | `lastanalysis` (`la`) | | |-------------|------| | Description | Search for files by the date and time of their last AV scan. Supported time format is UTC timestamp. | | Examples | Exact: `lastanalysis:2018-05-17T11:27:19Z`Range (time period):`lastanalysis:[2018-05-17T11:27:19Z TO 2018-05-24T11:27:19Z]` | | `lastseen` (`ls`) | | |-------------|------| | Description | Time when a file was last analyzed by Spectra Intelligence. Supported time format is UTC timestamp. | | Examples | Exact: `ls:2018-04-03T12:58:27Z`Range (time period):`lastseen:[2017-12-01T11:36:59Z TO 2018-03-06T11:36:59Z]` | | `macho-import` | | |-------------|------| | Description | Search for MachO files by the names of imported libraries found in them. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `macho-import:*/usr/lib/*`List (any of the values): `macho-import:[/usr/lib/libgcc_s.1.dylib, /usr/lib/libSystem.B.dylib]` | | `macho-section-count` | | |-------------|------| | Description | Search for MachO files by the number of sections they contain. The keyword accepts only integer values. | | Examples | Exact: `macho-section-count:10`Range: `macho-section-count:[5 TO 15]`More than 5: `macho-section-count:[5 TO *]` | | `macho-section-name` | | |-------------|------| | Description | Search for MachO files by the names of the sections they contain. Case-insensitive wildcard matching supported. | | Examples | Exact: `macho-section-name:data`List (any of the values): `macho-section-name:[bss, common, data]` | | `macho-segment` (`macho-segment-name`) | | |-------------|------| | Description | Search for MachO files by their segment names. Case-insensitive wildcard matching supported. | | Examples | Exact: `macho-segment:pagezero`List (any of the values): `macho-segment:[linkedit, pagezero, text]` | | `macho-segment-count` | | |-------------|------| | Description | Search for MachO files by the count of segments they contain. The keyword accepts only integer values. | | Examples | Exact: `macho-segment-count:30`Range: `macho-segment-count:[2 TO 8]`More than: `macho-segment-count:[11 TO *]` | | `macho-segment-sha1` (`macho-segment-hash`) | | |-------------|------| | Description | Search for files by the SHA1 hash of their MachO segment. The keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `macho-segment-sha1:116e279b55b58e5b9619aac80a8e85bfa9c839fc` | | `macho-symbol` | | |-------------|------| | Description | Search for MachO files by their symbol names. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `macho-symbol:f*`List (any of the values): `macho-symbol:[exit, malloc, umask]` | | `md5` | | |-------------|------| | Description | String of hexadecimal digits representing a MD5 hash of the file sample. Keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `md5:76baa04885ec40af25294a51d8e7c006`List (any of the values): `md5:[, ]` | | `mutex-config` | | |-------------|------| | Description | Search for files by their malware configuration mutexes detected by Spectra Core. The keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `mutex-config:")!VoqA.I4"`Exact: `mutex-config:"--((Mutex))--"`List (any of the values): `mutex-config:[111c, 2124]` | | `mutex-dynamic` | | |-------------|------| | Description | Search for files by malware configuration mutexes detected by ReversingLabs Dynamic Services. The keyword is case-sensitive and doesn't accept wildcards. | | Examples | Wildcard: `mutex-dynamic:111c*`List (any of the values): `mutex-dynamic:[111c, 2124]` | | `pdb-path (pdb)` | | |-------------|------| | Description | Search for files associated with specific PDB (program database) paths. Used to find files with the same PDB path created during file sample compilation. If the path contains restricted characters, enclose it in quotation marks. | | Examples | Exact: `pdb:"D:DevTin7InstallDir"`List (any of the values):`pdb:["C:Windows", "c:Program FilesPerforce"]` | | `pe-company-name` | | |-------------|------| | Description | Search for PE files by the contents of their company name field in the version information metadata. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `pe-company-name:*enix`List (any of the values): `pe-company-name:[microsoft, ADOBE]` | | `pe-copyright` | | |-------------|------| | Description | Search for PE files by the contents of their legal copyright field in version information metadata. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `pe-copyright:Copyright*`List (any of the values): `pe-copyright:[*Corporation, regsvr32]` | | `pe-description` | | |-------------|------| | Description | Search for PE files by the contents of their file description field in version information metadata. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `pe-description:*proged`List (any of the values): `pe-description:[DisplaySwitch, WizardFramework]` | | `pe-export` (`exports`) | | |-------------|------| | Description | Search for PE files by exported symbol names. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `pe-export:MS*`List (any of the values): `exports:[GetMemoSize, DeleteFile]` | | `pe-function` | | |-------------|------| | Description | Search for PE files by the name of the function that the PE file imports. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `pe-function:RegEnum*`List (any of the values):`pe-function:[RegEnumKeyW, GetUserNameA]` | | `pe-import (imports)` | | |-------------|------| | Description | Search for PE files by the name of the dynamic link library that the PE file imports. Case-insensitive wildcard matching supported. | | Examples | Exact: `pe-import:URLMON.DLL`List (any of the values): `imports:[win*, url*]` | | `pe-language` | | |-------------|------| | Description | Find PE files by languages mentioned in the PE file resources. Case-insensitive wildcard matching supported. [Supported Languages for PE and Document Formats](#supported-languages-for-pe-and-document-formats). | | Examples | Exact: `pe-language:russian`List (any of the values): `pe-language:[eng*, Russian]` | | `pe-original-name` | | |-------------|------| | Description | Search for PE files by the contents of their file description field in version information metadata, and any other fields using the original name of the file. The keyword can be used to investigate how the file was named during compilation. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `pe-original-name:crack*`List (any of the values): `pe-original-name:[*install.exe, "sample doc.exe"]` | | `pe-overlay-sha1` (`pe-overlay-hash`) | | |-------------|------| | Description | Find PE files by the SHA1 hash calculated for their overlay part. Overlay hashes are calculated by Spectra Core to better represent the true boundary of the file region. Users should use hash values calculated by ReversingLabs products with this keyword. Keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `pe-overlay-sha1:4b4a2436b827d42b204b1f112b45d7a6d1b7ca52`List (any of the values): `pe-overlay-sha1:[, , ]` | | `pe-product-name` | | |-------------|------| | Description | Search for PE files by the contents of their product name field in version information metadata. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `pe-product-name:*shop`List (any of the values):`pe-product-name:[Firefox, "Microsoft Word"]` | | `pe-resource` | | |-------------|------| | Description | Search for PE files by name or type of resources they contain. Case-insensitive wildcard matching supported. | | Examples | Exact: `pe-resource:Properties`List (any of the values): `pe-resource:[Tcpview, Aboutbox]` | | `pe-resource-sha1` (`pe-resource-hash`) | | |-------------|------| | Description | Find PE files by the SHA1 hash calculated for their resources part. Keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `pe-resource-sha1:4260284ce14278c397aaf6f389c1609b0ab0ce51`List (any of the values): `pe-resource-sha1:[, ]` | | `pe-section-count` | | |-------------|------| | Description | Search for PE files by the count of sections they contain. The keyword accepts only integer values. | | Examples | Exact: `pe-section-count:15`Range: `pe-section-count:[2 TO 10]`More than: `pe-section-count:[5 TO *]` | | `pe-section-name` | | |-------------|------| | Description | Search for PE files by names of the sections they contain. The maximum section name length is 8 characters. Case-insensitive wildcard matching supported. | | Examples | Wildcard: `pe-section-name:*rdata`List (any of the values): `pe-section-name:[.Rdata, .Ndata, *rsrc]` | | `pe-section-sha1` (`pe-section-hash`) | | |-------------|------| | Description | Find PE files by the SHA1 hash calculated for their section part. Section hashes are calculated by Spectra Core to better represent the true boundary of the file region. Users should use hash values calculated by ReversingLabs products with this keyword. Keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `pe-section-sha1:7640a007e39b487bf1dbbde6487724faa131f6a8`List (any of the values): `pe-section-sha1:[, , ]` | | `pe-timestamp` (`pets`) | | |-------------|------| | Description | Search for a PE file by the date when it was compiled. Supported time format is UTC timestamp. | | Examples | Exact: `pets:2017-06-26T00:00:00Z`Range (newer than): `pets:[2018-03-06T10:57:29Z TO *]` | | `sampletype` (`filetype`, `type`, `format`) | | |-------------|------| | Description | Search for files by type as detected by Spectra Core. Case-insensitive wildcard matching supported. `Appendix B - Supported Sample Types`_ | | Examples | Exact: `sampletype:Image/None`List (any of the values): `type:[elf*,macho*]` | | `sha1` | | |-------------|------| | Description | String of hexadecimal digits representing a SHA-1 hash of the file. Keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `sha1:f1a62a7092e49577206b7361bf1a7ff0776bb6a4`List (any of the values):`sha1:[, ]` | | `sha256` | | |-------------|------| | Description | String of hexadecimal digits representing a SHA-256 hash of the file sample. Keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `sha256:f35a3(...)1d2d5`List (any of the values): `sha256:[, ]` | | `signer-valid-from` (`cert-valid-from`) | | |-------------|------| | Description | Search for files that have been signed by certificates valid from a specific time. | | Examples | Range (newer than): `signer-valid-from:[2018-03-06T10:57:29Z TO *]` | | `signer-valid-to` (`cert-valid-to`) | | |-------------|------| | Description | Search for files that have been signed by certificates valid to a specific time. | | Examples | Range (newer than): `signer-valid-to:[2018-03-06T10:57:29Z TO *]` | | `similar-to` | | |-------------|------| | Description | Search for files that are functionally similar to the requested file hash. Functionally similar files are defined by RHA (ReversingLabs Hashing Algorithm) that identifies code similarity between unknown samples and previously seen malware samples. All hash types (MD5, SHA1, SHA256) can be used with this keyword. Only one similar-to keyword can be used in a single query. The keyword is case-sensitive and doesn't support wildcards. | | Examples | Exact: `similar-to: ` | | `size` | | |-------------|------| | Description | Search for files by size (in bytes). Accepts integers up to 2147483647. | | Examples | Exact: `size:30000`Range: `size:[1000 TO 50000]`Greater than: `size:[500000 TO *]` | | `software-author` | | |-------------|------| | Description | Search for software packages by their author/publisher. | | Examples | Exact: `software-author:"James Newton-King"`List (any of the values): `software-author:["Amazon Web Services", Microsoft]` | | `software-description` | | |-------------|------| | Description | Search for software packages by their description. | | Examples | Exact: `software-description:"This package consists of multiple activities that simplify the processes in Excel."` | | `software-package` | | |-------------|------| | Description | Search for specific software packages. The keyword is case-sensitive and doesn't accept wildcards. | | Examples | Exact: `software-package:tidal`List (any of the values): `software-package:[tidal, "AWSSDK.WorkLink"]` | | `submissions` | | |-------------|------| | Description | Search for files by the amount of times they have been submitted for analysis. The keyword accepts only integer values. | | Examples | Exact: `submissions:3`Greater than: `submissions:[3 TO *]`Less than: `submissions:[* TO 4]` | | `tag` | | |-------------|------| | Description | Search for files by metadata tags generated by Spectra Core. Tags identify interesting properties of a sample, such as being packed, password-protected, or digitally signed. [Supported Tags](#supported-tags). | | Examples | Exact: `tag:packed`List (any of the values): `tag:[capability-execution, cert, crypto]` | | `tag-yara` | | |-------------|------| | Description | YARA supports adding custom tags to rules. Files that match those rules get automatically tagged after analysis. This keyword looks for files tagged by YARA rules, including those that were classified by YARA tags ("malicious" and "suspicious"). Case-insensitive wildcard matching is supported. Note that changes to YARA tags are not immediately reflected in search results. For example, if a tag is removed from a YARA rule, it will still return search results until files that match the rule are reanalyzed with Spectra Core. | | Examples | Exact: `tag-yara:malicious`List (any of the values): `tag-yara:[malicious, suspicious]` | | `taggant-name` | | |-------------|------| | Description | Search for PE files by name of the packer that was used to pack them. Taggant is a technology that guarantees the packed file came from a reliable source. Case-insensitive wildcard matching supported. | | Examples | Exact: `taggant-name:themida`List (any of the values): `taggant-name:[enigma*, vmprotect*]` | | `taggant-valid-from` | | |-------------|------| | Description | Search for files by the time it was signed using taggant. | | Examples | Range (newer than): `taggant-valid-from:[2018-03-06T10:57:29Z TO *]` | | `taggant-valid-to` | | |-------------|------| | Description | Search for files by the expiry time provided by taggant. | | Examples | Range (newer than): `taggant-valid-to:[2018-03-06T10:57:29Z TO *]` | | `third-party-library` | | |-------------|------| | Description | Search for PE files by the name(s) of third-party libraries they contain. Case-insensitive wildcard matching is supported. | | Examples | Exact: `third-party-library:Microsoft.WindowsAPICodePack-Core`List (any of the values): `third-party-library:[*oak-json*, Microsoft.Web.WebJobs*]` | | `third-party-publisher` | | |-------------|------| | Description | Search for PE files by publishers of the third-party libraries found in the files. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `third-party-publisher:Microsoft*`List (any of the values): `third-party-publisher:[Microsoft*, "Xamarin Inc."]` | | `threatlevel` | | |-------------|------| | Description | Search for files by ReversingLabs scale of threat severity. Higher number indicates higher severity. Accepted values are 0-5. | | Examples | Exact: `threatlevel:3`Greater than: `threatlevel:[2 TO *]`Range: `threatlevel:[0 TO 3]`List (any of the values): `threatlevel:[2, 3]` | | `threatname` | | |-------------|------| | Description | Search for files by malware threat name according to [ReversingLabs malware naming standard](/General/AnalysisAndClassification/MalwareNamingStandard). Case-insensitive wildcard matching supported. | | Examples | Exact: `threatname:Win32.PUA.Casonline`List (any of the values):`threatname:["WIN32.PUA.casino eldorado", *crytex]` | | `trustfactor` | | |-------------|------| | Description | Search for files by the ReversingLabs trust factor. Trust factor indicates the trustworthiness of files. Lower number means higher trust. Accepted values are 0-5. | | Examples | Exact: `trustfactor:1`List (any of the values): `trustfactor:[4, 5]`Range: `trustfactor:[1 TO 3]`Greater than: `trustfactor:[3 TO *]` | | `upload-source` | | |-------------|------| | Description | Search for samples that were uploaded with a specific source parameter. Possible sources: `s3`, `fileshare`, `azure-data-lake`, `smtp`, `abusebox`, `icap-proxy`, `falcon`, `api`, `rlsdk` | | Examples | `upload-source:api`, `upload-source:s3` | | `upload-source-tag` | | |-------------|------| | Description | Search for samples uploaded with a specific user-defined source tag. | | Examples | `upload-source-tag:myapp`, `upload-source-tag:myemail*` | | `uri-config` (`c2`) | | |-------------|------| | Description | Malware configuration C&C (Command & Control), extracted by Spectra Core. C&C infrastructure is used to control malware, particularly botnets. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `c2:*dns*`List (any of the values): `uri-config:[dydns.org, hldns.ru]` | | `uri-dynamic` | | |-------------|------| | Description | Search for files by URI strings (URLs, domains) detected by ReversingLabs Dynamic Services. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `uri-dynamic:mozilla.org*`List (any of the values): `uri-dynamic:[*.tor,*.onion,*.exit]` | | `uri-source` (`itw`) | | |-------------|------| | Description | Search for files by the URI source from which they were downloaded. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `uri-source:*warez*`List (any of the values): `itw:[softonic.com, *cnet.com]` | | `uri-static` | | |-------------|------| | Description | Search for files by URI strings (URLs, domains) detected by Spectra Core. Case-insensitive wildcard matching is supported. | | Examples | Wildcard: `uri-static:mozilla.org*`List (any of the values): `uri-static:[*.tor,*.onion,*.exit]` | | `vertical` | | |-------------|------| | Description | Search for files by the type of vertical feed in which they were found. Case-insensitive wildcard matching is supported. | | Examples | Exact: `vertical:ransomware`List (any of the values): `vertical:[ransomware,apt,financial]` | ### Supported File Types and Subtypes - [Native files and subtypes](/General/SpectraCore/report-section/) - [Native binary file types and identifications](/General/SpectraCore/report-native-binary/) - [Native PE file types and identifications](/General/SpectraCore/report-native-pe/) - [Native PE SFX file types and identifications](/General/SpectraCore/report-native-pe-sfx/) - [Native multimedia file types and identifications](/General/SpectraCore/report-native-multimedia/) - [Native text file types and identifications](/General/SpectraCore/report-native-text/) - [Native ELF file types and identifications](/General/SpectraCore/report-native-elf/) - [Native ELF SFX file types and identifications](/General/SpectraCore/report-native-elf-sfx/) - [Native script file types and identifications](/General/SpectraCore/report-native-script/) - [Verified software and package identities](/General/SpectraCore/appid-software/) - [Known vulnerabilities](/General/SpectraCore/cve-list/) - [Supported unpacking format list](/General/SpectraCore/report-unpacking/) - [Spectra Core certificate trust store](/General/SpectraCore/cert-trust-store/) ### Supported Tags See the complete list of [supported tags](/General/SpectraCore/core-tags/). ### Indicators See the complete list of [Spectra Core Indicators](/General/SpectraCore/indicators/). ### Supported Languages for PE and Document Formats | | | | | -------------------------- | --------------------------- | ----------------------- | | afrikaans | english belize | kannada | | albanian | english can | kashmiri india | | arabic algeria | english caribbean | kashmiri sasia | | arabic bahrain | english eire | kashmiri | | arabic egypt | english jamaica | kazak | | arabic iraq | english nz | konkani | | arabic jordan | english philippines | korean | | arabic kuwait | english south africa | korean | | arabic lebanon | english trinidad | kyrgyz | | arabic libya | english uk | latvian | | arabic morocco | english us | lithuanian classic | | arabic oman | english zimbabwe | lithuanian | | arabic qatar | english | lithuanian | | arabic saudi arabia | esperanto | macedonian | | arabic syria | estonian | malay brunei darussalam | | arabic tunisia | faeroese | malay malaysia | | arabic uae | farsi | malay | | arabic yemen | finnish | malayalam | | arabic | french belgian | maltese | | armenian | french canadian | manipuri | | assamese | french luxembourg | maori | | azeri cyrillic | french monaco | marathi | | azeri latin | french swiss | mongolian | | azeri | french | nepali india | | basque | french | nepali | | belarusian | gaelic manx gaelic scottish | neutral | | bengali | gaelic | norwegian bokmal | | breton | gaelic | norwegian nynorsk | | bulgarian | galician | norwegian | | catalan | georgian | oriya | | chinese hongkong | german austrian | polish | | chinese macau | german liechtenstein | portuguese brazilian | | chinese simplified | german luxembourg | portuguese | | chinese singapore | german swiss | portuguese | | chinese traditional | german | punjabi | | chinese | german | rhaeto_romance | | cornish | greek | romanian moldavia | | croatian | gujarati | romanian | | croatian | hebrew | romanian | | czech | hindi | russian moldavia | | danish | hungarian | russian | | default | icelandic | russian | | divehi | indonesian | saami | | dutch belgian | invariant | sanskrit | | dutch surinam | italian swiss | serbian cyrillic | | dutch | italian | serbian latin | | dutch | italian | serbian | | english aus | japanese | sindhi | | slovak | spanish peru | tswana | | slovenian | spanish puerto rico | turkish | | sorbian | spanish uruguay | ukrainian | | spanish argentina | spanish venezuela | urdu india | | spanish bolivia | spanish | urdu pakistan | | spanish chile | spanish | urdu | | spanish colombia | sutu | uzbek cyrillic | | spanish costa rica | swahili | uzbek latin | | spanish dominican republic | swedish finland | uzbek | | spanish ecuador | swedish | venda | | spanish el salvador | swedish | vietnamese | | spanish guatemala | syriac | walon | | spanish honduras | sys default | welsh | | spanish mexican | tamil | xhosa | | spanish modern | tatar | zulu | | spanish nicaragua | telugu | | | spanish panama | thai | | | spanish paraguay | tsonga | | --- ## Spectra Analyze Alerts — Subscriptions, Delivery Methods & Notifications The Alerts feature allows real-time monitoring to track changes in malware classification and analysis results. It automates notifications for specific events, allowing users to utilize their time more efficiently and be informed of malware status changes on-the-go. ![The action menu on the Alerts page with Configure option highlighted](./images/analyze-alerts-page.png) Users can receive notifications about changes related to the samples on the appliance and in Spectra Intelligence. The notifications make it easy to monitor and track various malware analysis activities even when the users are away from the appliance. Users can choose how often they want to be notified, and where they want to receive the notifications. [Unresolved](#marking-alerts-as-resolved) alerts are indicated by a red badge on the alerts indicator in the top right of the interface, and by blue stripes to the left of the alerts grid. The conditions that define when an alert should be generated are called **subscriptions**. Every subscription can be associated with one or more **delivery methods**. Users can create and modify alert subscriptions from the [Subscriptions and Actions Management page](#managing-alerts). It is also possible to [create alert subscriptions for individual samples](#creating-a-new-alert-subscription) in the GUI. Every alert subscription can have one or more delivery methods. Users can create and modify actions from the [Subscriptions and Actions Management page](./alerts.md#managing-alerts). When an alert is triggered, the notification is delivered to the destination(s) specified in the associated action(s). If an alert subscription doesn’t have any associated delivery methods, the alert will only be visible on the Alerts page. The Alerts page provides an overview of all events on the appliance, including the ones generated by user-defined subscriptions. Users can filter the Alerts page to display only the alerts related to specific alert subscriptions. **Important notes about the Alerts feature** 1. Alerts are sent only if there is a new event matching the conditions configured in the subscription. Alerts cannot be sent for events that occurred before a subscription was created. 2. Multiple alerts triggered for one subscription in the selected notification period will be aggregated and sent in a single email message. 3. The SMTP service must be configured on the appliance in order to receive emails about new alerts. 4. The maximum amount of hashes that the user can add to one subscription is 1000. 5. Only SHA1 hashes can be added to alert subscriptions. 6. The maximum amount of subscriptions a user can create is 10 000. This limit is not user-configurable. When the limit is reached, a notification appears in the GUI, warning the user that some old subscriptions should be removed in order to create new ones. 7. Alert subscriptions are private and visible only to the user who created them. When other users are logged into their accounts, they will not be able to see or modify subscriptions created by other users. ## Alert Types The following table lists the currently supported types of alerts (notifications). To set up an alert and get notified when it is triggered, users have to [create a new subscription](./alerts.md#creating-a-new-alert-subscription). | Alert Type | Alert Trigger | |-------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Analysis Alerts | Analysis Alerts include notifications for Static Analysis, Threat Intelligence and RL Cloud Sandbox, tracking the progress of automated analysis. It also covers third-party sandbox integrations like Cuckoo and Joe Sandbox. | | Classification change | A sample’s threat status has changed (e.g., from “suspicious” to “malicious”). | | Sample available for download | A previously unavailable sample has become available for download from the Spectra Intelligence cloud. | | YARA matches | A new sample has matched one or more YARA rulesets. [YARA Retroactive Hunting matches](./yara-retro.md) are also included. | ## Managing Alert Subscriptions and Delivery Methods Users can view and manage their alert subscriptions from the [Alerts Configuration](#managing-alerts) page. To access the page, click *Configure* on the Alerts page. Alternatively, select **User menu > User profile** from the main menu. This opens the [User Settings](./initial-configuration.md#user-settings) page. Alternatively, click *Configure* on the Alerts page. The *Alerts Configuration* page is divided into two sections: *Alerts* and *Delivery Methods*. All currently active subscriptions are listed in the *Alerts* section. Users can modify existing subscriptions by clicking *Edit*, or [create new ones](#creating-a-new-alert-subscription) by clicking **New Alert**. ![List of active alert subscriptions on the User Settings page](./images/analyze-alerts-configuration.png) Similarly, all actions that can be associated with alert subscriptions are listed in the *Delivery Methods* section. Users can modify existing actions by clicking *Edit*, or [create new ones](#creating-a-delivery-method) by clicking **New Method**. ### Creating a New Alert Subscription To create a new alert subscription, open the *Alerts Configuration* page, and click **New Alert**. The *New Subscription* dialog allows users to choose one or more event types and the associated delivery methods. Users can also add a custom name and description to every subscription, which makes it easier to distinguish between subscriptions and quickly check their scope. In order to associate an alert subscription with a delivery method, users must first [create some delivery methods](#creating-a-delivery-method). When a delivery method is associated with an alert subscription, the details configured for the delivery method apply to that particular subscription. Any method can be associated with any number of active alert subscriptions. Likewise, any subscription can have more than one delivery method associated with it. For example, if an action can define that email should be sent to *test@user.example* every 2 hours, any associated alert subscription will send alerts to that particular address at that particular frequency. Saving the alert subscription makes it active on the appliance. ![Dialog for creating a new alert subscription](images/analyze-subscription-dialog.png) ### Event Types Depending on the selected event types, additional options become available in the *New Subscription* dialog. **Analysis Alerts** Subscribe to notifications for ReversingLabs services (Static Analysis, Threat Intelligence and RL Cloud Sandbox) and third-party services. Available alert stages depend on the analysis method and may include *Started*, *Finished*, and/or *Failed*. **Classification Changes** The Classification Changes event type allows choosing which changes in sample threat status will trigger an alert (for example, from goodware and suspicious to malicious). It is possible to select multiple threat status values in both configuration fields. ![Dialog for creating a new Classification change alert subscription](images/analyze-alerts-classification-changes.png) There are several optional checkboxes that allow setting up more fine-grained alerting criteria. 1. **On Risk Score change** - becomes visible for this event type when threat status values that support it are selected. Selecting the checkbox sends an alert when there is a transition in the [risk score](/General/AnalysisAndClassification/Classification/#risk-score) of a sample (for example, from low (6) to medium (7)), even if the threat status does not change. For example, a sample's risk score can increase from 6 to 9, but the sample will still be classified as malicious. The alert is not sent when the risk score transition is within the same severity range (for example, medium (8) to medium (7)). Transitions in risk score values for goodware samples do not generate alerts. 2. **On threat name change** - selecting it will send an alert when the threat name of a sample changes, even if its threat status remains unchanged. 3. **Notify on specific hash(es)** - when this checkbox is selected, an additional *Sample hash values* text field becomes available. If any SHA1 hashes are entered into the text field, the alert is sent only for classification changes to those samples. When this checkbox is not selected, and no SHA1 hashes are provided, the alert is sent for classification changes to all samples on the appliance. **Sample Available for Download** The Sample available for download event type requires users to provide a list of at most 100 SHA1 hashes to monitor for availability. If **Fetch and analyze** is enabled, samples will automatically be downloaded to the appliance when they become available. **YARA Matches** The YARA Matches event type accepts a list of YARA rulesets that a sample should match to trigger the alert. There is an autocomplete pull-down list with names of all YARA rulesets on the appliance. The maximum number of YARA rulesets is 20. ### Creating a Delivery Method To create a new delivery method, open the *Alerts Configuration* page, and click **New Delivery Method**. The *New Delivery Method* dialog allows users to choose the delivery method (where to send alerts about triggered events) and set the notification frequency (how often the alerts should be sent). Users can also add a custom name for every action. Currently supported delivery methods include *E-mail* and *Syslog*. Clicking *Save* saves the method, making it visible in the *Delivery Methods* section and available when creating or modifying an alert subscription ![Dialog for creating a delivery method](images/analyze-alerts-delivery-method.png) #### Creating a New Email Delivery Method When the *E-mail* method is selected, an additional checkbox **Deliver to logged in user** becomes available. Selecting this checkbox will send alerts to the currently logged-in user, in addition to other email addresses provided in the *Additional email addresses* field. Each address should be in its own line in the text field. The SMTP service must be configured on the appliance in order to receive alert emails from it. Appliance administrators should make sure the settings are properly configured in the **Administration > Configuration > SMTP** dialog. When delivered to the configured address, the alert email contains information about which subscriptions have new alerts, and how many new alerts were generated for each subscription. The email also contains a link for each subscription. The link leads to the Alerts page on the appliance, filtered to display only alerts for the specified subscription. #### Creating a New Syslog Delivery Method When the *Syslog* method is selected, additional options become available in the dialog. **Host** - requires the user to input the host address of the syslog server that should receive alerts **Port** - requires the user to configure the port on which the syslog server should listen for alerts. **Transport protocol** - allows choosing which communication protocol should be used to send alerts to the syslog server. Supported protocols are TCP and UDP. The protocol selected here affects which port number can be configured in the previous field The **Message format** option allows users to customize how the alerts will appear in the logs. Users can combine any number of supported fields with custom text to adjust the contents of each alert. The default format (``*{sha1} - {alert_type} - {description}*``) is already configured here when creating a new action. **Custom Message Examples** ``` {sha1} - {alert_type} - {description} b8b3b60c13d3fbb6e1932dab72264410bab0a2ce - classification_change - Threat Status changed from 2 to 1 via Spectra Intelligence ``` ``` The alert {label} was triggered on sample {sha1} The alert New Malicious Detection was triggered on sample b8b3b60c13d3fbb6e1932dab72264410bab0a2ce ``` ``` New {alert_type} alert received on file {sha1} with status {threat_status} New classification_change alert received on file b8b3b60c13d3fbb6e1932dab72264410bab0a2ce with status 1 ``` **Supported Message Fields** | FIELD NAME | DESCRIPTION | TYPE | | ---------------------- | ------------------------------------------------------------ | ------------- | | alert_type | Type of the triggered alert - can be one of the currently supported types: classification_change, sample_available, ticloud_analysis_complete, ticloud_upload_complete, cuckoo_analysis_complete, yara_match | string | | category | File category of the sample for which the alert was triggered, expressed as a number. The numbers correspond to the following categories: 0 - Other, 1 - Application, 2 - Mobile, 3 - Document, 4 - Web, 5 - Archive, 6 - Media, 7 - Email | integer | | classification | Classification of the sample in human-readable format. | string | | threat_status | Threat status assigned to the sample that triggered the alert, expressed as a number. The numbers correspond to the following threat statuses: 0 - No Threats Found, 1- Good, 2 - Suspicious, 3 - Malicious | integer | | classification_reason | Reason for the classification that the sample received, expressed as a number. The numbers correspond to the following classification reasons: 0 - No Threats Found, 1 - Antivirus, 2 - Signature, 3 - Certificate, 4 - Format, 5 - Exploit, 6 - YARA, 7 - RHA1, 128 - User, 256 - Cloud | integer | | threat_name | Detected malware threat name for the sample that triggered the alert (for example, Win32.Trojan.Adams). | string | | classification_source | Source of the classification for the sample that triggered the alert, expressed as a number. The numbers correspond to the following sources: 0 - None, 1 - Spectra Intelligence, 2 - [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis), 4 - User, 257 - Spectra Intelligence from Parent, 258 - Spectra Core from Parent, 260 - User from Parent, 513 - Spectra Intelligence from Child, 514 - Spectra Core from Child, 516 - User from Child | integer | | description | The full description of the triggered alert, as displayed in the *Details* column on the Alerts page. Depending on the alert, it may include the username of the user who submitted the sample for analysis, the analysis duration, analysis parameters, or the reasons the analysis failed (for example, "Static Intelligence Analysis Started for 72bbd2513901b1bda0f3a0fdc257d347531a8e7a by admin" or "Threat Intelligence analysis complete with classification Goodware."). | string | | string | | factor | The risk score value, expressed as a number from 0 to 10, where 0 indicates the most trusted samples, and 10 indicates the most dangerous threats. | integer | | family_name | The malware family name part of the threat name detected for the sample (for example, Marsdaemon, Orcus, Androrat…). | string | | file_size | Size of the sample for which the alert was triggered, expressed in bytes. | integer | | file_subtype | Subtype of the sample for which the alert was triggered, as detected by ReversingLabs static analysis (for example, TIFF, Clojure, HTML…). [See full list of file subtypes](./advanced-search.md#supported-file-types-and-subtypes) | string | | file_type | Type of the sample for which the alert was triggered, as detected by ReversingLabs static analysis (for example, Document, Image, PE…). [See full list of file types](./advanced-search.md#supported-file-types-and-subtypes) | integer | | hostname | Hostname of the system on which the sample that triggered the alert was analyzed or submitted. | | identification_name | Identification of the sample for which the alert was triggered, as detected by ReversingLabs static analysis. Identification is not generated for all file types and subtypes; it is an optional field in the static analysis report that indicates a more detailed (usually signature-based) file detection. | string | | identification_version | Version of the identified file format for the sample that triggered the alert (for example, Generic, 1.0, Container…). This is an optional field that is not generated for all file types and subtypes. | string | | json | Returns all available information about the alert, serialized in JSON format. | string | | label | Name of the alert subscription provided in the *Name* field of the *New Subscription* dialog. Users can modify the name for all alert subscriptions on the appliance. | string | | malware_type | The malware type part of the threat name detected for the sample that triggered the alert (for example, Trojan, Adware, Rootkit…). | string | | platform | The platform part of the threat name detected for the sample that triggered the alert (for example, Win32, Script-PHP, Linux…). | string | | sha1 | SHA1 hash of the sample for which the alert was triggered. | string | | subplatform | The subplatform part of the threat name detected for the sample that triggered the alert (for example, HTML, Macro, PDF…). The subplatform part is present in the threat name only for specific platforms (ByteCode, Document, Script). | string | | timestamp | Date and time when the alert was triggered. | UTC timestamp | | uuid | Unique string identifying the triggered alert. | string | ### Modifying Existing Alert Subscriptions Only the user who created a subscription can modify it. To modify an existing alert subscription, click the *Edit* link next to the subscription name in the *Subscriptions* tab on the *Subscriptions and Actions Management* page. The dialog that opens is the same as for [creating a new subscription](#creating-a-new-alert-subscription). Here you can enable more, or modify any existing event types, or add/remove delivery methods. If a YARA ruleset with an active subscription gets deleted from the appliance, the dialog will warn the user when modifying the subscription for the missing ruleset. ### Deactivating Alert Subscriptions There are several ways to deactivate an existing alert subscription. 1. Click the *Edit* link next to the subscription name in the *Subscriptions* tab. In the dialog that opens, modify the *Actions* field and remove all selected actions from it. The subscription itself remains on the appliance, and can be reactivated at any point by adding new actions. 2. Click the *Delete* link next to the subscription name in the *Subscriptions* tab. This completely removes the subscription from the appliance. Alerts for the removed subscription will no longer be sent. ## Managing Alerts The Alerts page displays all generated events on the appliance. Users receive alert emails only for those events that match the criteria defined in their subscriptions. Alerts are displayed in a list and sorted by the time when they occurred (in descending order, with newest alerts at the top). The list contains information about every alert, including: - alert resolution status (unresolved alerts are indicated by a blue stripe to the left of the table) - type of alert; - threat status indicator of the sample associated with the alert; - time when the alert was triggered; - full notification text - threat name (if detected); - file format and size; ![Overview of the Alerts page](images/analyze-alerts-page.png) Clicking the sample name in the notification column opens the [Sample Details](./Sample%20Details/sample-details-summary.md) page for the selected sample. Clicking any of the sample rows will expand it, showing additional information about a sample like in other pages on the appliance. The action menu (☰) to the right of the *Resolved* column contains the *Configure* item, which opens the [Subscriptions and Actions Management page](#managing-alerts). The total amount of triggered alerts on the appliance is displayed in the pagination section at the bottom of the page. ### Filtering Alerts The toolbar above the list of alerts allows the users to filter alerts: - by status (All, Resolved, Unresolved) - by event type (All, Classification change, Sample available for download, YARA match, Cuckoo analysis complete, Spectra Intelligence upload complete, Spectra Intelligence analysis complete); - by classification of the sample associated with an alert (All, Unknown, Goodware, Suspicious, Malicious); - by the time when the alert was triggered (All, Custom, Today, Last Week…); - by user or by alert subscription When a subscription is deleted from the appliance, it is no longer visible in the user/subscription filtering menu. However, alerts triggered by that subscription will still be visible when the *My Alerts* option is selected. ![Subscriptions menu that allows filtering alerts by user-created subscriptions](images/analyze-alerts-my-alerts.png) Filters can be combined to get a more precise overview of the triggered alerts (for example, show all YARA match alerts for subscriptions created this week that returned malicious samples). When the *Classification change* alert type is selected in the toolbar, there is an additional option to choose which classification changes should be listed (for example, from no threats found to malicious or from suspicious to malicious). For the *YARA match* alert type, an additional field *Enter ruleset name* becomes available, allowing users to display only the alerts related to a specific YARA ruleset. ### Exporting Alerts The *Export* pull-down menu in the toolbar allows copying the list of SHA1 hashes or exporting the list of alerts as a CSV file. Users can select which columns should be included in the exported CSV file. Only the current page will be copied/exported, not the entire list of all alerts on the appliance. The number of alerts per page - that is, the number of alerts that will be included in each exported file - can be increased or decreased in the navigation bar at the bottom of the Alerts page. ### Marking Alerts as Resolved When an incoming alert has been acknowledged and dealt with, it is possible to mark it as *Resolved* to keep the Alerts page up-to-date. To mark an alert as *Resolved*, select the *Resolve* option in the action menu on the right. The blue highlight will then be removed from the alert. ![Triple bar menu on the Alerts page with the Resolve option highlighted](images/analyze-alerts-resolve.png) It is also possible to resolve multiple alerts at once. This is done by selecting several alerts and choosing the *Resolve selected* option from the topmost action menu (☰) in the Alerts page toolbar. Only the current page of alerts can be resolved. Up to 1000 alerts can be resolved at once by increasing the amount of alerts per page in the navigation toolbar to 1000, then selecting and resolving all alerts on the page. When the selected alerts are resolved, a notification appears with the total number of resolved alerts. If some alerts had previously been resolved, the appliance will detect this and will not count them towards the total. Filtering the Alerts page by resolution status will show only resolved alerts, hide resolved alerts, or show all alerts on the appliance. The filter can also be used to show and export all resolved alerts as described in the [Exporting Alerts](#exporting-alerts) section. Users can also undo the alert resolution by clicking *Unresolve* (for a single alert) or *Unresolve Selected* (for multiple alerts) in the action menu on the Alerts page. ### Automatically Removing Alerts **Only appliance administrators can enable this option.** Users who would like to enable this option should contact their appliance administrator. Depending on the number of alert subscriptions configured on the appliance, alerts can quickly accumulate, making it difficult to browse the Alerts page. To prevent this, appliance administrators can define how long the alerts should be kept on the appliance. When the alerts are older than the configured period, they are automatically removed from the appliance. The alert retention period can be configured in the **Administration > Configuration > Alert Management** dialog. By default, alerts are kept for 3 months and automatically deleted after that period. Other supported options are 1 month and 6 months. ## Monitoring Samples with Alerts Apart from creating and editing alert subscriptions on the [Subscriptions and Actions Management page](#managing-alerts), users can quickly modify subscriptions by adding new samples from different sections of the GUI. This is useful for adding interesting samples to an existing alert, or for creating a new alert that will monitor the activity on multiple samples. The following table lists the interface sections where samples can be added to a subscription using the *Subscribe* option, or removed from a subscription using the *Unsubscribe* option. The table also indicates which sections support bulk actions (adding/removing multiple samples to/from a subscription). | Section | Supported Alert Types | Bulk Actions | | -------------------------------- | ---------------------------------------------------- | ------------ | | Advanced Search Results - Local | Classification change, Sample available for download | Yes | | Advanced Search Results - Cloud | Classification change, Sample available for download | Yes | | YARA ruleset matches | Classification change, Sample available for download | Yes | | Sample Details | Classification change, Sample available for download | No | | Sample Details > RHA results | Classification change, Sample available for download | Yes | | Sample Details > Extracted Files | Classification change, Sample available for download | Yes | | Tags | Classification change, Sample available for download | Yes | To start monitoring a sample, open the action menu (☰) on the right side of the sample and select the *Subscribe* option. ![Section of the Spectra Analyze Web Interface with Subscribe option highlighted in the Sample Actions menu](images/analyze-alerts-subscribe.png) This opens the *Subscribe to Hash Alert* dialog that allows creating a new alert or selecting an existing alert from the pull-down list. Supported alert types are **Classification change** and **Sample available for download**. The dialog for creating a new alert is the same as in the [Creating a New Alert Subscription](#creating-a-new-alert-subscription) section. If an existing alert is chosen in the pull-down list, the SHA1 hash of the selected sample is automatically added to the alert subscription. Users can then modify other fields in the subscription dialog, such as actions, before clicking *Submit* to save changes. ### Subscribing to Samples in Bulk It is also possible to add more than one sample to an existing subscription, or to create a new subscription that will monitor several samples. When multiple samples are selected on a page, the bulk actions menu (☰) is available in the header row of the sample list. To add the selected samples to an alert subscription, click the *Subscribe* item in the bulk actions menu. Sample hashes can also be added to an existing alert subscription manually. To do that, access the *Subscriptions* tab on the [Subscriptions and Actions Management page](#managing-alerts) and click the *Edit* link next to the desired subscription. In the dialog that opens, paste the SHA1 sample hashes into the *Sample hash values* text field and click *Submit*. Each hash should be in its own line in the text field. ### Removing Samples from Alert Subscriptions Samples can also be removed from existing subscriptions. To remove one or more samples, use the *Unsubscribe* item from the sample action menu (or the bulk actions menu, if two or more samples are selected). A dialog appears, prompting the user to confirm the action. Clicking *Unsubscribe* in the confirmation dialog removes the selected sample hashes from all subscriptions to which they were added. The subscriptions themselves are not removed or deactivated, and if they were configured to monitor other samples, the alerts will still be sent out for those samples. Alternatively, samples can be removed from alert subscriptions manually, by editing the subscription from the *Subscriptions* tab. Click the *Edit* link next to a subscription, and in the dialog that opens, remove sample hashes from the *Sample hash values* text field. ## Tracking Unknown Hashes with Alerts When searching for sample hashes on the appliance, the results can sometimes include unknown hashes. Those are hashes of samples that have not yet been found in the ReversingLabs system, or that are not available locally. (Note: this does not refer to hashes found in the system, but classified as Unknown.) If the list of search results includes unknown hashes, they will be listed separately and accessible by clicking the *Not Found* filter/tab on the search results page. Users can subscribe to those hashes and get notified when they appear in the ReversingLabs system for the first time. Supported alert types for unknown hashes are **Classification change** and **Sample available for download**. Unknown hashes can be added to new or existing alert subscriptions in the same way as described in the [Monitoring Samples with Alerts](#monitoring-samples-with-alerts) section. ## Setting Up Alerts for New YARA Ruleset Matches The list of YARA rulesets on the YARA page also integrates with the Alerts feature. Apart from subscribing to individual samples from the list of matches on a YARA ruleset, it is also possible to subscribe to rulesets themselves. This type of alert is triggered when there is a new match for the subscribed ruleset. To subscribe to a YARA ruleset from the YARA page, open the action menu (☰) on the right side of the ruleset and select the *Subscribe* option. This opens the [New Alert dialog](#creating-a-new-alert-subscription) with the YARA Matches event type enabled, and the selected YARA ruleset automatically filled in. Alternatively, users can create YARA ruleset match alert subscriptions from the [Alerts Configuration page](#managing-alerts). --- ## Spectra Analyze Dashboard — Submission Stats, Trends & PDF Reports The **Dashboard** displays statistics related to the amount and type of files that have been [submitted](./file-submissions.md) and processed on the appliance within a specified time range. ## Dashboard UI | UI element | Description | | ---------- | ------------------- | | ![Advanced Search box](images/a1000-dash-advancedsearch.png) | [Advanced Search box](./#advanced-search-box) | | ![Time range filter](images/a1000-dash-timefilter.png) | [Time range filter](./#time-range-filter) | | ![Download Report button](images/a1000-dash-downloadreport.png) | [Download Report button](./#download-report-button) | ### Advanced Search box The [Advanced Search box](./advanced-search.md) is a text field where users can enter search queries and get a list of all supported keywords. Recent search queries are preserved across the **Dashboard** and [**Search & Submissions page**](./search-page.md). Advanced search also supports [non-keyword](../advanced-search/#non-keyword-queries) searches, allowing the users to quickly build queries without using keywords. ### Time range filter The time range filter allows users to set the period for which the data is displayed on the dashboard page. All time values refer to UTC time. By default, the time range filter is set to **Last Week**. ### Download Report button Clicking the **Download Report** button exports the current state of the dashboard page into a PDF file. ## Analysis Statistics ![Dashboard section showing analysis statistics.](images/a1000-dash-1-analysisstatistics.png) **Analysis Statistics** breaks down the number of files submitted to the appliance into the total number of submitted files, total number of files extracted from submitted files, and total size of all submitted files. These are further categorized into analyzed emails, archives, apps and other file types. Finally, this section also displays the [classification distribution](/General/AnalysisAndClassification/Classification) of analyzed files in the selected time range and their corresponding graphs. Hovering over the graphs displays the number of samples with that classification at specific points in time. ## Top Malicious Family Detections by Malware Type ![Dashboard section showing top malicious family detections by malware type.](images/a1000-dash-2-malwaretype.png) **Top Malicious Family Detections by Malware Type** displays the most common malware types detected in files analyzed in the specified time range. The most prevalent threat families for each malware type are listed with the count of samples they were detected in. Clicking any of the entries in this section opens the advanced search page with local search results for the selected threat name and time range criteria. ## YARA Matches ![Dashboard section showing a tabular overview of favorited YARA rules.](images/a1000-dash-3-yara.png) **YARA Matches** show an overview of YARA hunting activities on the appliance and in the Spectra Intelligence cloud. Users can see a list of their [**Favorite**](../yara/#results) YARA rulesets or **Last Matched** rulesets. If there are no favorited YARA rulesets, the table defaults to the **Last Matched** view. Both table views can be further filtered by match sources. Filtering can be configured to show **All**, **Local**, **Cloud**, **Local Retro**, or **Cloud Retro** matches. Each table row shows the YARA ruleset name, its owner, one or more match sources, the number of matches per classification, the latest match time, and the number of new matches in the last 24 hours/7 days/30 days. Rules with matches in the last 24 hours have the label **NEW**. Clicking a specific ruleset name opens the ruleset on the YARA page. ## Timeline Analysis for Detected Malware Samples ![Dashboard section showing a tabular overview of all malicious samples on the appliance, broken down into categories depending on the time difference between their local classification time and the "first seen" date in the Spectra Intelligence cloud](images/a1000-dash-4-malwaresamples.png) **Timeline Analysis for Detected Malware Samples** displays all malicious samples present on the appliance, highlighting the difference between their local and cloud classification times. This section always shows all available information as it is not affected by the time range filter at the top of this page. The data set displayed here includes all malicious samples on the appliance, regardless of whether they were submitted to the appliance by the users or downloaded from Spectra Intelligence. Depending on the time difference between the local and cloud classification times, all malicious samples available on the appliance fall into one of the following categories. | Category | Description | | --------------------- | ------------------------------------------------------------ | | **Never Seen** | Samples classified as malicious by the appliance that were never seen in the Spectra Intelligence cloud. If the appliance is not connected to Spectra Intelligence, all malicious samples are categorized as **Never Seen**. | | **More Than 30 Days** | Samples that were first seen in the Spectra Intelligence cloud more than 30 days after local classification as malicious. | | **30 Days Prior** | Malicious samples that were first seen in the Spectra Intelligence cloud 7-30 days after local classification as malicious. Apart from the aggregate number of samples on top, this section is broken down into columns with daily totals. | | **7 Days Prior** | Malicious samples that were first seen in the Spectra Intelligence cloud up to 7 days after local classification as malicious. Apart from the aggregate number of samples on top, this section is broken down into columns with daily totals. | | **Same Day** | Samples that were locally classified as malicious and first seen in the Spectra Intelligence cloud on the same day. | | **7 Days After** | Malicious samples that were first seen in the Spectra Intelligence cloud up to 7 days prior to local classification as malicious. Apart from the aggregate number of samples on top, this section is broken down into columns with daily totals. | | **30 Days After** | Malicious samples that were first seen in the Spectra Intelligence cloud 7-30 days prior to local classification as malicious. Apart from the aggregate number of samples on top, this section is broken down into columns with daily totals. | | **More Than 30 Days** | Malicious samples that were first seen in the Spectra Intelligence cloud 30-90 days prior to local classification as malicious. | | **More Than 90 Days** | Malicious samples that were first seen in the Spectra Intelligence cloud more than 90 days prior to local classification as malicious. | ## Top Email Threats Collected During Sample Analysis ![A table showing a list of email samples, with associated threat names and counts.](images/a1000-dash-5-emailthreat.png) **Top Email Threats Collected During Sample Analysis** shows the list of email samples grouped by email address or email subject. It can be filtered and exported using the following options: - **Classification filter**: by default, the table shows email samples classified as malicious, but it can be filtered to show goodware, suspicious, or samples with no threats found. Setting this filter to *All Classified* shows all email samples regardless of classification. If this filter option is selected, some table columns are left empty because every extracted email address/subject can originate from samples with varying classifications, and data can’t be aggregated in a meaningful way. - **Filter by Email Address/Subject and Threat Name**: text fields that can be used to filter results by threat name and, depending on the option selected under *Pivot by*, either email address or subject. Both fields support autocompletion, and suggest only those options that exist in the table. Filtering by threat name is possible only when the table is filtered to show malicious or suspicious samples. - **Pivot by**: drop-down list which controls the contents of the *Pivoting Information* column in the table (email address or email subject), and affects what can be filtered using the email filter text field. - **Email address type filter**: available only when you select *Pivot by Email address*. This filter can be used to display samples that have the extracted email address in either the *To* or *From* field. This filter is disabled when the table is filtered to *Pivot by Email Subject*. - **Export**: export the entire table as a CSV file. To export a single page, select it by clicking the checkbox on the far left, and then click *Export*. Selecting and exporting individual table rows is currently not supported. The table data is organized into the following columns: - **Checkbox**: click to select a single page when exporting data. Selecting and exporting individual table rows is currently not supported. - **Risk Score**: all emails are assigned a [risk score](/General/AnalysisAndClassification/Classification/#risk-score) that corresponds to the severity value of the most prevalent threat name detected in samples containing the extracted email address or subject. - **Last Seen**: indicates when the appliance last recorded the sample(s) containing the threat. - **Threat Name/Threat Count**: the name of the threat with the most detections related to each entry, and the number of samples containing both the pivoting information and the detected threat. Click the threat name in this column to perform an [Advanced Search](./advanced-search.md) query. - **Total Classified**: shows the number of classified email samples that contain the listed pivoting information. The title of this column dynamically changes to match the classification selected in the *Classification filter*. - **Type**: this column is visible when using *Pivot by Email Address* and shows the type of email. - **Pivoting Information**: contains the email addresses/subjects, depending on the option selected in the *Pivot by* filter. To see a list of samples containing a specific address/subject, click the links. Email-related tags are displayed below each of the entries. Click any of the links or tags to perform an [Advanced Search](./advanced-search.md) query. - **Total Emails**: total number of samples containing the pivoting information. ## Top URIs Collected During Sample Analysis **Top URIs Collected During Sample Analysis** shows a list of malicious and/or interesting URIs extracted from files analyzed in the specified time range. It can be filtered and exported using the following options: - **Filter by Threat Name**: select a threat name from the drop-down list, or type a threat name into the text field. This field supports autocompletion, and suggests only those threat names that can be found in the table. To reset the filter, select “–” from the drop-down list. - **Type**: from the drop-down list, select **Domain**, **Link** or **IP** to show URIs recognized as domains, links or IPv4 or IPv6 addresses. Click **All** to reset the filter to show all URIs. - **Origin**: from the drop-down list, select whether to see URIs extracted as a result of static analysis, dynamic analysis, and/or those that have been extracted from malware configuration files. All three origin sources are enabled by default. - **Export**: export the entire table as a CSV file. To export a single page, select it by clicking the checkbox on the far left, and then click *Export*. Selecting and exporting individual table rows is currently not supported. The table data is organized into the following columns: - **Checkbox**: click to select a single page when exporting data. Selecting and exporting individual table rows is currently not supported. - **Risk Score**: all URIs are assigned a [risk score](/General/AnalysisAndClassification/Classification/#risk-score) that corresponds to the severity value of the most prevalent threat name detected in samples containing the extracted URI. - **Last Seen**: indicates when the appliance last recorded the sample(s) containing the URI. - **Threat Name/Threat Count**: if any threats are found within samples containing a specific URI, this column lists the most prevalent threat name, and the number of samples containing both the URI and the detected threat. Click the threat name in this column to perform an [Advanced Search](./advanced-search.md) query. - **Total Count**: shows the number of samples containing the URI. Click the URI in the *Pivoting Information* column to perform an [Advanced Search](./advanced-search.md) query. - **Type**: categorizes the URI as *Domain*, *Link*, or *IP*. URIs recognized as links have a **Download** button next to them, allowing the user to download and submit the URI for analysis. If the URI points to a web page, the appliance crawls its contents and downloads the web page as a compressed file. The page is crawled one level deep, regardless of the domain. URIs pointing to files are treated as single file downloads/submissions. All downloaded URIs can be found on the [Advanced Search](./advanced-search.md) page. - **Pivoting Information**: contains all or specific URIs extracted from files on the appliance, depending on the option selected in the *Type* filter. To see a list of samples containing a specific URI, click the links. URI-related tags are displayed below each of the entries. Click any of the tags to perform an [Advanced Search](./advanced-search.md) query to find all files tagged with the same System Tag in the time range selected on the dashboard. - **Origin**: indicates the source from which the appliance extracted each URI. Supported sources are static analysis results, dynamic analysis results, and malware configuration files. --- ## Analysis services ## Overview Spectra Analyze supports optional integration with multiple first-party and third-party static and dynamic analysis services. Through these integrations, samples can be automatically submitted for dynamic analysis or reanalyzed on demand using any of the supported services. All integrations work with samples submitted through the graphical user interface, as well as with those submitted via the [Submissions API](../API%20Documentation/submissions/). Analysis results are organized by analysis type in the navigation bar on the left of the [Sample Details Page](../Sample%20Details). First-party integrations are: - [ReversingLabs Cloud Sandbox](#reversinglabs-cloud-sandbox) for dynamic analysis. - [ReversingLabs Auxiliary Analysis](#static-analysis) for static analysis. Third-party integrations are always used for dynamic analysis, and they are: - [CAPE Sandbox](#cape-sandbox) - [Cisco Secure Malware Analytics](#cisco-secure-malware-analytics) - [Cuckoo Sandbox](#cuckoo-sandbox) - [FireEye Sandbox](#fireeye-sandbox) - [Joe Sandbox](#joe-sandbox) - [VMRay](#vmray) ## Integrations configuration Analysis services must be configured on the **Administration > Integrations & Connectors > Integrations** page by an administrator. For more information, see [Integrations](../Administration/integrations-connectors/integrations). ### Notes on configuration options #### Submitting distinct files Some analysis services have the option to submit only distinct files; however, it is disabled by default. Administrators can [enable this option](../Administration/integrations-connectors/integrations) to allow submitting only distinct files to an analysis service. When this option is enabled, if a file has already been submitted to Spectra Analyze and analyzed, it is not sent for reanalysis when it is submitted again. This option applies to files submitted using the GUI and the API. It does not affect the reanalysis feature - you can still submit files for reanalysis with any of the integrations even if the files have already been analyzed. #### Queue limits and behavior Some analysis services have queue limit for how many submissions can be simultaneously queued for analysis. Samples are considered queued if they are waiting for analysis or, in some cases, if they are in a processing state. If the queue is full, the appliance attempts to resubmit a sample up to five times, with a delay of 20 seconds between each attempt, before timing out. If it fails to resubmit the sample, that sample is removed from the queue. ## Dynamic analysis ### ReversingLabs Cloud Sandbox For more information about configuring this integration, see [ReversingLabs Cloud Sandbox](../Administration/integrations-connectors/integrations#reversinglabs-cloud-sandbox). Spectra Analyze is integrated with the ReversingLabs dynamic analysis API, providing historical information on all dynamic analyses performed on the detonated sample, with detected indicators of compromise available through [Advanced Search](./advanced-search.md) using the `uri-dynamic` and `ipv4-dynamic` keywords, as well as through sections on the [Sample Details Summary](../Sample%20Details/sample-details-summary/) page. **Warning: Prerequisites** For this service to be available, the appliance has to be connected to [Spectra Intelligence](../Administration/configuration-update/configuration/#spectra-intelligence). If the service is enabled, historic dynamic analysis results are shown for all samples that have them. #### Dynamic analysis reports Full report details consist of: - **General file details**: file name, hash, classification and other information. - **Screenshot gallery**: click the thumbnail on the right to open a gallery of all collected screenshots, with the option to automatically advance through them in a slideshow. At the top of the gallery dialog, users can switch between different analyses to see the related screenshots. - **History of dynamic analysis results**: table with the option to download dropped files and other artifacts for every individual analysis. These files are available for download for 1 year. If the analyzed sample is an email, this table also contains expandable sections with analysis reports for attachments and URLs found within the analyzed email sample. - **Additional information**: available in a tabbed section with specific information obtained in dynamic analysis. This section can be filtered to show information from all performed analyses, or from a specific analysis. See the [ReversingLabs Cloud Sandbox API sections](/SpectraIntelligence/API/DynamicAnalysis) for a detailed explanation of individual fields. On this page, you can also click the buttons in the top right corner to do the following: - **Reanalyze**: reanalyze the sample using the ReversingLabs Cloud Sandbox. - **Actions**: - **Create PDF/HTML**: download the report as HTML or PDF. When a PDF or HTML report is created, a new one cannot be created before 30 minutes have expired. Downloading HTML or PDF reports for dynamic analysis is also possible via API. If a new report export is necessary before the 30 minute period expires, use the [Dynamic Analysis Report API](./API%20Documentation/dynamic-analysis-api.md). - **Download Latest Dropped Files**: download the latest dropped files. - **Send Latest Dropped Files to Static Analysis**: send the latest dropped files to static analysis. ![Dynamic analysis report](./images/analyze-rlcs.png) #### Downloading artifacts After a dynamic analysis run is completed, the following artifacts are available for download: - **Screenshots** - **PCAP file** - **Memstrings** - **Dropped files** Downloading artifacts is possible only through the GUI. The artifacts depend on each dynamic analysis run and can be downloaded from the **History of Dynamic Analysis Results** table, while the dropped files are available for download in the **Dropped Files** tab. These files are available for download for 1 year, the standard retention period for the Cloud Sandbox. The artifacts are downloaded as `.7zip` archives and their password is ***infected***. #### Analysis options | Option | Description | |--------|-------------| | **Platform** | Determines the platform the sample is executed on. Available as a per-analysis setting during file submission, and default settings per file type can be configured under [Administration > Integrations & Connectors > Integrations](../Administration/integrations-connectors/integrations). | | **Timeout** | How long in seconds a sample can run in the sandbox virtual machine. Default: `200`, Min: `30`, Max: `500`. ReversingLabs Cloud Sandbox may allow the VM to run longer if additional runtime could extract more intelligence. The **timeout** value may differ from the total analysis time which is reported as **duration**, and which includes sample execution including any extra time granted by RLCS, VM restore, client setup, and post-processing, for example, sending dropped files to the host. | | **Custom filename** | A custom filename that helps track the file across ReversingLabs services. If omitted, the system applies smart sample naming automatically. | | **Internet simulation** | Defaults to `false`. If set to `true`, dynamic analysis is performed without connecting to the internet and uses a simulated network instead. Setting to `false` is the same as omitting it from the request. HTTPS traffic information is not monitored when using this parameter. | | **Network traffic obfuscation** | When internet simulation is disabled/omitted and samples are allowed to connect to the internet, network traffic obfuscation measures are implemented to protect the analysis infrastructure. Traffic obfuscation is achieved through VPN routing that masks the origin of network requests, preventing external entities from identifying the analysis environment. | | **Geolocation** | Geographic location associated with the sample's network activity, reflecting the configured country from which the network traffic is egressed, set via VPN or similar routing methods. | | **Interactive analysis** | Execute the sample and interact with it in an interactive session. Only available through **Analyze/Reanalyze** dialogs. If selected, a new tab opens with the interactive session. Once the session expires or is stopped by the user, its results are visible under [Dynamic Analysis > Cloud Sandbox](../Sample%20Details/dynamic-analysis-results/). | ![Interactive dynamic analysis session](./images/analyze-dynamic-interactive.png) #### Configuration settings For more information about configuring this integration, see [ReversingLabs Cloud Sandbox](../Administration/integrations-connectors/integrations#reversinglabs-cloud-sandbox). By default, Spectra Analyze automatically retrieves existing ReversingLabs Cloud Sandbox reports for files submitted to the appliance. If a file wasn’t previously scanned in the ReversingLabs Cloud Sandbox, it can be manually uploaded for dynamic analysis, or you can enable [Automatic upload](../Administration/integrations-connectors/integrations#integrations) to do this automatically. **Important: While the retrieval of existing reports is a basic Spectra Analyze feature, submitting files for dynamic analysis using the ReversingLabs Cloud Sandbox is available only as a feature preview with an upload limit of five samples per day. When the analysis quota is exceeded, the appliance shows a warning message whenever a new file is manually submitted for analysis. ** Full access to this feature is available at additional cost. For more information, contact [ReversingLabs Sales Support](mailto:insidesales@reversinglabs.com). If the [Automatic file retrieval](../Administration/integrations-connectors/integrations#integrations) option is enabled, all files dropped during dynamic analysis that are within configured file size limits are downloaded to the appliance and analyzed locally. To allow dynamic analysis results to affect the final sample classification, enable the [Include in classification](../Administration/integrations-connectors/integrations#integrations) option. When enabled, all future sample uploads, as well as any reanalyzed samples, may receive their final classification from the ReversingLabs Cloud Sandbox. Samples that already had a recent dynamic analysis classification before the option was enabled update their classification once their [Sample Details Summary](../Sample%20Details/sample-details-summary/) page is opened, or during regular appliance synchronizations with Spectra Intelligence. Only ReversingLabs Cloud Sandbox can be configured to affect the final sample classification. Other analysis results do not affect the overall final classification of the sample, but are, rather, another source of information for analysts. Administrators can also specify which [File types](../Administration/integrations-connectors/integrations#file-types) are automatically submitted for dynamic analysis. The maximum supported file size of each individual sample submitted to the ReversingLabs Cloud Sandbox is 400 MB. ### CAPE Sandbox For more information about configuring this integration, see [Integrations - CAPE Sandbox](../Administration/integrations-connectors/integrations#cape-sandbox). | Maximum supported file size | Submitting only distinct files | Queue limit & behavior | | --------------------------- | ------------------------------ | ---------------------- | | 400 MiB | [Supported](#submitting-distinct-files) | Up to 60 submissions. Samples are considered queued if waiting for analysis. Running/processing samples do not count towards limit. For more information, see [Queue limits and behavior](#queue-limits-and-behavior). | CAPE analysis reports are added under [Dynamic Analysis > CAPE](../Sample%20Details/dynamic-analysis-results/). CAPE offers two types of analysis: *Behavioral* and *Network*. If enabled by an administrator, there is also a **See Task on CAPE** button at the top right of the section. This button redirects to the CAPE web interface, where it is possible to see more information about the file, and compare it to other analysis results. ### Cisco Secure Malware Analytics For more information about configuring this integration, see [Integrations - Cisco Secure Malware Analytics](../Administration/integrations-connectors/integrations#cisco-secure-malware-analytics). | Maximum supported file size | Submitting only distinct files | | ------------------------------ | ------------------------------ | | 250 MiB | Not supported | When Cisco Secure Malware Analytics finishes processing a sample, a report with the analysis results is sent to the Spectra Analyze appliance. This report can be accessed under [Dynamic Analysis > Cisco Secure Malware Analytics](../Sample%20Details/dynamic-analysis-results/). Available reports from this integration include: - **Dropped files** - **Indicators of compromise** - **Networking** ### Cuckoo Sandbox For more information about configuring this integration, see [Integrations - Cuckoo Sandbox](../Administration/integrations-connectors/integrations#cuckoo-sandbox). | Maximum supported file size | Submitting only distinct files | Queue limit & behavior | | ---------------------------- | ------------------------------ | ---------------------- | | 400 MiB | Not supported | Up to 60 submissions. Samples are considered queued if waiting for analysis. Running/processing samples do not count towards limit. For more information, see [Queue limits and behavior](#queue-limits-and-behavior). | Cuckoo reports are added under [Dynamic Analysis > Cuckoo](../Sample%20Details/dynamic-analysis-results/). Cuckoo offers two types of analysis: *Behavioral* and *Network*. If enabled by an administrator, there is also a **See Task on Cuckoo** button at the top right of the section. This button redirects to the Cuckoo interface, where it is possible to see more information about the file, and compare it to other analysis results. ![Cuckoo results section with visible See Tasks on Cuckoo button](images/analyze-cuckoo-tasks.png) ![Cuckoo Web application interface with analysis results](images/analyze-cuckoo-page.png) ### FireEye Sandbox For more information about configuring this integration, see [Integrations - FireEye Sandbox](../Administration/integrations-connectors/integrations#fireeye-sandbox). | Maximum supported file size | Submitting only distinct files | Queue limit & behavior | | ---------------------------- | ------------------------------ | ---------------------- | | 100 MiB | Not supported | Up to 100 submissions. Samples are considered queued if waiting for analysis or already being processed. For more information, see [Queue limits and behavior](#queue-limits-and-behavior). | If FireEye is enabled by an administrator, the **Fetch profiles** button retrieves a list of profiles available on the FireEye instance. Supported file types can be assigned to profiles that are used for dynamic analysis. Each file type can be assigned to only one profile. New samples of the supported file type assigned to a profile are automatically sent for dynamic analysis. When FireEye finishes processing a sample, a report with the analysis results is sent to the Spectra Analyze appliance. This report can be accessed under [Dynamic Analysis > FireEye](../Sample%20Details/dynamic-analysis-results/). For more details on configuring and using the FireEye integration, contact ReversingLabs Support ([support@reversinglabs.com](mailto:support@reversinglabs.com)). ### Joe Sandbox For more information about configuring this integration, see [Integrations - Joe Sandbox](../Administration/integrations-connectors/integrations#joe-sandbox). | Maximum supported file size | Submitting only distinct files | Queue limit & behavior | | ---------------------------- | ------------------------------ | ---------------------- | | 400 MiB | [Supported](#submitting-distinct-files) | Up to 20 submissions. Samples are considered queued if waiting for analysis. Running/processing samples do not count towards limit. On timeout, displays "Failed Upload" status message under [Dynamic Analysis > Joe Sandbox](../Sample%20Details/dynamic-analysis-results/). If this happens, the failed sample no longer remains in the queue. For more information, see [Queue limits and behavior](#queue-limits-and-behavior). | If Joe Sandbox is enabled by an administrator, the **Fetch profiles** button retrieves a list of profiles available on the Joe Sandbox instance. Supported file types can be assigned to profiles that are used for dynamic analysis. Each file type can be assigned to only one profile. New samples of the supported file type assigned to a profile are automatically sent for dynamic analysis. Appliance administrators can check the status of the Joe Sandbox service on the [System Status](/SpectraAnalyze/Administration/usage-alerts/system-status/) page, under **External Services Connectivity**. Joe Sandbox analysis reports are added under [Dynamic Analysis > Joe Sandbox](../Sample%20Details/dynamic-analysis-results/). Clicking the section name in the sidebar opens the page with general information about Joe Sandbox, and details about the latest analysis. If enabled by an administrator, there is also a **See Task on Joe Sandbox** button at the top right of the page. ![Preview of the Joe Sandbox results on the Sample Details page](images/analyze-joe-sandbox-preview.png) The **Behavior Analysis** tab contains the process tree menu obtained from the Joe Sandbox JSON report. The **Network Analysis** tab displays all network activity detected during dynamic analysis. The following protocols are listed: TCP, UDP, DNS, HTTP, HTTPS, FTP, ICMP, IRC and SMTP. The **Domains/IPs/URLs** tab shows the extracted URIs in three separate tabs as they are differentiated in the HTML report. Public and private IP addresses are not in separate tabs; instead, they have a boolean attribute *Private* visible in the list. ### VMRay For more information about configuring this integration, see [Integrations - VMRay](../Administration/integrations-connectors/integrations#vmray). | Maximum supported file size | Submitting only distinct files | | ------------------------------ | ------------------------------ | | 305 MiB | Not supported | There is no need to retrieve available profiles/environments from VMRay and assign file types to specific platforms. Samples are sent to dynamic analysis according to how the VMRay instance is configured. When VMRay finishes processing a sample, a report with the analysis results is sent to the Spectra Analyze appliance. This report can be accessed under [Dynamic Analysis > VMRay](../Sample%20Details/dynamic-analysis-results/). ## Static analysis For more information about configuring this integration, see [Integrations - Static analysis](../Administration/integrations-connectors/integrations#static-analysis). ### Auxiliary analysis reports | Maximum supported file size | Submitting only distinct files | | ------------------------------ | ------------------------------ | | 100 MiB | Not supported | When ReversingLabs Auxiliary analysis finishes processing a sample, a report with the analysis results is sent to the Spectra Analyze appliance. This report can be accessed from the [Sample Details page](../Sample%20Details) under **Auxiliary Analysis**. The report for this integration includes the following report fields, including, but not limited to: general sample information, detected heuristics, ATT&CK information, extracted files, IOCs, threat score, and more. The threat score of each file, including the sample and all extracted files, indicates its severity and helps interpret the classification. | Score | Classification interpretation | |-----------|-------------------------------| | - 1000 | Clean | | 0 - 299 | Informational | | 300 - 699 | Suspicious | | 700 - 999 | Highly suspicious | | >= 1000 | Malicious | ### Auxiliary analysis services | Service name | Speciality | |:---------------------|:-------------------| | APKaye | Android APK | | Ancestry | File genealogy | | Batchdeobfuscator | Deobfuscation | | CAPA | Windows binaries | | Characterize | Entropy analysis | | ConfigExtractor | IoC extraction | | DeobfuScripter | Deobfuscation | | DocumentPreview | Visualization | | EmlParser | Email | | Espresso | Java | | Extract | Compressed file | | Floss | IoC extraction | | FrankenStrings | String extraction | | JsJaws | Javascript | | MetaPeek | Meta data analysis | | Oletools | Office documents | | Overpower | PowerShell | | PDFId | PDF | | PE | Windows binaries | | PeePDF | PDF | | PixAxe | Images | | Swiffer | Adobe Shockwave | | TorrentSlicer | Torrent files | | Unpacker | UPX Unpacker | | ViperMonkey | Office documents | | XLMMacroDeobfuscator | Office documents | --- ## File and URL Submissions ## Submitting Files for Analysis Files can be submitted to the appliance: - [manually](#manual-uploads) through the graphical user interface: - as a direct file upload - as a link to the file (which the appliance then downloads) - using the [Submissions API](API%20Documentation/submissions.md) - using [Connectors](/SpectraAnalyze/Administration/integrations-connectors/connectors/), which pull files from one or more configured sources - by pivoting from a previously analyzed sample: - from the [Spectra Detect Manager dashboard](/SpectraDetect/Usage/Dashboard/#product-integration-with-spectra-analyze) - from an S3 file link in a [Spectra Detect Worker report](/SpectraDetect/Usage/Analysis) - using the [ReversingLabs browser extension](https://chromewebstore.google.com/detail/reversinglabs-browser-ext/hdnaiimefhaiglkbjdcioegcoofbnomm). **Tip: The ReversingLabs browser extension is available on the [Chrome Web Store](https://chromewebstore.google.com/detail/reversinglabs-browser-ext/hdnaiimefhaiglkbjdcioegcoofbnomm). It enables you to query domains, URLs, IP addresses, and hashes from web pages and submit files for analysis.** ### Manual uploads ![An image showing the file submission dialog](images/analyze-uploads-add.png) Samples can be manually uploaded to Spectra Analyze from any page in the interface by clicking the **Submit** button in the header bar and selecting **File Analysis**. Alternatively, you can submit files programmatically using the [Submissions API](./API%20Documentation/submissions.md) as part of an automated workflow. In the File Analysis dialog, click **Browse Files** to upload files from your local system, or enter a direct File URL. Then click **Submit** to process the files using the current appliance configuration. To customize the submission, use the [link below](#advanced-analysis-options) the form to choose specific analysis services. If you're using the RL Cloud Sandbox, you can also specify a custom filename, the target platform, locale, geolocation, toggle Internet Simulation on/off, and set an execution timeout (between `30` and `500` seconds, the default value is `200`). and request an interactive analysis session. A progress bar in the header indicates the upload status while files are uploading. Navigating away from the page or refreshing the browser tab during upload is not supported and will cancel the upload. #### Advanced Analysis Options Uploaded samples will follow your current appliance settings, but you can make a one-time adjustment for each upload, such as sending the file to Threat Intelligence (Spectra Intelligence) for threat reputation info, or using one of the sandbox integrations. ![Additional options for file uploads](images/analyze-uploads-additional.png) - If **Local Only Analysis** is selected, the file is analyzed exclusively on the appliance. The sample is **not** sent to any configured integrations, sandboxes, or Spectra Intelligence services. - **Protected File**: Some security solutions opt to put suspicious and malicious files, such as email attachments, into password-protected archives before passing them on for further analysis, using the archive format as a secure means of transport. If you're submitting such a password-protected archive, you can also provide a one-time password here. **Note: This feature expects the ZIP file to contain **only one file** and will, upon successful extraction, upload only the extracted file and discard the archive. Only CRC32 encrypted ZIP files are supported. AES encryption is not supported at this time.** This specific unpacking mechanism is triggered by providing a password. For general password-protected archive usage uploads (where multiple files can be extracted and processed), perform a regular upload with a preconfigured password list (*Administration > Configuration > Password List*). ### File Processing When a file is submitted to the appliance, it is [processed with Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis). Depending on the appliance configuration, samples with supported file types can be automatically sent to dynamic analysis services after they are submitted to the appliance. The file becomes visible in the **Local** tab on the **Search & Submissions** page. Detailed analysis results can be viewed in the [Expanded Details](./search-page.md#expanded-details) section, or on the file's [Sample Details](./Sample%20Details/index.md) page. The duration of the analysis depends on file sizes and file types, as well as the number of files extracted during analysis. Extracted files are also analyzed separately. After initial processing by Spectra Core, users can optionally [submit it to Spectra Intelligence and/or supported dynamic analysis services](./search-page.md#reanalyzing-files-and-urls), if configured on the appliance. Spectra Intelligence scanners that support archive unpacking and heuristic analysis automatically perform those steps during processing. **Note: For CAPE and Joe Sandbox, previously analyzed files will not be automatically sent for analysis again if the *Submit only distinct files* option is configured. Administrators can configure this on the **Administration ‣ Integrations** page.** ### File Size Restrictions - The maximum supported file size for upload on Spectra Analyze is 10 GB. This value can be configured in *Administration > Configuration > General > File Size Limit.* - Files larger than 400 MB cannot be submitted for dynamic analysis (individual dynamic analysis integrations have even lower limits). - YARA rulesets are not applied to extracted files larger than 700 MB. New files cannot be submitted to Spectra Analyze if the disk space usage on the appliance exceeds the set value. If this happens: - [manually remove old samples](./search-page.md#deleting-submitted-files-and-urls) - ask the administrator to run the [Backup and Purge](/SpectraAnalyze/Administration/configuration-update/backup-purge/) action before continuing to submit new files - increase the disk size in *Administration > Configuration > Resource Usage Limits* (see [System Configuration](/SpectraAnalyze/Administration/configuration-update/configuration/#resource-usage-limits) for more information). ### Pivoting from Spectra Detect If Spectra Analyze is connected to Spectra Detect (either to individual Workers, or a cluster managed by Spectra Detect Manager), it can pull files from a preconfigured S3 bucket or directly from Spectra Detect Manager. Both of these options must first be [configured](/SpectraDetect/Usage/Dashboard#product-integration-with-spectra-analyze) on Spectra Detect. The pivot link is present in the dashboard of Spectra Detect Manager, as well as in the Worker JSON report under `file_link`. When you open the link, Spectra Analyze will pull the previously analyzed file from the preconfigured source and will reanalyze it. Imported files will be [tagged](../tags) with the `spectra_detect` tag. ## Submitting URLs for Analysis The URL analysis service provides comprehensive analysis of submitted URLs through advanced web intelligence gathering and threat detection capabilities. The service performs DOM analysis to detect malicious content, captures visual evidence, maps network infrastructure (IP addresses, DNS, SSL/TLS certificates, domain registration), and executes URLs in sandbox environments to observe runtime behavior and track redirection chains. URLs can be manually uploaded to the Spectra Analyze from any page of the interface by clicking the **Submit** button on the header bar and selecting **URL Analysis** from the menu, or via the [Submissions API](./API%20Documentation/submissions.md) as part of an automated workflow. In the URL submission dialog that opens, enter the full URL of a website including the protocol (*https://www.example.org*), or a full link to a single file (*http://www.example.org/documents/reports/year-report.pdf*). Supported protocols are HTTP and HTTPS. **Important**: Files are downloaded only from the submitted URL with no recursion (crawl depth = 1). For example, if you submit `http://www.example.com/freshcontent`, only that specific URL will be analyzed—`http://www.example.com/freshcontent/newest` will not be included. The service downloads and analyzes up to 50 samples per analysis (each up to 100 MB), with files processed through the ReversingLabs threat detection pipeline. ### Crawling Methods ### Crawling Methods Optionally, users can enable URL Crawling to download and analyze files from a submitted URL. - **Spectra Intelligence**: By default, URLs are crawled using the **Spectra Intelligence** crawling method. This requires Spectra Intelligence to be configured on the appliance. When using the Spectra Intelligence crawling method, users have the additional option of submitting the URL for [dynamic analysis](./dynamic-analysis.md#reversinglabs-cloud-sandbox) to the ReversingLabs Cloud Sandbox. In addition to automated dynamic analysis, users can choose to enable [interactive analysis](./dynamic-analysis.md#interactive-analysis), which provides manual control over the browser session during execution. - **Local**: If enabled by the appliance administrator, users can also select the **Local** crawling method. This method doesn't require Spectra Intelligence to be configured, and disables all Spectra Intelligence features, such as dynamic and interactive analysis. **Important: Privacy** For more information on these methods, refer to the [Privacy of Submitted Files and URLs](#privacy-of-submitted-files-and-urls) chapter. Click **OK** to confirm URL submission, or **Cancel** to close the submission dialog. The submission cannot be confirmed if the URL is invalid. The [Search & Submissions](./search-page.md) page displays comprehensive analysis results for the submitted URL, including all downloaded files, network intelligence data, visual evidence, and behavioral analysis findings. If any of the analyzed components are malicious or suspicious, the overall verdict for the URL reflects the highest threat level detected. The submission type indicator icon on the left side of the page helps distinguish between files downloaded to the appliance via a URL (the link icon) and files directly submitted to the appliance (the folder icon). **Analyzing Data from Submitted URLs** The analysis duration depends on multiple factors including the number of files downloaded (up to 50), their sizes and file types, DOM complexity, network infrastructure resolution, and dynamic execution requirements. Each downloaded file is also analyzed separately through the complete threat detection pipeline. The timeout for URL submissions is 45 minutes. URL submissions undergo comprehensive analysis including static file analysis with the Spectra Core engine, network infrastructure mapping, DOM analysis, and visual documentation. Users can manually send components for additional analysis to Spectra Intelligence and/or configured dynamic analysis services using the [Reanalyze option](./search-page.md#reanalyzing-files-and-urls). This integration with Spectra Intelligence and dynamic analysis services must be configured by appliance administrators. All files and websites downloaded to the appliance via the URL submission dialog are automatically assigned the *URL Download* User Tag. This tag is visible in the Expanded Details and on the Sample Details page for every file and website. Clicking the tag opens the Tags page filtered to display all files with the *URL Download* tag. Users can then sort the files and perform bulk actions, such as reanalyzing them or adding them to alert subscriptions. ### URL Submission Restrictions The URL analysis service has the following limitations: - **File Limits**: Up to 50 files can be downloaded and analyzed per URL submission, with each individual file limited to 100 MB - **Total Data Limit**: The maximum allowed size of all data downloaded from submitted URLs can be configured by the appliance administrator. By default, it is limited to 200 MB. This value is configurable by appliance administrators in the **Administration ‣ Configuration ‣ URL Analysis** dialog. The maximum configurable value is 700 MB. - **Crawl Depth**: Analysis is limited to the submitted URL only (crawl depth = 1) with no recursive crawling of linked pages In addition to these limits, submitting a URL using the Spectra Intelligence crawling method will also compare individual components of the submitted URL to the Maximum Fetch File Size value in **Administration > Configuration > Spectra Intelligence**. Any files going over this limit will be skipped. The maximum configurable value is 2000 MiB. If the download request fails, the URL submission is marked as failed. Users can attempt to reanalyze the submission by selecting the **Retry analysis** option in the actions menu (☰). This option is available for individual submissions only (not for multiple submissions at once). ## Privacy of Submitted Files and URLs **Note: Refer to the [Privacy](/General/SecurityAndAccessControl/Privacy/) chapter for more information and best practices.** ### File Submissions All files submitted to the appliance are accessible to all users with accounts on that Spectra Analyze instance. While each submission is associated with a particular user (the one who submitted the file or URL), actual files on the local appliance system are not owned by any of the users in the traditional sense of file ownership. Therefore, all users on the Spectra Analyze instance can download, reanalyze, subscribe/unsubscribe, add tags, and manually change classification for any file uploaded by another user. ### URL Submissions When submitting URLs for analysis, be aware of the following privacy implications: - URL analysis can only access and analyze publicly reachable online resources - All submitted URLs and downloaded files are treated as public and accessible to all Spectra Intelligence users - URLs are automatically normalized during submission, which may remove or convert duplicate and empty elements #### Crawling Methods Depending on which crawling method is selected, files obtained from the submitted URLs are treated differently. - The `Spectra Intelligence` crawling method (default) is more reliable when working in restricted network conditions and ensures fewer failed URL analyses. However, all downloaded files are treated as public, and will be visible and accessible to all Spectra Intelligence users. **The prerequisite for this is a properly configured Spectra Intelligence account on the appliance.** - The `Local` crawling method will treat the URL as any other locally submitted file. The contents of the URL are crawled and downloaded directly. This method can be used without a Spectra Intelligence account and must be enabled by the appliance administrator. If Spectra Intelligence is configured and is using a proxy, the same proxy will be used to crawl the URLs when using this method. Appliance administrators can delete files submitted by other users. Regular users can only delete their own submissions. ### Spectra Intelligence If the appliance is connected to Spectra Intelligence, all submissions can be: - Manually uploaded to be analyzed with AV engines. This is done with the [Reanalyze option](./search-page.md#reanalyzing-files-and-urls). - Automatically uploaded (*Administration > Configuration > Spectra Intelligence > Automatic Upload to Spectra Intelligence*). This is disabled by default. Whether submitted files will be shared with other ReversingLabs customers depends on the role configured for the Spectra Intelligence account used by the appliance. **Spectra Intelligence accounts created to be used with Spectra Analyze appliances are always configured as private (non-shareable)**, meaning that other ReversingLabs customers may only be able to access analysis results for the files, but not retrieve their contents. However, if those same files are uploaded to Spectra Intelligence as shareable from another source, they will cease to be treated as private. In that case, other ReversingLabs customers may be able to download the files, their metadata, and their analysis results through other ReversingLabs solutions (such as APIs and Feeds). If Spectra Intelligence is not configured on Spectra Analyze, files are only preserved on the local appliance system and accessible only to users on that instance. ### ReversingLabs Cloud Sandbox Whether submitted files, PCAP files, dropped files, and memory string dumps will be shared with other ReversingLabs customers depends on the role configured for the Spectra Intelligence account used to upload files. If the account is configured to upload all files as not shareable (private), other ReversingLabs customers will only be able to access analysis results, but not retrieve the actual contents of uploaded files, dropped files, PCAP files or memory string dumps. **This is the default setting for Spectra Intelligence accounts created to be used on Spectra Analyze appliances.** If the account is configured to upload all files as shareable (not private), other ReversingLabs customers will be able to access analysis results, but also download the uploaded files, dropped files, PCAP files, and memory string dumps generated during file execution. --- ## Graph — Spectra Analyze [PREVIEW] The Graph page provides an interactive visualization of relationships between malware samples, files, domains, IPs, and other entities. Users can create, view, and manage graphs to explore these connections. The landing page displays a list of saved graphs, each represented by a thumbnail preview, name, creator, and last modified date. Users can search for graphs using the search bar at the top. Select any existing graph or create a new one to get started. ### Graph Navigation ![A graph workspace, showing multiple nodes and an expanded sidebar.](./images/analyze-graph.png) Use the mouse to navigate: click and drag the empty canvas to move around, scroll to zoom in or out. Click and drag the nodes to move them around. Users can interact with nodes by clicking and dragging them. Select a node to open a [sidebar](#sidebar) on the right side of the canvas, providing information on that specific node. Right-click a node to access the context menu with options specific to that node. [Control nodes](#control-nodes), for example, offer options to load more child nodes or export a list of them. ### Nodes #### Root Nodes For graphs created by adding a single hash or by navigating from the sample summary page, the icon in the center of the graph represents the sample. Sample nodes will also be referred to as [root nodes](#root-nodes), as they serve as the starting points in the graph. To add more root nodes to the graph, use the search bar at the top of the page, or promote an existing file node to a new root node. File nodes can be promoted to root nodes via the sidebar or right-click menu. Selecting **Expand** or **Fetch & Analyze** (for cloud samples) transforms the node into a new root node, adding [control nodes](#control-nodes) and integrating its relationships into the existing graph. #### Control Nodes Root nodes branch into control nodes, each representing a distinct type of relationship. Control nodes act as entry points for exploring data. Each node is visualized as a color-coded pie chart showing the number of subnodes per classification: goodware, suspicious, malicious, or unknown. Initially, each control node displays up to 20 unique subnodes. For cloud samples, fewer control nodes are available until the sample is fetched and analyzed from either the node sidebar or the search page to display additional relationship data. The available control nodes are: - Dropped files - Extracted Files - Parents and Sources - (RHA) Similarity - Static Network References - Dynamic Network References - Network References: Contacted URLs, Domains and IPs ### Sidebar The sidebar provides details and actions based on the selected node. Control nodes show statistics on subnodes, extracted file nodes display file reputation and threat names, and network reference nodes provide third-party reputations and classification reasons. When available, network reference nodes for extracted and contacted IP addresses display their country of origin. To load more subnodes, select a control node and click **Show more**. Highlighted and underlined items in the sidebar can be clicked to navigate to a different sample summary page, or to perform an advanced search query. ### Filtering Graphs can be filtered by classification, file type or file name. The filtering options are accessible via the **Show Filters** button in the top left of the canvas. ![Graph filtering options](./images/analyze-graph-filter.png) ## Saving and Updating Click **Save** when working on a new graph to open the save dialog, allowing you to choose a name. When you open a saved graph, it will load all references from the previous session, plus any new ones discovered since then. --- ## Setup and initial configuration ## Setup and configuration checklist ✅ Ensure [system requirements](#system-requirements) are met. ✅ [Import](#importing-the-appliance-image) the appliance image. ✅ Perform the [initial configuration](#initial-system-configuration-via-console) via console. ✅ [Log in](#first-login-to-spectra-analyze) and [update administrator credentials](#change-default-administrator-credentials). ✅ [License](#appliance-licensing) the appliance. ✅ Update your [user profile](#user-profile) as needed. **Info: For assistance with Spectra Analyze setup, contact [ReversingLabs Support](mailto:support@reversinglabs.com).** ## System requirements ### Minimum system requirements The minimum system requirements are: - 1 TB of SSD storage - 8 CPU cores - 32 GB RAM These are the minimum resource settings, and are fine for average use with dozens of samples per day. The settings might have to be adjusted upwards depending on the expected usage. ### Recommended system requirements For heavier usage, the recommended settings are: - 2 TB of SSD storage - 12 CPU cores - 64 GB RAM **Info: Spectra Analyze configured to use the recommended settings can analyze 7 or more samples per minute. File retention should be set to 7 days, as the performance drops off by 5-10% per week with longer retention times.** ### Hosted instances For hosted instances, ReversingLabs provides the following resources: - 512 GB of storage - 16 CPU cores - 64 GB RAM Hosted instances support up to 10 000 files per day with a 3-month retention period depending on the file size. ## Importing the appliance image The package contains the following files: ``` a1000.md5 a1000.ova ``` The .md5 file contains the plain text MD5 hash of the .ova file for file integrity verification. The .ova file utilizes the VMDK disk format and can be imported by ESXi version 5.5 or higher. The following screenshots show how to import the .ova file in vSphere Version 5.5, hypervisor ESXi 5.5: ![../_images/analyze-installation-1.png](images/analyze-installation-1.png)![../_images/analyze-installation-2.png](images/analyze-installation-2.png)![../_images/analyze-installation-3.png](images/analyze-installation-3.png)![../_images/analyze-installation-4.png](images/analyze-installation-4.png)![../_images/analyze-installation-5.png](images/analyze-installation-5.png)![../_images/analyze-installation-6.png](images/analyze-installation-6.png)![../_images/analyze-installation-7.png](images/analyze-installation-7.png) ## Initial system configuration via console For physical appliances, first connect a keyboard, mouse and display to access and interact with the console menu. When the appliance is running, either as a physical device or in the virtual environment, the following configuration screen is used to configure it after completing the installation. ![../_images/analyze-initial-console-setup.png](images/analyze-initial-console-setup.png) To select an option, enter the menu item number and follow the prompts. The menu contains the following options: 1. Change the hostname. 2. Select the network interface. 3. Set static IP. Mutually exclusive setting with dynamic IP via DHCP. 4. Set dynamic IP via DHCP. Mutually exclusive setting with static IP. 5. Configure 2 DNS servers. 6. Configure up to 4 NTP servers. 7. Configure SNMP and set the SNMP community string. 8. Change the password for the "admin" user. 9. Shut down or restart the appliance. To restart the appliance after modifying the settings, enter the number next to `Shutdown / Restart`, then answer "n" to the shutdown question and "y" to the restart question. The console configuration tool does not provide any detailed success messages in response to configuring the network settings. To continue working with the appliance after the initial setup, open the web browser and enter the appliance IP address displayed in the console configuration tool. ### Network ports The Spectra Analyze appliance supports the following ports for inbound connections: - 80/TCP and 443/TCP for connecting to the web interface. - 161/UDP for SNMP monitoring. Outgoing connections to the internet via the following ports are also supported: - 53/UDP for DNS. - 123/UDP for NTP. However, it is strongly recommended that users configure the system to use their own DNS and NTP infrastructure if necessary. For outgoing connections to Spectra Intelligence at **https://appliance-api.reversinglabs.com**, the destination port is 443/TCP. The DNS name is **appliance-api.reversinglabs.com** and the connection supports HTTPS only. ### IP/domain allowlisting It is necessary to allowlist certain IP ranges and domains for connectivity with our cloud-based services. The primary range to allowlist is Cloudflare's public IP range, as outlined in their [official documentation](https://www.cloudflare.com/en-gb/ips/). Additionally, allowlist the IP range `185.64.132.0/22`, which covers various ReversingLabs endpoints crucial for full functionality and communication between your systems and the Spectra Analyze appliance. Furthermore, if the **Enable Spectra Analyze Networking Toolkit** option is enabled, the appliance attempts to gather additional network data from trusted sources such as `whois`, `bgpview.io`, `GeoLite City`, and DNS services. ## First login to Spectra Analyze From a web browser, visit the host address configured in the [Initial system configuration via console](./initial-configuration.md#initial-system-configuration-via-console) section to access the login screen of the Spectra Analyze web interface. If the Spectra Analyze is configured to support OpenID authentication, additional login options are displayed under the default login form. ![../_images/analyze-login.png](images/analyze-login.png) Log in using the default case-sensitive administrator credentials for the Spectra Analyze web application, provided by [ReversingLabs Support](mailto:support@reversinglabs.com). Failed login attempts are recorded in the appliance logs and visible in the `/var/log/rlapp/rlapp.log` file. Once the user logs in, a popup window is displayed for all accounts on the appliance. The popup contains the latest release highlights, informing the users about the changes delivered in the new Spectra Analyze version. Users can dismiss the popup, and it is not displayed on subsequent logins. To view the popup again at any time, go to the appliance footer, and click **What’s New**. ### Change default administrator credentials #### Password After logging in, look for the password change notice at the top of the page. Click the ***your password*** link and change the administrator account password in the dialog that opens. Alternatively, from the top navigation bar go to the 👤 **> [User Profile](#user-profile)** page with additional configuration options, and update your password from there. If password requirements are configured on the appliance under [Administration > Configuration > Configuration & Update > Authentication](../Administration/configuration-update/configuration/#authentication), the new administrator account password must comply with them. #### Email The email address associated with the default administrator account should also be changed. If you need to reset your password, on the [Login](#first-login-to-spectra-analyze) screen, click the ***Forgot your password?*** link. The emails sent from the appliance go to the email address associated with the administrator account. To change the email address, go to [Administration > Users & Personalization > Users](../Administration/users-personalization/users/) and [search](../Administration/users-personalization/users/#searching-for-users) for the "admin" user. Click it to open the **Change user** dialog and [modify](../Administration/users-personalization/users/#editing-users) the email address specified there. ## Appliance licensing On first login after installing or updating the appliance, the appliance must be licensed. For detailed instructions, see [Licensing](../Administration/configuration-update/licensing/). ## User Profile To update the current user's profile, go to 👤 **> User Profile**. The **User Profile** page contains options for configuring the profile of the user currently logged into the appliance, and for managing alert subscriptions. ### Profile Under **Profile**, modify the current user's profile. Apart from setting a first name, last name, and email address, users can also change the password for logging into the appliance. If password requirements are configured on the appliance, the new user account password must comply with them. The **Profile** section displays a warning if the new password does not satisfy the configured password requirements. To save your changes, click **Submit**. ### Alerts Configuration Under **Alerts Configuration**, you can see all alert subscriptions for the current user. For more information, see [Managing Alert Subscriptions and Delivery Methods](../alerts/#managing-alert-subscriptions-and-delivery-methods). --- ## Search & Submissions Page The Search & Submissions page is accessible from the top navigation menu. ![The search page focused on the network tab, showing a list of network resources submitted to the appliance.](images/analyze-search-network.png) At the top of the page is the [Advanced Search Box](#advanced-search-box). It is functionally identical to the one embedded in the global header bar on the other pages. Below the search box is the list of [recent submissions](#results-list) in the **Files** tab. The Files tab contains both local submissions and Spectra Intelligence results, which can be filtered using the Local/Cloud filter options. The **Network** tab lists URL, Domain and IP submissions. The appliance also crawls and downloads the submitted URLs to analyze them as files in the Files tab, so this makes it easier to distinguish between the downloaded content in the Files tab and the reputation of the network location itself. ## Advanced Search Box The *Advanced Search* box is a text field where users enter [search queries](./advanced-search.md#how-to-write-search-queries) and get a pull-down list of all [supported keywords](./advanced-search.md#supported-search-keywords). To quickly position the cursor into the search box, press ALT+S. This also applies to the search box present on the other pages of the Spectra Analyze interface. Alternatively, use the [Quick Search](#quick-search) dialog to construct a query, or the filters below the search box to add more search criteria. Read more about the filtering features in the [Filtering Results](#filtering-results) section. The maximum length of a single search query that can be entered into the *Advanced search* box is 1024 characters. For keywords that support searching by date, a date picker will open instead of the pull-down list. The *Advanced search* also has the following buttons: - the **preferences** button opens the [Quick Search](#quick-search): dialog where users can construct search queries using a graphical interface, perform [bulk searches](#bulk-search), and browse through suggested, recent and favorited search queries. - the **suggestions** button opens the Quick Search screen straight to the [Show Suggestions](#show-suggestions) tab. - the **share button** opens the dialog for sending the current query to other users via email. Available only when the users types into the *Advanced search* box. - the **star button** adds the current query and filter configuration to the list of Favorites displayed in the [Suggestions](#show-suggestions) tab of the [Quick Search](#quick-search) dialog. If the current page state is already saved as a favorite, the star will be yellow and open the “Edit favorite query” on click. Favorite queries can also be used as [a default filter](#changing-the-default-search-page-behavior) for the Search page. - the **Search** button (magnifying glass) runs the search query. Once the query is running, an option to stop it appears next to the **Files** and **Network** tabs below the search box. ### Quick Search The Quick Search dialog can be opened by clicking the Preferences button inside the Advanced Search box, or by pressing the ALT+Q keyboard shortcut. It contains multiple tabs. ![An image of the Quick Search dialog, currently showing the Files tab.](images/analyze-quick-search.png) #### Files The Files tab can be used to construct search queries without having to type them out manually. Users can fill out the form by entering values and picking options from drop-down menus, and then click the Search button in the bottom right of the dialog. #### Network The Network tab offers URL/Domain/IP search capabilities. Query can be filtered by classification, first seen date and resource type. #### Bulk Search The Bulk Search tab allows users to input up to 10 000 SHA1, SHA256, or MD5 hashes and search for them all at once. The results will be sorted into the Local and Cloud sections, depending on their availability. The list of hashes can be uploaded as a file or typed into the text field. In both cases, hashes should be separated by commas, spaces, or newlines. Mixing hash types is allowed, so all supported types of hashes can be submitted in one search request. The following examples illustrate different ways to separate hashes: ``` hash1,hash2,hash3,hash4 (...) hash1000,hash1001 ``` ``` hash1 hash2 hash3 hash4 (...) hash1000 hash1001 ``` ``` hash1 hash2 hash3 ``` To perform a bulk hash search, click the *Search* button in the dialog. Clicking *Cancel* closes the dialog. **Note: Bulk hash search differs from regular Advanced Search in several ways.**** - Bulk hash queries cannot be bookmarked in the browser or saved as favorites on the Spectra Analyze appliance. - Bulk hash queries do not appear in the *Recent queries* list. - Pivoting and combining search keywords with bulk hash queries is not possible. - Bulk hash search can take up to one minute, depending on the number of requested hashes. - Due to the way samples are processed in the Spectra Intelligence cloud, some discrepancy can arise between sample metadata in bulk search results versus regular search results. - By default, bulk hash search results are sorted by *First Seen* in descending order (most recent to oldest). This cannot be changed; the results page for bulk hash search does not support custom sorting by columns. - Different types of hashes can be submitted in one bulk hash query, but subscribing is only supported for SHA1 hashes. If multiple different hash types are selected in the list of search results, and the user attempts to subscribe to them, only the SHA1 hashes will be added to the subscription. Other hash types will be automatically filtered out from the subscription. **Tip: Instead of using the Bulk hash search dialog to look up large numbers of different hash types, the **hashes** keyword can be used for a similar purpose. The **hashes** keyword allows mixing different types of hashes in one search query, without the need to explicitly name the hash type or to group hashes by type. All hash types (MD5, SHA1, SHA256) can be used with this keyword.** An example query: `hashes: [, , , , , ]` When the **hashes** keyword is used (or when a query contains only hashes), the search automatically switches to Bulk Search, bypassing the 1024-character limit of standard queries. ### Show Suggestions Clicking the `Suggestions` button to the right of the search box will open the Quick Search pop-up window directly to the Suggestions tab, containing trending, recent and favorite advanced search queries. The tab is divided into three sections, from left to right: - **Interesting search examples** - predefined search queries that users can run to get a better understanding of the data in the results. - **Recent queries** - a list of up to 20 latest search queries performed on the Spectra Analyze instance. Clicking on any of the queries in the list automatically runs that query again, and moves it to the top of the list. Recent queries are deduplicated, so consecutively running the same query does not cause duplicate entries in the list. All recent queries from this list are also displayed in the pull-down keyword list when the user starts typing into the search box. The list of recent queries is private and visible only to the Spectra Analyze user who performed them, not to all Spectra Analyze users. - **Favorites** - a list of up to 20 user-defined favorite search queries. Users can save any query as a favorite by clicking the star button in the search box when the query is active. When a query is added to the Favorites list, clicking it automatically runs that query. Hovering over a favorite in the list displays a triple-dot menu with options to edit the query (change its name, search keywords, parameters…), copy it to clipboard, remove it from the Favorites list, or share it via email. The list of favorites is private and visible only to the Spectra Analyze user who created it; in other words, every user has their own Favorites list. Favorite queries can also be used as [a default filter](#changing-the-default-search-page-behavior) for the Search page. To start using Advanced Search, click any of the predefined search examples, or start typing into the *Advanced search* box to [create a custom search query](./advanced-search.md#how-to-write-search-queries). ### Keyword Auto-Suggestion Pull-Down List As soon as the user starts typing into the search box, the auto-suggestion pull-down list matching either the [supported search keywords](./advanced-search.md#supported-search-keywords) or their predefined values will open. User queries will be matched against keywords and their predefined values starting from any position, and not only from the beginning. For example, entering `aliciou` as a search query will match and return the entire `classification:malicious` keyword and value pair. Click the *…more* link to expand the list of keywords. Every keyword has a short explanation with examples on the right side of the list. Some keywords offer predefined values that the user can select from the pull-down list. The pull-down list also displays 20 most recent search queries performed on the Spectra Analyze appliance. Recent queries are offered as auto-suggestions when the user starts typing into the search box (if they match some of the text the user has typed). ### Date and time picker When a keyword that supports searching by date and time is typed into the search box, the date picker menu opens instead of the usual pull-down list. The user can select any of the predefined date and time values, or click “Custom” to enter their own date and time range. ![Date picker pull-down with predefined time ranges](images/analyze-search-date.png) ### Query Builder [PREVIEW] Query Builder is an alternative way of constructing and writing search queries. When activated in the gear menu on the Search Page, queries are represented as tags or blocks, allowing for better context awareness and auto-completion when adding new items in the middle of the query. Hovering the mouse between any two blocks displays a “+” button which can be used to insert an additional item anywhere in the query. Clicking the “X” button on a specific block deletes it. While the search box is focused, tags can be navigated using the arrow keys on the keyboard, and deleted using Backspace or Delete. ## Threat Intelligence Cards Threat Intelligence Cards make use of extensive ReversingLabs file metadata to provide an informative, educational overview and analytics on malware types and families in an easily accessible format on the Search page. The prerequisite for this feature is a properly configured Spectra Intelligence account on the appliance. When users perform an exact search with the `threatname` keyword for a single threat name or a list of threat names, one or more collapsible sections become available above the list of search results. At most 10 cards will be displayed, even if more threat names were provided in the query. ![A threat intelligence card displaying information about the Emotet malware family.](images/analyze-threat-intelligence-card.png) Depending on available data, Threat Intelligence Cards will either show information about a specific malware family (for example, Emotet), or a more generalized description of the family type (trojan). Cards can contain all, or a subset, of the following information: - first and last seen dates - current latest version - packer - targeted OS - targeted industries - threat name - threat name alias - actor aliases - malware family/type description - reference links - malware prevalence graph - top file types used by this malware family - weekly rankings for the current and the previous week - overall total number of unique samples containing this threat. To better reflect the popularity of the malware family in the Spectra Intelligence cloud, the **Current week** and **Last week** columns count all sample uploads and increase on repeated uploads of the same sample. The **Total number of detected samples** column shows a deduplicated count of unique samples belonging to that malware family. The malware family/type descriptions displayed here are also visible on the Sample Details > Summary page in the Malware Description section. ## Results List **Note: The terms “samples”, “files”, “submissions”, and “search results” are often used interchangeably in the appliance interface and in this documentation.** The list of files and URLs submitted to the Spectra Analyze can be found on the Search page, below the [Search box](#advanced-search-box). Drop-down menus under the Search box allow [filtering](#filtering-results) of the results list by multiple criteria. The results section of the page is divided into these general sections: 1. The action bar with the filter tabs. The action bar allows filtering the results by location (local or cloud) and availability (public or private); exporting them; configuring how the dates will be displayed in the results grid and if optional columns will be displayed. 2. Filter tabs: - **Files** - **Local** - filters the search results to display only the samples found on the local Spectra Analyze instance. The tab also shows the number of results. - **Cloud** - applies only to Spectra Intelligence search results. The maximum amount of Cloud results that can be returned for a search query is 100 000. Although the number may indicate there are more samples in the Spectra Intelligence cloud, Spectra Analyze will only allow browsing through 100 000 of them. - **Network** - lists URL, Domain and IP submissions. The appliance also crawls and downloads the submitted URLs to analyze them as files in the Local tab, so this makes it easier to distinguish between the downloaded content in the Local tab and the reputation of the network location itself. 3. Bulk Hash search only tabs: - **Not found** - displayed only when the list of results contains hashes that were not found in the ReversingLabs system. For Local results, *Not found* refers to hashes not found on the current Spectra Analyze instance. For Cloud results, *Not found* refers to hashes not found in the Spectra Intelligence cloud. - **Invalid** - displayed only when the list of results contains incomplete or erroneous hashes. It helps users identify potential formatting mistakes in their lists of hashes. 4. The samples/results list 5. The navigation bar at the bottom The samples/results list contains basic information about submitted files and URLs, or matching local/Spectra Intelligence search results after performing a search. Local results are sorted by *Last Submitted* by default, even if that column is disabled in the gear icon menu in the top right of the submissions grid. Spectra Intelligence results are sorted by *First Seen* date. Selecting the main checkbox in the topmost row will select all submissions on the current page. When submissions are selected, their highlight color changes to yellow. **Note: To improve search query responsiveness and performance, Cloud results prioritize *First Seen* within the last month by default. In case there are no results, the results page automatically extends the search query backward up to one year, until there are results. If still no results are found, users can choose to expand the search further by clicking the `Extend Search Timeframe` option. This progressively expands the search, year by year, until first results are found.** The list consists of the following columns: **Selection checkbox** - for selecting individual submissions on the page to perform bulk actions on them. When one or more items are selected on the page, an additional actions menu (☰) with bulk actions becomes active to the right of the *Size* column. **Submission type indicator** - the icon indicating whether the submission is a file or a URL. **Classification indicator** - classification status of submitted files and URLs is indicated by colored symbols: red square is malicious, orange rhombus is suspicious, green circle is goodware, black square with a circle cutout is no threats found. Inside of each of these symbols is the sample’s [risk score](/General/AnalysisAndClassification/Classification/#risk-score). Samples with no threats found (samples without classification) don’t have a risk score. Samples with a risk score of 5 are represented using a unique icon, as the indicators found during analysis were deemed insufficient to convict the file as definitively malicious or benign. These samples may be of interest, as they have a higher chance of changing classification and/or risk factor as soon as any new information becomes available. **First Seen** - time when a sample was first received by Spectra Intelligence (for Cloud results) or uploaded and analyzed on the appliance (for Local results). Open the gear icon menu next to the *Export* button on the far right of the page to change how the dates are displayed in this column (exact or relative dates). **Threat** - detected threat name for a submission, formatted according to the [Malware naming standard](/General/AnalysisAndClassification/Classification/#malware-naming-standard). Threat names are displayed only for malicious and suspicious samples. If the threat name is not detected for a suspicious submission, it receives a generic “Suspicious” threat name label. Clicking the name performs an Advanced Search for samples with the same threat name. **Name** - file name of the submission (as uploaded). For URLs, the file name corresponds to the URL submitted by the user. If the file name doesn’t contain any of the standard characters ([A-Z, a-z, 0-9]); for example, if the file name is !!!, its SHA1 hash is displayed instead to make it easier to click. Clicking the name opens the [Sample Details](./Sample%20Details/index.md) page with more information about the submission. Predicted filenames are displayed under the original filenames. They are generated by [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis), and can be helpful for distinguishing files that only have a hash value as their filename. Clicking a predicted filename redirects to the Advanced Search page and performs a search query to find other samples with the same predicted filename. The predicted filename won’t be shown if it’s a hash value. **Last Submitted** - indicates when the submission was last submitted to the appliance. Open the gear icon menu next to the *Export* button on the far right of the page to change how the dates are displayed in this column (exact or relative dates). The **Last Submitted** column can also be enabled/disabled from this menu. **Username** - indicates which Spectra Analyze user submitted each file or URL. This column can be enabled/disabled by clicking the gear icon next to the Export button. **Format** - file format of the submission, either represented as a combination of file type and subtype (e.g., PE/EXE) or as an identified format (e.g., NSIS, PDF). URLs submissions not pointing to a single file are represented as ZIP files in this column. **Files** - total count of files extracted from the submission, including the original uploaded file. **Size** - indicates the size of the submitted file, or of the contents downloaded from a submitted URL. If the file size is undefined for a Cloud result, the “Sample size currently unknown” message is displayed in the *Size* column on the Cloud results page. Those samples will be unavailable for download (the *Fetch & Analyze* option will be disabled) because it cannot be determined whether they exceed the maximum file size that can be downloaded from Spectra Intelligence (500 MB). ### Local Sample Actions **Actions menu** (☰) - contains various actions the users can perform on each submission. The actions include: - **Download sample** – downloads the submission from Spectra Analyze to local storage (compatible with modern ZIP tools such as `unzip` 6.00+, Python's `zipfile`, 7-Zip, or `bsdtar`). - **Download container** - if the file is a child (extracted from another file during analysis), downloads its top-most parent from Spectra Analyze to the local storage. - **Set classification** - opens a dialog where users can manually override the classification assigned to a submission. - **Subscribe** and **Unsubscribe** - adds or removes the sample from alert subscriptions. - **Reanalyze** - opens a floating dialog where users can reanalyze the submission with static, dynamic, or Spectra Intelligence analysis services. - **Delete sample** - removes the submission from Spectra Analyze. This action is visible to users only for files and URLs they have submitted themselves. Regular users cannot delete files and URLs submitted by other users. Administrators can see this menu item for all submissions, and delete all submissions regardless of who created them. - **Copy to my submissions** - displayed only if the sample was submitted by a different user. Selecting this adds the sample to the submissions for the current user. - **Download Extracted Files** - displayed if the sample has any extracted files. Selecting this downloads files extracted from the sample to local storage. **URL** submissions have a differently organized menu with mostly the same options. The menu is split into two sections: **Payload actions** which features the options above (as well as the option to open the [Sample Summary page](/SpectraAnalyze/Sample%20Details/sample-details-summary/#report-summary) for the payload), and **URL actions** with the option to reanalyze the URL. ### Expanded Details Each result row on the Search page can be expanded to see more details about a submission. Click anywhere in a row when the cursor changes to an up-down arrow. Expanded rows for local samples contain more information and different actions than those of samples available in the Spectra Intelligence cloud. This section describes local results. For more information on Public/Private Spectra Intelligence results, refer to the [Differences Between Local and Cloud Results chapter](#differences-between-local-and-public-cloud-results). ![Expanded row for a sample on the Search page](images/analyze-search-local-expanded-row.png) The information displayed here depends on the file type, and can include: - sample hashes - sample sources - first and last seen times (on the appliance and/or in the Spectra Intelligence cloud) - link to the parent file (if the file was extracted from another file) - RHA statistics (for supported file types) - User Tags and System Tags - comments for the sample (if any) - classification reason (see the [Threat Classification Descriptions](./Sample%20Details/threat-classification-sources.md#threat-classification-descriptions) section for more information about classification reasons) - number and percentage of Community Threat Detections - file format details - document metadata - email metadata - capabilities - indicators of interest - certificate information To add User Tags to a sample directly from the expanded details, click the **Add** link in the list of User Tags. Clicking the **Hex Preview** button opens the [Preview Sample / Visualization](./Sample%20Details/file-preview.md) pop-up modal. To see more information about a sample, click the file name to open the [Sample Details](./Sample%20Details/index.md) page. ## Filtering Results Users can filter samples by uploader (user accounts and connector services), as well as by a number of filtering criteria using the appropriate drop-down menus below the Search box. These filters vary depending on where the search query is being performed. When looking at **local** files, available filters are: time type and range, classification, submission user, processing status, and sample origin. Alternatively, instead of drop-down filters, use the following **local-only** search keywords: - `submission-user` - `submission-time` - `processing-status` - `tag-user` (not available as a menu selection) Being local-only, these keywords do not work on the **Cloud** tab. The classification filter uses the globally available `classification` keyword. The **Cloud** file list contains the time type and range filters, classification, and the cloud search dropdown, allowing the users to search for all cloud data, or just for private/public samples. **Note: Typing these keywords in a query will override and disable the respective drop-down menu under the search bar, to avoid filtering conflicts.** When added using the drop-down menus, the keywords will not be shown in the Advanced Search box as part of the query, but they will still be applied to the results, saved to the Recent queries list, and shared using the Share query button. In addition to these keywords, users can use any of the [supported keywords](./advanced-search.md#supported-search-keywords) to write [search queries](./advanced-search.md#how-to-write-search-queries) and search for samples available locally or in the Spectra Intelligence cloud. To reset the existing query and filters, click the **Clear** button next to the filters. The page also provides access to analysis reports ([Sample Details](./Sample%20Details/index.md)) and various actions that can be performed on each submission. ### Changing the default Search page behavior When the search page is opened, all of the configured filters on the page return to their default values. The gear icon next to the Export menu contains **Initial Search Results** options to change the default search page behavior: - **Use Default Filters**: If this option is selected, the search page will apply the default filters. - **Use Last Query**: If this option is selected, the search page will save the query and any filter configurations of the last performed query, and restore those values when the page is next opened. - **Use Favorite Query**: If this option is selected, users can choose a specific favorited search query from a dropdown list. This favorite will be used to configure the filters and populate the search box when the search page is next opened. - **Automatically Expand Search Period**: If this option is selected, the search period will expand until some results are found. This selection persists between sessions, and applies only to the current user. ## Exporting Results The **Export** menu contains options to export the whole page or just the selected samples. For the **Selected** samples option to become available, one or more samples on the page have to be selected. To export multiple pages of results, browse pages one by one and manually export them. It is possible to adjust the amount of results displayed per page in the navigation bar, thus increasing or decreasing the number of results that will appear in the exported file when exporting the entire page. Data can be exported as CSV, JSON or XML. To copy only the sample hashes for the desired set of samples, the menu contains options to copy SHA1, SHA256 and MD5 hashes to the clipboard. Hashes are delimited by a whitespace, so that they can be directly pasted into the search bar. The exported file can contain one or more of the following columns. They can be enabled in the **Export** menu, under **Show more Export options**. - **Files** - the number of files extracted from a sample (if the sample is a container). For Cloud results, this number is always 0, and they cannot be searched using the local-only **filecount** keyword. However, the Spectra Analyze interface displays `--` in the *Files* column for all Cloud results. For Local files, the correct file count is displayed, and extracted files can be browsed by clicking the file name and selecting the *Extracted Files* option on the Sample Details page. - **Format** - file format of each sample in the results grid. - **Name** - name under which a sample was uploaded to the appliance or to Spectra Intelligence. - **Threat** - detected threat name for malicious and suspicious samples, formatted according to ReversingLabs Malware Naming Standard. - **First seen** - time when a sample was first received by Spectra Intelligence (for Cloud results) or uploaded and analyzed on the appliance (for Local results). - **Size** - indicates the size of a sample. - **Available** - Spectra Intelligence results only. Indicates whether a sample can be downloaded from Spectra Intelligence. - **Classification** - indicates the classification status (malicious, suspicious, goodware, no threats found) assigned to a sample by any of the ReversingLabs classification sources and technologies. - **Last seen** - time when a sample was last received by Spectra Intelligence (for Cloud results) or uploaded and analyzed on the appliance (for Local results). - **Community Detections** - indicates the number of AV scanners that were used to analyze a sample in the Spectra Intelligence cloud. - **Risk Score** - the [numerical value assigned to a sample](/General/AnalysisAndClassification/Classification/#risk-score), representing its trustworthiness or malicious severity. - **SHA256, MD5** - additional sample hashes that can be included in the exported file. The SHA1 hash is always included by default. - **Last Submitted, Username** - available only for local results. If selected for Cloud results, the fields will be empty. ## Differences between Local and Public (Cloud) Results ### Spectra Intelligence Results List #### File format information In some cases, file format displayed on the Local files view can be different from the file format displayed on the Cloud files view for the same sample. Some samples will not have file format information at all. This happens when the information about a sample is returned from different sources. #### Extracted files information The number of extracted files for Cloud search results is always 0, but the Spectra Analyze interface displays `--` in the *Files* column for all Cloud results. #### Browsing and sorting Cloud results The maximum amount of Cloud results that can be returned for a search query is 100 000. Although there may be more samples matching the query in the Spectra Intelligence cloud, Spectra Analyze will only allow browsing through 100 000 of them. If the list of search results has more than 2000 samples, it is not possible to sort the results by any of the columns. ### Expanded Row Information Clicking a sample’s row in the list of results expands it to display more information about the sample. For Local results, this information includes the Spectra Core analysis report, and is generally more detailed. For Cloud results, the information in the expanded row includes sample classification received from Spectra Intelligence, sample hashes, first and last seen dates, and the results of AV scanning. Clicking the file name opens the Spectra Intelligence version of the [Sample Details](./Sample%20Details/index.md) page with more information about the sample. The **See All Scans** link leads directly to the Community Threat Detections page on the sample’s Summary page. ### Sample Availability The Local files view includes only those samples that are currently available on the local Spectra Analyze instance. The Cloud tab includes samples found in the Spectra Intelligence cloud. Select **Cloud Search** and pick *All Cloud Data* or further differentiate between public and private results. Private results may include files that have been deleted and are no longer available for download. Selecting **Public Cloud Data** shows data uploaded to Spectra Intelligence as shareable that may be available for download to the local Spectra Analyze instance. Alternatively, use the `available:true` keyword in the search query. Sample availability is indicated by a cloud icon on the left side of the results page. Samples with a black cloud icon are available for download; those with a grey icon are not. ### Spectra Intelligence Sample Actions The Cloud results page offers only the **Fetch and Analyze**, **Fetch and Analyze (Advanced)**, **Reanalyze in Spectra Intelligence** and **Subscribe/Unsubscribe** actions for each sample. **Fetch and Analyze**, **Fetch and Analyze (Advanced)**, and **Reanalyze in Spectra Intelligence** are available only for public and available samples (indicated by a black cloud icon). To learn how to subscribe to or unsubscribe from samples in the search results list, consult the [Alerts](./alerts.md) section. ### Bulk Actions for Managing Search Results Both Local and Cloud views allow users to select multiple samples at once and perform some actions on them. - **Local files view:** - Reanalyze - Download samples - Download Extracted Files - Apply User Tags - Set Classification - Delete - Subscribe - Unsubscribe - **Cloud files view:** - Fetch & Analyze (public results only) - Fetch & Analyze (Advanced) (public results only) - Subscribe - Unsubscribe To perform bulk actions, first select a number of samples to activate the bulk actions menu (☰). There are several ways to select multiple samples. 1. Manually select the checkbox on the left side of every sample. 2. Click the *Select all N samples* link that appears at the top when one or more samples are selected. This will select all samples on all pages in the results list. 3. Click the *Select All* checkbox in the top left corner of the results table. This will select all samples on the current page. Click the checkbox again to deselect all selected samples. When multiple samples are selected, a new bulk actions menu (☰) appears on the right side of the results page. Select the desired option in the menu. ![Bulk actions menu with option to download local samples highlighted](images/analyze-search-download-samples.png) ## Downloading Files from Spectra Analyze to Local Storage **Important: - Submissions selected for download are compressed and downloaded as a single ZIP file protected by a user-defined password.** - Currently, bulk downloads are limited to 15 000 files at once. The total combined size of all files selected for download should not exceed 4 GB. - If the appliance is unable to create the ZIP file within 20 minutes, the download process will time out. Files on the Spectra Analyze appliance can be downloaded to the local storage using the *Download sample* option from the actions menu (☰) for every submission (compatible with modern ZIP tools such as `unzip` 6.00+, Python's `zipfile`, 7-Zip, or `bsdtar`). Apart from downloading submissions one by one, users can also download multiple submissions at once (bulk download) to their local storage. This option is available from the **Local search results** on the **Advanced Search page\***, **YARA ruleset matches**, **Sample Details > File Similarity > Local** and from the **Submissions page**. There are several ways to select multiple items on the Search page: 1. Manually select the checkbox on the left side of every submission. 2. Select one or more submissions, then click the **Select all N samples** link that appears at the top of the page. This will select all submissions on all currently accessible pages. 3. Click the main checkbox in the topmost row of the submissions list. This will select all submissions on the current page. Click the checkbox again to deselect all selected submissions. When one or more items are selected, an additional actions menu (☰) appears highlighted on the right side of the page. Select the **Download samples** option in the menu. When the download process starts, a notification appears in the Spectra Analyze interface. The notification contains a link for canceling the download process. If the notification is dismissed (by clicking the X button), it will appear again when the page is refreshed, or when the user navigates to another part of the Spectra Analyze interface. When the download process is done, another notification appears even if previous notifications were dismissed. ## Downloading Container Files When a file is extracted from another file during analysis, the file from which it was extracted is referred to as its “container”. Extracted files have the **Download container** option in their action menu (☰) on the Search page. Selecting this option downloads the top-most container of the extracted file from Spectra Analyze to the local storage. If a file has been extracted from multiple different containers, the oldest one found on the appliance will be downloaded. Extracted files for which the top-most container has been removed from the appliance do not have this option in their action menu. The option to download the container is also available on the **YARA ruleset matches**, **Sample Details > Summary**, **Sample Details > Extracted Files**, **File Similarity > Local**, and on the **Local search results** page of the Advanced Search page. ## Downloading Files from Spectra Intelligence Apart from submitting files to the appliance manually or [through the API](./API%20Documentation/submissions.md), users can also obtain new files by downloading them from Spectra Intelligence. Downloaded files are stored on the appliance, and accessible from the **Local** files view on the Search page. The prerequisite is that the appliance is connected to Spectra Intelligence. In order to download a file from Spectra Intelligence, it has to be marked as available (in other words, uploaded to the cloud as shareable). The interface indicates this with a black cloud icon. Files with a gray cloud icon are marked as private, and are not available for download. Keep in mind that appliance administrators can configure the allowed file size for downloads from Spectra Intelligence. When this is configured, users can’t download files exceeding that file size. Regardless of the configuration, the maximum allowed file size is 500 MB per file. Files that are too large to be downloaded from Spectra Intelligence will have a special indicator icon instead of the black/gray cloud icon. Files can be downloaded from Spectra Intelligence to the Spectra Analyze appliance on the following pages: - Advanced Search results - Public tab - YARA > ruleset matches - Sample Details > File Similarity > Spectra Intelligence filter - Submissions > search results > Spectra Intelligence filter On each of the pages, files can be selected in several ways, depending on the page: 1. Manually select individual checkboxes on the left side of each file. 2. Select one or more files, then click the **Select all N samples** link that appears at the top of the page. This will select all files on all currently accessible pages. 3. Click the main checkbox (usually at the top left of the list). This will select all files on the current page. Click the checkbox again to deselect all selected files. When one or more files are selected, an additional actions menu (☰) appears highlighted on the right side of the page. Depending on the page, the following options may be available in the menu: - *Fetch selected samples* downloads the selected files from the current page to the Spectra Analyze appliance - *Subscribe* and *Unsubscribe* are used to [create an alert subscription for selected files](./alerts.md#monitoring-samples-with-alerts) Some pages on the Spectra Analyze appliance display the **Fetch All** button at the top right of the page when no files are selected. In those cases, clicking the button downloads all available files from the page to the appliance. It is also possible to choose an arbitrary amount of files by selecting individual checkboxes on the left side of each file. In this case, the **Fetch All** button changes to **Fetch Selected**. When downloading is confirmed, Spectra Analyze displays a notification about files being queued for download. Navigating away from the page will not stop the downloads. The process should only take a couple of seconds, after which the downloaded files will appear on the Search page. ## Managing Unknown (Not Found) Files **Note: “Unknown” in this context means “unknown to the system”, “not found in the system”. It does not refer to the *Unknown* status that files have when they are not classified yet.** Spectra Analyze can detect and track files that are currently not present on the appliance or in the Spectra Intelligence cloud. Users can look up hashes of specific files and set up alert subscriptions to get notified when the files become available in the cloud. **Search page** To track unknown files using the [Advanced Search](./advanced-search.md) feature, users should submit hashes through the *Bulk Hash Search* dialog. When one or more hashes are submitted through the *Bulk Hash Search* dialog, unknown hashes are automatically filtered out and displayed in a separate *Not found* tab on the search results page. When one or more **SHA1** hashes in the *Not found* list are selected, the actions menu (☰) provides options to subscribe or unsubscribe to them. Users can then create an alert description as described in the [Tracking Unknown Hashes with Alerts](./alerts.md#tracking-unknown-hashes-with-alerts) section. ## Deleting Submitted Files and URLs It is possible to manually delete files and URLs from the Search page. Users can delete only those submissions that they have added, while administrators (“superusers”) can delete any submission regardless of who added it to the appliance. There are several ways to delete a submission: 1. by selecting *Delete sample* in the actions menu (☰) on the Search page. This menu item is visible to users only for their own submissions. Administrators can see this menu item for all submissions. 2. by clicking a submission name to access its Sample Details page, then selecting *Delete sample* in the *ACTIONS* menu ![Sample Details Summary page with highlighted Actions menu](images/analyze-sample-details-actions.png) Multiple submissions can be deleted at once. To do this, select one or more items on the Search page using the checkboxes on the left side. Selected submissions are highlighted in yellow. To select all submissions at once, click the **Select all N uploads** message at the top of the list. Next, open the bulk actions menu (☰) highlighted on the right side of the list. Select *Delete* to remove selected submissions. A confirmation dialog appears, displaying the number of submissions to be deleted. Click **Yes** to proceed with deleting the submissions. Navigating away from the page will not stop the deletions. Clicking **No** or **Cancel** closes the confirmation dialog and returns to the Search page. ## Reanalyzing Files and URLs Users can perform additional analysis activities at any point after the files/URLs have initially been submitted and processed on the Spectra Analyze appliance. *Reanalyze* options are accessible on the following pages: - Advanced Search > Local, Cloud and Network tabs - the [Summary section](./Sample%20Details/sample-details-summary.md) of the Sample Details page - Extracted Files on the Sample Details page - The list of YARA matches (*local* and *local-retro* only) Depending on the currently focused search page tab and view, files/URLs can be submitted for reanalysis to the following supported services: ### Local Files View 1. Static Analysis - Spectra Core. Users can choose to reanalyze files with a newer version of the Spectra Core static analysis engine after updating the appliance. 2. Threat Intelligence - Spectra Intelligence. Users can submit a file to be reanalyzed with AV engines in the Spectra Intelligence cloud. 3. First-party analysis includes dynamic analysis with ReversingLabs Cloud Sandbox and static analysis through Auxiliary Analysis. Samples larger than 400 MB cannot be submitted to dynamic analysis services. When submitting a file to RL Cloud Sandbox, users have the option to launch an [interactive session](./dynamic-analysis.md#interactive-analysis) and interact with the sample directly. 4. Third-party dynamic analysis using supported analysis services: Cuckoo, FireEye, Joe Sandbox, CAPE, Cisco Secure Malware Analytics or VMRay. Samples larger than 400 MB cannot be submitted to dynamic analysis services (100 MB for FireEye). ### Cloud Files View 1. Reanalyze on Spectra Intelligence - submit the sample to a new Spectra Intelligence analysis. Use **Fetch & Analyze** or **Fetch and Analyze (Advanced)** to make the sample available locally for more options. ### Network Tab Click the **Advanced Analysis Options** to reveal the following settings: #### Dynamic Analysis - Cloud Sandbox - submits the URL for automated dynamic analysis in ReversingLabs Cloud Sandbox - Interactive Analysis - allows real-time interaction with the URL in a controlled sandbox environment - Third-party integrations - submits the URL to configured third-party dynamic analysis services #### URL Crawling - Analyze Crawled Files (Local, On-Device) - this option crawls the URL and performs static analysis on the downloaded content. This makes the downloaded payload available as a file in the Files tab. **This setting is disabled by default. Check with your administrator to enable it.** - Analyze Crawled Files (Cloud) - the URL will be crawled using Spectra Intelligence. ### Bulk Actions On the **Files** tab, up to 100 files can be selected and reanalyzed at once. When the *Reanalyze* option is selected, a floating dialog opens, where users can select which services to use for reanalysis. ![Dialog with options for reanalyzing samples](images/analyze-reanalyze-menu.png) Multiple services can be selected in the dialog. If any of the selected files have last been analyzed with a Spectra Core version older than the current version on the Spectra Analyze appliance, the Spectra Core static analysis service is selected by default. For other services, the date of the last successful analysis is displayed under the service name. The ReversingLabs Cloud Sandbox dynamic analysis service allows the users to select the platform to which the sample will be submitted: Windows 7, Windows 10, macOS Big Sur, or Ubuntu 20 (Linux). If a service is unavailable or misconfigured on the appliance, this is indicated by a status label under the service name. If a file is already being processed by a service when submitted for reanalysis, that service will be disabled. After selecting the desired service(s), confirm by clicking **Reanalyze** to submit files for reanalysis. Integrations and assessment analysis services have additional limitations. Consult the [Analysis Service Integrations](./dynamic-analysis.md) section for more details. **Important: Files without available sources cannot be reanalyzed using the *Reanalyze* option/dialog.** If file crawling is enabled, submitted URLs on the Files tab will behave exactly as files, so everything applies to URL submissions as well. To crawl a URL again, select the *Reanalyze* option in its action menu (☰). This will open the URL submission pop up window with the URL already inserted, allowing the users to select/change the crawling method. To see the URL network analysis results, switch to the *Network* tab. ## Reanalyzing Files and URLs in Error State In some cases, submissions on the Spectra Analyze appliance can remain in error state if there have been some issues during their previous analysis. Files in error state have a "Fail" indicator instead of the classification indicator on the Search page. Hovering over the indicator displays a tooltip with information about the last failed analysis. If the file name is crossed-out, that indicates the file doesn’t have a source on the appliance. Files without available sources cannot be submitted for reanalysis. To reanalyze a file or a URL submission in error state, select the *Retry Analysis* option in its actions menu (☰). This option is available for individual submissions only (not for multiple submissions at once). During reanalysis, users can normally browse pages on the Spectra Analyze appliance, including the Sample Details page of the submission that is being reanalyzed. Following a successful reanalysis, the submission will no longer display the *Fail* indicator, and will receive a classification from one of [the supported sources](./Sample%20Details/threat-classification-sources.md#threat-classification-descriptions). ## Analyzing Files with RHA Similarity Algorithm The [RHA (ReversingLabs Hashing Algorithm)](https://www.reversinglabs.com/technology/reversinglabs-hash-algorithm) is a functional similarity algorithm that classifies files based on the similarity of their features. Format-specific attributes of a file are abstracted into categories, and then evaluated for similarity at four different precision levels (25%, 50%, 75% and 100%). The precision levels represent the degree of similarity: the higher the level, the more similar attributes there are between files. **The Spectra Analyze appliance can display RHA information for PE, MachO, and ELF samples. Only the first precision level (25%) is supported for those file formats.** If RHA results are available for a file, they will be displayed next to a red RHA icon on the left side of the [Expanded Details](./search-page.md#expanded-details) section, in the sidebar of the [Sample Details](./Sample%20Details/index.md) page, and in the File Similarity section on the Summary section of the Sample Details page. ![The File Similarity section in the Sample Details page sidebar](images/analyze-file-similarity-sidebar.png) To get a list of functionally similar samples for a file: - click the links in the File Similarity section on the Sample Details page - expand the row on the Search page and click any of the numbers in the RHA section (above User and System Tags) ![RHA table with numerical callouts indicating RHA-related options](images/analyze-file-similarity.png) This will open the RHA results with samples functionally similar to the selected file. They can be filtered by threat status (All threats, Goodware, Unknown, Malicious, Suspicious) and by source (All, Local, Spectra Intelligence). Additionally, *Subscribe/Unsubscribe* options are available for those files, making it possible to create alert subscriptions for them. When the cursor changes to an up-down arrow in the RHA results list, click to expand the row and get more information about the file. ![Expanded sample row in the RHA table](images/analyze-expanded-similarity.png) Files marked with a gray cloud icon are unavailable for download, while those marked with a black cloud icon can be downloaded to the appliance. Clicking **Fetch All** at the top of the RHA results list will download all available files to the appliance, up to a maximum of 1000 files. It is also possible to manually select one or multiple files from the list and click **Fetch Selected**. The procedure is identical to the one described in the [Downloading Files from Spectra Intelligence](#downloading-files-from-spectra-intelligence) section. **Tip: When the functionally similar samples are downloaded and analyzed, it is useful to look at their “Protection” features and “Indicators” (in the Sample Details > Spectra Core section), and compare them.** --- ## Spectra Analyze Tags — System Tags, User Tags & Advanced Search The Spectra Analyze appliance supports two types of tags: System Tags and User Tags. Both types of tags are visible for every sample in the [Expanded Details](./search-page.md#expanded-details) and on the **Sample Details ‣ Summary** page. [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) automatically adds System Tags to new samples during analysis based on their metadata properties. It is not possible to change or add new System Tags to samples manually. Samples analyzed with Spectra Core version 3.7.1 will not have any System Tags. Users should submit such samples for reanalysis with Spectra Core to receive System Tags. User Tags are completely custom; any Spectra Analyze user can add and remove User Tags on all local samples on the appliance. User Tags are shared between all users of a Spectra Analyze instance. In other words, every user can see the tags created by other users, assign those tags to samples, or remove them from samples. User Tags are local-only and unique to each Spectra Analyze instance. They are not stored in the Spectra Intelligence cloud when a sample with User Tags is submitted for analysis. ## Searching for Samples by Tags There are several ways to find all samples tagged with a particular tag. 1. Use the Sample Search bar on the uploads page. Type in the hash symbol (#) followed by the tag (for example, #desktop). The search feature supports partial matching, so searching for #desktop will also find tags that contain the word “desktop” (for example, #desktop_app, #malicious_desktop, #zdesktop) 2. Or, click the tag in the list of User Tags on [Expanded Details](./search-page.md#expanded-details) or on the **Sample Details ‣ Summary** page In both cases, the Tags page will be displayed and filtered by the selected tag. This applies to both System Tags and User Tags. 3. If access to the [Advanced Search](./advanced-search.md) feature is enabled on the appliance, use the `tag` keyword to find samples with System Tags, and the `tag-user` keyword to look for samples with User Tags. Both keywords are case-insensitive and support wildcard matching. All samples with the queried tags are listed on the search results page, and divided into Local and Cloud depending on their location. ## Adding User Tags to Samples All Spectra Analyze users can create their own User Tags or add already existing ones to samples on the appliance. Keep in mind the following rules and limitations when creating new User Tags: - User Tags are case-sensitive and distinguish spaces from underscores. In other words, `Example` and `example` are two separate tags; `test_tag` and `test tag` are separate tags, too. - A single tag must be between 2 and 40 characters long (including spaces), matching the supported pattern. Supported characters for tags are [a-z0-9][a-z0-9 /._-]. The following examples are valid tags: `malware_1, ClassifiedByYARA, false-positive, Better.Test/Tag`. The following examples illustrate invalid tags: `Ž++, #hashtag, *.exe`. - It is possible to create a User Tag with the same name as a System Tag. For example, there can be an `indicators` System Tag, and a user can create their own `indicators` tag. Removing a User Tag does not affect the System Tag in any way. - If the user attempts to add a tag that is identical to a System Tag already added to the sample, it will be ignored (not added to the sample). However, it is possible to add a User Tag identical to a System Tag if the sample does not have that System Tag. For example, `antisandbox` is a System Tag. If a sample has this System Tag, attempting to add a User Tag called `antisandbox` will fail. If the sample doesn’t have it, `antisandbox` will be added as a User Tag. User Tags can be added to local samples (or removed from them) in several ways. **Adding User Tags to individual samples** 1. In the [Expanded Details](./search-page.md#expanded-details) for a sample, click **Edit Tags** button in the list of User Tags, or click the icon to the right of the list. This can be done from any Spectra Analyze page that supports Expanded Details (Search results, Alerts, YARA, Tags, Sample Details > Extracted Files). 2. Or, on the **Sample Details ‣ Summary** page, find the list of User Tags and click **Edit Tags**. 3. Or, on the Search & Submissions page, select a sample and click the triple bar button next to the *Size* column title. In the menu that opens, select the **Apply tags** option. In all three cases, the *Edit tags* dialog opens and displays the current list of User Tags if any are assigned to the sample. Start typing into the dialog and a pull-down menu with suggestions will appear if the letters match any of the existing tags. Select the desired tags from the pull-down, or type in the name of a new tag. To remove existing tags from the sample, if there are any, delete them from the input field in the dialog. Click **Save** to confirm the changes. 4. Use the [Modify tags API](./API%20Documentation/tags-api.md) to list User Tags for a sample, add new ones, and remove existing ones. **Adding User Tags to samples in bulk** Select one or more samples on any of the pages that support bulk tagging (Submissions, YARA, Tags, Sample Details > Extracted Files). While the samples are selected, choose the **Apply tags** option from the bulk actions menu next to the *Size* column title. This opens the *Edit tags* dialog. Start typing into the dialog and a pull-down menu with suggestions will appear if the letters match any of the existing tags. Select the desired tags from the pull-down, or type in the name of a new tag. To add multiple tags at once, separate them with commas (e.g., *tag1*, *tag2*, *tag3*). To remove existing tags from the sample, if there are any, delete them from the input field in the dialog. ![Dialog for adding tags with autocomplete pull-down menu](images/analyze-tags-new.png) Click **Save** to confirm the changes. The dialog will automatically close, and a notification will appear in the upper right corner, informing about changed tags. To add User Tags to samples from Spectra Intelligence, the samples must first be [downloaded to the appliance](./search-page.md#downloading-files-from-spectra-intelligence). When they are downloaded, User Tags can be added in any of the ways described on this page. --- ## Spectra Analyze Troubleshooting — Upload Failures, API Errors & Licensing # Troubleshooting This guide covers common issues with [Spectra Analyze](./index.md) and the steps to resolve them. --- ## File upload fails or hangs **Symptom** The upload progress bar in the header stalls or the browser shows no response after selecting a file via the **Submit** button. The upload may fail silently, or the browser displays a network error. **Cause** The most common causes are: - The file exceeds the configured maximum upload size. - The browser tab was navigated away from or refreshed during upload, which interrupts the transfer. - A network timeout or proxy between the browser and the appliance terminated the connection. - Disk space on the appliance is critically low, causing the upload to be rejected at the server. **Solution** 1. Check the appliance's configured maximum file size limit under **Administration > Configuration > General** and compare it to the file you are trying to submit. Files larger than this limit are rejected with an HTTP 413 error when submitted via the API. When submitted via the GUI, files exceeding the limit display a modal: "File Size Exceeds Spectra Analyze Upload Limit! Analyze large files with Spectra Detect!" 2. Do not navigate away from the page or refresh the browser tab while an upload is in progress. The upload status bar in the header indicates active transfers. 3. If you are uploading over a proxy or VPN, check that the proxy is not terminating idle connections prematurely. Increase the proxy timeout or upload directly if possible. 4. Check available disk space on the appliance using the [Health Status Indicator](./index.md#health-status-indicator) or the **Administration > System Status** page. See [Storage space warning](#storage-space-warning) for remediation steps. 5. For large files, use the [Submissions API](./API%20Documentation/submissions.md) directly with a tool like `curl`: ```bash curl -X POST https:///api/submit/file/ \ -H "Authorization: Token " \ -F "file=@/path/to/large-file.exe" ``` --- ## File stuck in "Processing" status **Symptom** A submitted file remains in the **Processing** state on the [Search & Submissions](./search-page.md) page for an extended period (more than 15–30 minutes for typical files) and never transitions to a classified state. **Cause** - The appliance analysis queue is backed up due to a large number of submissions. - A Spectra Core analysis instance has become unresponsive processing an unusually complex or deeply nested archive. - The appliance has insufficient CPU or memory resources to keep up with the current submission rate. **Solution** 1. Check the [Health Status Indicator](./index.md#health-status-indicator) for CPU, memory, and queue size warnings. Navigate to **Administration > System Status** for a detailed breakdown. 2. If the queue is congested, reduce the rate of new submissions and allow existing jobs to drain. 3. Check the appliance system logs for errors related to analysis workers. If an individual analysis instance is hung, the appliance will typically restart it automatically. 4. For files that are persistently stuck, try resubmitting the file after the queue clears. You can also use the [Reanalyze API](./API%20Documentation/reanalyze-api.md) to trigger reanalysis: ```bash curl -X POST https:///api/samples//analyze/ \ -H "Authorization: Token " \ -F "analysis=core" ``` 5. If the issue persists across multiple files, contact [ReversingLabs Support](mailto:support@reversinglabs.com) with system logs. --- ## Classification result is unexpected (false positive or false negative) **Symptom** A file is classified as malicious when it is known to be clean (false positive), or a known malicious file is classified as goodware or receives a low risk score (false negative). **Cause** - A custom YARA rule with overly broad pattern matching is triggering on clean files. - The file is new and has not yet been seen by enough classification sources to produce a confident verdict. - The classification was manually overridden by another user on the appliance. - The file is a compressed or packed variant not yet in the threat database. **Solution** 1. Review the **Classification Reason** in the [Report Summary Header](./Sample%20Details/sample-details-summary.md) to identify which classifier produced the final verdict. 2. If the classification originates from a YARA rule, navigate to the [YARA](./yara.md) page and review the matching ruleset. Tighten the rule conditions to reduce false positives. See [Interpreting Analysis Results](./getting-started.md#understanding-the-classification-reason) for guidance on evaluating classification confidence. 3. Check whether a user override is in effect. The classification sources table on the sample details page shows if a manual override is present. 4. For false negatives on known malware, check whether Spectra Intelligence enrichment is configured and reachable (see [Cannot connect to Spectra Intelligence for enrichment](#cannot-connect-to-spectra-intelligence-for-enrichment)). Intelligence feed data significantly improves classification confidence. 5. If the sample is a known false positive, apply a classification override directly on the sample details page or use the [Set Classification API](./API%20Documentation/set-classification-api.md). See [Administering classification overrides](./getting-started.md#administering-classification-overrides) for details. 6. Submit persistent false positives or false negatives to ReversingLabs for analyst review at [support@reversinglabs.com](mailto:support@reversinglabs.com). --- ## Cannot log in / authentication error **Symptom** Attempting to log in to the Spectra Analyze web interface returns "Invalid credentials", "Account locked", or redirects back to the login page without an error message. **Cause** - Incorrect username or password. - The user account has been disabled or locked after repeated failed login attempts. - An SSO or LDAP integration is misconfigured, causing authentication to fail silently. - The session cookie is stale or the browser has cached an old session. **Solution** 1. Verify you are entering the correct credentials. Passwords are case-sensitive. 2. Clear browser cookies and cache, then retry the login. 3. If the account is locked due to failed login attempts, it cannot be manually unlocked by an administrator. The account will automatically unlock after the configured block timeout expires. See [Login Security](./Administration/configuration-update/configuration.md#login-security) for details on the lock duration. 4. If LDAP or SSO integration is in use, verify the configuration under **Administration > Configuration**. Confirm that the LDAP server is reachable from the appliance and that bind credentials are correct. 5. If you have lost administrator access entirely, contact [ReversingLabs Support](mailto:support@reversinglabs.com) to arrange account recovery. --- ## Appliance shows license warning or expiry **Symptom** A banner appears in the interface warning that the license is expiring soon or has already expired. The [Health Status Indicator](./index.md#health-status-indicator) shows a red icon. Certain features or API endpoints may become unavailable. **Cause** - The appliance license has reached or passed its expiration date. - A new license file has been issued but not yet applied to the appliance. - The appliance cannot reach the ReversingLabs license server for validation (for cloud-validated licenses). **Solution** 1. Navigate to **Administration > Configuration > Licensing** to view the current license status and expiration date. See [Licensing](./Administration/configuration-update/licensing.md). 2. If you have received a new license file, upload it through the licensing page. 3. If the license appears valid but the warning persists, verify that the appliance can reach the ReversingLabs license server. Check for firewall rules blocking outbound HTTPS traffic. 4. Contact your ReversingLabs account manager or [support@reversinglabs.com](mailto:support@reversinglabs.com) to request a license renewal. --- ## API returns 401 Unauthorized **Symptom** API requests return: ```json {"detail": "Invalid token."} ``` or ```http HTTP/1.1 401 Unauthorized ``` **Cause** - The API token is missing from the request headers. - The token has been revoked or has expired. - The token is being passed incorrectly (wrong header name or format). **Solution** 1. Confirm that every API request includes the `Authorization` header with the correct token format, for example: ```bash curl -X POST https:///api/samples//analyze/ \ -H "Authorization: Token " \ -F "analysis=core" ``` 2. Verify that the token is still valid by checking **Administration > Tokens** if you manage tokens for multiple users. See [Tokens](./Administration/users-personalization/tokens.md). 3. If the token was recently regenerated or revoked, update all integrations and scripts that use the old token. 4. Check that the token belongs to a user account that is still active and has not been disabled. --- ## API returns 403 Forbidden **Symptom** API requests return: ```http HTTP/1.1 403 Forbidden ``` ```json {"detail": "You do not have permission to perform this action."} ``` **Cause** - The authenticated user's role does not grant access to the requested endpoint or resource. - A per-user or per-group permission restriction is in place for the specific operation (for example, deleting samples or accessing administration endpoints). **Solution** 1. Check the user's assigned role under **Administration > User Roles**. See [User Roles](./Administration/users-personalization/user-roles.md) for a full list of permissions associated with each role. 2. If the operation requires elevated privileges, ask an administrator to assign the appropriate role or a higher-privilege role to the user. 3. If the 403 is returned for a specific resource (for example, a private sample), check whether the sample's visibility settings restrict access. --- ## Slow analysis performance **Symptom** Files take significantly longer than expected to complete analysis. The queue displayed on the **Administration > System Status** page is consistently non-zero. Analysis that normally completes in seconds takes minutes. **Cause** - The appliance is receiving more submissions than its current Spectra Core capacity can handle. - Large or deeply nested archives (for example, multi-layer ZIP files) are consuming disproportionate analysis time. - CPU or memory resources on the appliance are under heavy load from other processes. - The appliance hardware does not meet recommended specifications for the current submission volume. **Solution** 1. Check CPU and memory utilization via **Administration > System Status** or the [Health Status Indicator](./index.md#health-status-indicator). Look for sustained CPU above 90% or memory above 85%. 2. Review the configured system health thresholds under **Administration > Configuration > System Health Indicator** and adjust alert thresholds to get earlier warnings. 3. Identify if specific file types (large archives, firmware images) are consistently slow. Consider rate-limiting submission of such files during peak periods. 4. If the appliance is persistently under-resourced, consider adding a Spectra Detect Worker cluster and connecting it via [Spectra Detect Manager](/SpectraDetect/Admin/ManagerSettings/) to offload processing. 5. Review the [platform requirements](/General/DeploymentAndIntegration/PlatformRequirements) to confirm the appliance meets minimum hardware specifications for the current submission rate. --- ## Storage space warning **Symptom** The [Health Status Indicator](./index.md#health-status-indicator) shows a red icon with a disk-related warning. The hover tooltip or **System Status** page displays a message similar to: ``` Disk usage: 94% — threshold exceeded ``` New file uploads may be rejected. **Cause** - The appliance disk has accumulated a large number of stored samples and analysis results. - Log files or system backups have consumed available space. - The configured disk usage threshold has been set too conservatively relative to actual growth patterns. **Solution** 1. Navigate to **Administration > Configuration > Backup & Purge** to configure automated sample purge policies. Samples older than a specified retention period can be removed automatically. See [Backup & Purge](./Administration/configuration-update/backup-purge.md). 2. To reclaim space immediately, navigate to **Administration > Configuration & Update > Backup & Purge** and click **Run Purge** to trigger a manual purge according to the configured retention criteria. 3. If log files are consuming space, rotate or archive them per your appliance OS procedures. 4. Adjust the disk usage alert threshold under **Administration > Configuration > System Health Indicator** if the current threshold triggers too early for your capacity. 5. If disk space is chronically insufficient, consider expanding the appliance storage volume or archiving older samples externally before deleting them from the appliance. --- ## Cannot connect to Spectra Intelligence for enrichment **Symptom** The [Health Status Indicator](./index.md#health-status-indicator) shows a red icon. The hover tooltip displays: ``` Spectra Intelligence: not reachable ``` Samples on the appliance lack Spectra Intelligence enrichment data such as scanner detections, threat names, or community classifications. **Cause** - The Spectra Intelligence credentials configured on the appliance are incorrect or have expired. - Network connectivity from the appliance to `data.reversinglabs.com` is blocked by a firewall or proxy. - The Spectra Intelligence account has exceeded its quota, causing requests to be rejected with HTTP 429. **Solution** 1. Navigate to **Administration > Configuration > Spectra Intelligence** and verify that the username and password are correct. Test the connection using the built-in connectivity check. 2. From the appliance, test outbound connectivity to the Spectra Intelligence API server: ```bash curl -I https://data.reversinglabs.com ``` If this times out or is refused, check firewall rules and proxy configuration. The appliance must be able to reach `data.reversinglabs.com` over HTTPS (port 443). 3. If a proxy is required, configure the proxy address in **Administration > Configuration > General** under the proxy settings, or contact your network team to allow direct HTTPS access to `data.reversinglabs.com`. 4. Check your Spectra Intelligence quota via the **Quota Indicator** in the appliance header bar, or via the [Customer Usage API](/SpectraIntelligence/API/CustomerUsage/tca-9999). If the quota is exhausted, contact [support@reversinglabs.com](mailto:support@reversinglabs.com) to review your entitlements. --- ## YARA Retroactive Hunting The YARA Retroactive Hunting feature is an extension of the already powerful yara capabilities available on the Spectra Analyze appliance. With standard YARA hunting, only new samples are compared against YARA rules, and new matches are reported as they happen. Retroactive Hunting allows users to scan through old samples to uncover YARA rule matches that would otherwise remain hidden. Three types of YARA Retroactive Hunting are supported on Spectra Analyze, with one being available only if the appliance is in a [Spectra Detect](/SpectraDetect/) cluster: - [Cloud Retro Hunting](#local-retro-hunting), which applies to samples in the Spectra Intelligence cloud that were analyzed in the past 90 days. Archives and samples larger than 200 MB are excluded from the sample set, but samples extracted from those archives are not. - [Local Retro Hunting](#local-retro-hunting), which applies to all samples on the local Spectra Analyze instance regardless of their age and analysis date. **Local Retro Hunts across large datasets are processing-intensive and may negatively impact the performance and stability of the appliance.** - **Remote Storage Hunting**, which applies to samples stored in S3 buckets configured on a Spectra Detect Hub group S3 connector service. Such retro hunts will be executed using the Worker appliances, and the results will be visible on the Manager dashboard. Links to remote storage retro hunts can be found in the **YARA Retro Hunt** pop-up dialog. ## Local Retro Hunting **Important notes** 1. Local Retro Hunting can be performed on all active rulesets on the appliance, including [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) rulesets. 2. Local Retro Hunting cannot be performed on individual rulesets - it runs on all active rulesets at once. 3. To exclude a ruleset from Local Retro Hunting, disable it before running Local Retro. 4. Local Retro does not submit samples to dynamic analysis services (such as Cuckoo or FireEye) during a scan. ### Starting a Local Retro Hunt To start a Local Retro scan on all active rulesets on the appliance, click the **Run Retro Hunt** button at the top right of the YARA page. This opens a pop-up with the options to perform a retro hunt on local samples, or to initiate a retro hunt on remote storage. **Tip: Performing retro YARA hunts on remote storage is available only if the appliance is in a Spectra Detect cluster comprising a Spectra Detect Manager, Hub and Worker with a configured S3 connector on the Hub group. Users can select the buckets from those added to the Hub group’s S3 connector and configure the desired file age and folder/prefix for the files to be included in the hunt. Such retro hunts will be executed using the Worker appliances, and the results will be visible on the Manager dashboard. Links to remote retro hunts can be found in the **YARA Retro Hunt** pop-up dialog.** ![Section of the YARA page with the Local Retro indicator](images/analyze-yara-local-retro.png) To stop a retro scan, open the **YARA Retro Hunt List** menu (the clock icon) and click the stop button next to the retro hunt listing. Local Retro does not submit samples to dynamic analysis services (such as Cuckoo or FireEye) during a scan. When the scan completes, the results will be available in the list of YARA matches for every active ruleset. The results for Local Retro scans cannot be cleared, and samples cannot be removed from the list of matches. ### Stopping and Restarting a Local Retro Hunt While a Local Retro scan is in progress, it can be stopped by clicking the stop button next to the retro hunt listing in the **YARA Retro Hunt List** dialog. After a Local Retro scan is stopped or completed, it can be restarted at any point. However, the next scan behaves as a fresh start - it does not continue from where it previously stopped. The *YARA Retro Hunt List* dialog contains detailed information about all previous Local Retro scans, including whether they were stopped or not. ## Cloud Retro Hunting **Important notes** 1. Cloud Retro Hunting can be performed only on rulesets that are saved in the Spectra Intelligence cloud, because the Cloud Retro sample set resides there. For a successful Cloud Retro scan, a ruleset needs to be able to access that sample set. 2. The Cloud Retro sample set includes samples analyzed in the Spectra Intelligence cloud over the last 90 days. Archives and samples larger than 200 MB are excluded, but samples extracted from those archives are not. The sample set is not static, and it changes daily. 3. Cloud Retro Hunting cannot be performed on Spectra Core rulesets. 4. The maximum number of Cloud Retro scans that can be executed depends on the license purchased by the customer. 5. The maximum amount of rules and rulesets that can be scheduled for Cloud Retro Hunting at a time depends on the license purchased by the customer. ### Starting a Cloud Retro Hunt To enable Cloud Retro Hunting for a ruleset, the selected ruleset must first be saved to Spectra Intelligence. To do this, open the ruleset editor by clicking the *Edit* item in the ruleset action menu (☰). In the editor, select the *Run ruleset continuously in Spectra Intelligence* checkbox and click the *Save* or *Save & Close* button. Alternatively, open the ruleset's page and enable the *Run Ruleset in Cloud* switch. The indicators on the YARA page should show that the ruleset is active in the cloud and ready for Cloud Retro. An indicator of synchronization status will be visible in the upper right part of the YARA page. The indicator displays the date and time of last synchronization, and allows skipping the synchronization until the next day. The action menu for the ruleset should now have the *Run Cloud Retro Hunt* option. Selecting it will schedule the ruleset for a Cloud Retro Hunting scan. If the ruleset has been successfully validated, the retroactive scan will start shortly. Alternatively, the appliance administrator can enable automatic Cloud Retro hunting for YARA rulesets saved to Spectra Intelligence in the **Administration > Configuration > YARA Cloud Settings** configuration dialog. If this option is enabled, YARA rulesets that are synchronized with Spectra Intelligence will be automatically scheduled for a Cloud retro scan that will start after successful ruleset validation. This applies to new rulesets created on the appliance, and to existing rulesets that are edited and synchronized with Spectra Intelligence by selecting the “Run ruleset continuously in Spectra Intelligence” checkbox in the YARA ruleset editor. ![Section of the YARA page with highlighted Run Cloud Retro Hunt option](images/analyze-yara-run-retro-sample.png) If new samples are uploaded to the Spectra Analyze appliance while a Cloud Retro scan is in progress, they will be also matched against the ruleset that is being scanned. During a Cloud Retro scan, the progress for the ruleset is displayed next to the status indicator icons on the YARA page. ![Section of the YARA page with Retro Hunt progress tooltip](images/analyze-yara-retro-running.png) When the Cloud Retro scanning for a ruleset is completed, the *RETRO* indicator on the YARA page will reflect that the scan was successful. Hovering over the indicator displays a tooltip with more information on the ruleset’s Cloud Retro status. If the scan has completed successfully, the tooltip will show the date and time when the latest Cloud Retro hunt started and finished. ### Stopping and Restarting a Cloud Retro Hunt It is possible to stop a Cloud Retro scan if the user needs to modify the ruleset for accuracy, or because of performance concerns. While a Cloud Retro scan is in progress, click the triple bar button (☰) on the right side of the selected ruleset. Choose the *Stop Retro Hunt* item from the menu. After a Cloud Retro scan is stopped or completed, it can be restarted at any point (as long as the ruleset remains saved to Spectra Intelligence). However, restarting a Cloud Retro scan on a ruleset will remove the results from the previous Cloud Retro scan. The restarted scan behaves as a fresh start - it does not continue from where it previously stopped. ## Managing YARA Retroactive Hunting Results To view and manage the results of Retroactive Hunting scans, click a ruleset name to expand the list of matched samples. Cloud Retro scan results will have their source displayed as a cloud icon. If the file is not available for download from Spectra Intelligence, a light gray cloud icon is displayed The list of matches for Retroactive Hunting contains similar options as described in the [Managing YARA Ruleset Matches](yara.md#managing-yara-ruleset-matches) section. Users can choose the ruleset version from the dropdown list below the ruleset name to compare scan results for each version. Every sample in the list of results can be selected and managed individually using the options in the action menu (☰). Clicking the file name opens the local or the Spectra Intelligence version of the [Sample Details](./Sample%20Details/index.md) page for the selected file, depending on if the sample is locally available. The Retroactive Hunting results can also be managed in bulk. Selecting one or more samples in the list of results activates the triple bar button (☰) on the right side of the *Size* column. The button opens the bulk action menu with options for: - downloading and analyzing selected samples on the Spectra Analyze appliance, if the samples are available for download (*cloud* and *cloud-retro* only), - removing samples from the list of matches (*cloud* and *cloud-retro* only), - reanalyzing samples (*local* and *local-retro* only), - applying tags to samples (*local* and *local-retro* only), - downloading samples to local storage, - adding/removing samples in new or existing alert subscriptions, - exporting data about selected samples as a CSV file. ### Removing Samples from the Cloud Retro Results YARA rulesets that are saved to Spectra Intelligence can match up to 10 000 samples. When a ruleset reaches that many matches, it will be capped and new matches will no longer be stored on the Spectra Analyze appliance. To free up space and continue collecting new matches for the ruleset, users have to remove at least 1000 old matches. - Samples can be removed from the list of Cloud Retro results manually by selecting them and choosing the *Remove Cloud Matches* option from the action menu (☰). Using this method, samples can be removed one by one, or the user can select and remove multiple samples at once. - Additionally, it is possible to clear all Cloud Retro results for a selected YARA ruleset. To do this, select a ruleset and click the triple bar button (☰) on the right side of the ruleset row. In the action menu that opens, select *Clear Retro Hunt*. ![Section of the YARA page with Clear Retro Hunt option highlighted](images/analyze-yara-retro-clear.png) When the Cloud Retro results for a ruleset are cleared, the icon for the *RETRO* indicator on the YARA page will change, showing that the results have been cleared. Users can then run a new Cloud Retro scan on the ruleset at any point. --- ## YARA Hunting ## Classifying files with YARA rules If a file has been classified by a YARA rule, the **Static Analysis > Classification** section on the [Sample Details](./Sample%20Details/index.md) page includes links to YARA rules that the file has matched. Clicking the link to a ruleset redirects to the YARA page containing the selected ruleset and its matches. To find files tagged by YARA rules on the appliance or in the Spectra Intelligence cloud, use the **tag-yara** keyword on the Search & Submissions page. For example, the search query `tag-yara:malicious threatname:*ransomware*` returns all files classified as malicious by YARA rules, with “ransomware” in their detected threat name. All files that match the YARA rule are automatically tagged with those custom tags after analysis. If a file has any YARA tags, they can be found in the **Static Analysis > Classification > YARA > Tags** section on the [Sample Details](./Sample%20Details/index.md) page. **Note: Changes to YARA tags are not immediately reflected in the Search results. When a YARA tag is added, changed, or removed from a rule, the files that match the rule must be reanalyzed with [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) first. Alternatively, you can also run a local [retro scan](./yara-retro.md#starting-a-local-retro-hunt).** For example, if a YARA tag “test” is renamed to “test2”, searching for *tag-yara:test* returns the same results until all matching files are reanalyzed with Spectra Core. After they are reanalyzed, the files can be found with the new tag, *tag-yara:test2*, and the old tag will not return any results. ## How YARA Hunting Works on Spectra Analyze Spectra Analyze contains default YARA rulesets, but also allows users to [create and edit custom rulesets](#creating-new-yara-rulesets), as well as to synchronize rulesets with other ReversingLabs products. The default YARA rulesets are called Spectra Core rulesets. They can be accessed by selecting the “Spectra Core Rulesets” option in the filter pull-down menu on the YARA Hunting page. Those rulesets are applied only to local files on the appliance (through Continuous Local Matching). Spectra Core rulesets cannot be saved to the cloud or modified (edited, disabled, or deleted) in any way by any type of user. Rulesets present on Spectra Analyze by default are open source, and published by ReversingLabs in [a public GitHub repository](https://github.com/reversinglabs/reversinglabs-yara-rules). Additional rulesets can be imported as files, or from online sources, straight from the YARA page. For more information on importing rulesets, refer to the [YARA Repository Management](#yara-repository-management) chapter. Files are matched against YARA rulesets on the Spectra Analyze appliance in four different ways. 1. **Continuous Local Matching (Default)** When a YARA ruleset is created on or uploaded to Spectra Analyze, it continuously looks for matches among local files that are added to the Spectra Analyze appliance after the ruleset has been created. This is the default behavior when Spectra Intelligence is not configured on the appliance, and when a YARA ruleset is not saved to Spectra Intelligence. Local matches for rulesets cannot be removed from the list of matches on the YARA page. 2. **Continuous Cloud Matching** When a new or existing YARA ruleset is selected to run continuously in the Spectra Intelligence cloud, it looks for matches among files that are added after the ruleset has been created. This includes local files added to the Spectra Analyze appliance, and files in the cloud uploaded through various ReversingLabs services (such as APIs). The prerequisite for Continuous Cloud Matching is that the Spectra Analyze appliance is connected to Spectra Intelligence. Important Note Every Spectra Analyze instance must be connected to a different Spectra Intelligence account. Sharing accounts between multiple instances can interfere with the functionality of the appliance (particularly with YARA rule synchronization). Appliance administrators should ensure that the cloud account credentials are properly configured in the **Administration ‣ Configuration ‣ Spectra Intelligence** dialog. YARA rulesets that are saved to Spectra Intelligence can match up to 10 000 samples. When a ruleset reaches that many matches, it will be capped and new matches will no longer be stored on the Spectra Analyze appliance. To free up space and continue collecting new matches for the ruleset, users have to remove at least 1000 old matches. Find more information in the [Hunting for Malware in the Spectra Intelligence cloud with YARA](#hunting-for-malware-in-the-spectra-intelligence-cloud-with-yara) section. 3. **Retroactive Cloud Matching** When a new or existing YARA ruleset is selected to run continuously in the Spectra Intelligence cloud, and the YARA Retroactive Hunting feature is enabled on the appliance, the ruleset can be selectively matched against files in the cloud that have been analyzed in the past 90 days. The ruleset will continuously look for new matches among local files that are added to the Spectra Analyze appliance while a Cloud Retro scan is in progress. Find more information in the [Cloud Retro Hunting](./yara-retro.md#cloud-retro-hunting) section. 4. **Retroactive Local Matching** When the YARA Retroactive Hunting feature is started, all rulesets that are active (enabled) on the appliance will be included in a Local Retro scan. This also includes the default Spectra Core rulesets. While a Local Retro scan is in progress, the appliance still continues to analyze new files that are added to the appliance. The processing priority is given to newly added files to prevent delays. The Local Retro scan runs against all local files regardless of their age and analysis date. If users want to prevent the Local Retro scan for one or more rulesets, those rulesets have to be disabled on the appliance. Local Retro matches for rulesets cannot be removed from the list of matches on the YARA page. Find more information in the [Local Retro Hunting](./yara-retro.md#local-retro-hunting) section. ## Understanding the YARA Page Clicking the *Yara* item in the top-level menu brings the user to the YARA Hunting page, which displays a list of rulesets (collections of YARA rules) and matching statistics. ![Overview of the YARA page](images/analyze-yara.png) ### Filters The search bar allows users to filter the list of rulesets by typing the full or partial name of a ruleset. Enclose the search string in quotation marks for exact name matching. Next to the search bar, several menus allow filtering according to ruleset ownership, status and source (for imported rulesets). Some notes on the possible choices: - Under the ownership menu, "User Rulesets" will show all rulesets from all users except the current user ("My Rulesets") - Under the status menu, you can find rulesets with warnings (for example, with a syntax error), as well as rulesets that were [capped in Spectra Intelligence](#managing-yara-rulesets-with-warnings) **Tip: Filters are reflected in the URL (as query parameters), which means that you can share a filtered page. For example:** ``` ?name=yippee-ki-yay&owner=owner_user_jmcclane&status=enabled ``` ...will present a page where the ruleset names contain "yippee-ki-yay", where their owner is user `jmcclane` (note the "owner_user_" prefix), and where such rulesets are enabled. ### Actions The indicators and buttons in the top right of the page display the actions and status information related to [YARA Retroactive Hunting](./yara-retro.md). - The **Add Ruleset** button provides the options to [create, import or upload rulesets](#creating-new-yara-rulesets). - The **Actions** button allows starting a [Local Retro Scan](./yara-retro.md#local-retro-hunting) on all active rulesets on the appliance, and contains links to the list of previous retro hunts and YARA repository management. The retro hunt list is also accessible via the *Open Retro Hunt List* button. - The **gear icon** menu allows the users to customize the YARA table columns and date format. - The synchronization status indicator is visible only if Spectra Intelligence is correctly configured on the appliance, and if at least one YARA ruleset on the appliance is saved in the Spectra Intelligence cloud. Hovering over the indicator with the mouse activates a “Skip To” link that can be used to postpone ruleset synchronization. ### Results If a ruleset is greyed out on the page, it is disabled. Disabled rulesets are not active on the appliance and are not included in any of the Local, Cloud, or Retro scans. Clicking the star icon next to the ruleset adds it to favorites for the current user. Favorite rulesets are listed in the YARA Matches section on the [Dashboard](./dashboard.md). Up to 10 rulesets can be added to favorites. Every ruleset has one or more status indicators. **Hovering over the indicator icon displays a tooltip with more information about every status.** The far right action menu (☰) for each ruleset contains options for testing, editing, exporting, disabling, and deleting the ruleset, and options to subscribe (create [Alerts](./alerts.md) for new YARA matches) or unsubscribe from a ruleset. **Note: Spectra Core rulesets can only be edited and exported, but not disabled or deleted. Selecting the **Edit** option for a Spectra Core ruleset opens the ruleset editor where the user can preview and copy the contents of the ruleset, but cannot save any changes to it.** ### Creating New YARA Rulesets YARA rulesets on the Spectra Analyze appliance can be created from scratch, uploaded as files, or imported from a number of popular online sources. There is also a [YARA API](./API%20Documentation/yara-api.md). ReversingLabs products, including Spectra Analyze, support the following YARA modules: - PE - ELF - Dex - Macho - String - Math - Hash - Time - Dotnet **Note: “Import” and “include” statements are not supported.** To upload or import rulesets from external files and sources, click the **Add Ruleset** button at the top right of the YARA page. The dropdown menu offers three options: **Create Ruleset**, **Upload File** and **Import from Online Sources**. Importing from online sources provides a list of predefined and custom (if configured) sources, and contains an additional step of allowing the selection of which rulesets or individual rules to import. Invalid rules will, depending on the parser, be either imported as disabled or not imported at all. ![The ruleset import selection screen, allowing granular import of rulesets and individual rules.](images/analyze-yara-ruleset-import-online.png) Once the process finishes, imported rulesets can be edited and synced to other appliances if YARA sync is enabled. To write a new ruleset, select the **Add Ruleset > Create Ruleset** option in the top right of the page. This opens the YARA Ruleset Editor **Tip: For more information on writing YARA rulesets, consult one of the following sources:** - ReversingLabs publishes guidance for using YARA on the official blog. See the blog posts [“Level up your YARA game”](https://blog.reversinglabs.com/blog/level-up-your-yara-game) , [“Writing detailed YARA rules for malware detection”](https://www.reversinglabs.com/blog/writing-detailed-yara-rules-for-malware-detection) and [“Five Uses of YARA”](https://blog.reversinglabs.com/blog/five-uses-of-yara) to learn more. - The official YARA documentation offers detailed advice on [how to write YARA rules](https://yara.readthedocs.io/en/stable/writingrules.html). - Use Spectra Core rulesets present on the appliance as reference. #### YARA Ruleset Restrictions - Naming restrictions: - YARA ruleset names must be between 3 and 48 characters. - The underscore ( _ ) should be used instead of spaces, and any other special characters should be avoided. Ruleset names should only use numbers (0-9) and a-z/A-Z letters. - Ruleset size restrictions: - A ruleset file should not be larger than 40 MB. - A ruleset larger than 1 MB (1048576 bytes) cannot be saved and run in the Spectra Intelligence cloud. When creating a new ruleset in the editor, you can select whether the ruleset will be immediately enabled after saving changes by selecting the **Enable Ruleset** checkbox. If the checkbox is not selected, the ruleset will be disabled after it is created, and files will not be matched against it until the you enable it. To save a newly created ruleset in the Spectra Intelligence cloud, select the *Run ruleset continuously in Spectra Intelligence* checkbox in the ruleset editor. The appliance administrator can enable automatic upload of YARA rulesets to Spectra Intelligence in the **Administration > Configuration > YARA Cloud Settings** configuration dialog. If this option is enabled on the appliance, the *Run ruleset continuously in Spectra Intelligence* option will be automatically selected by default in the YARA ruleset editor. When done making changes to the new ruleset in the editor, click the *Save & Close* button. If the new ruleset is validated, it appears in the list of rulesets on the YARA page. Clicking *Save & Publish* will save the ruleset and add it to the synchronization queue. This button will be visible only to users with the appropriate [user role](/SpectraAnalyze/Administration/users-personalization/user-roles/). If one or more rulesets are saved in Spectra Intelligence, an indicator of synchronization status is visible in the upper right part of the YARA page. ### Editing Rulesets and Restoring Previous Ruleset Versions All Spectra Analyze users with the appropriate user roles can edit or delete all YARA rulesets on the appliance, except Spectra Core rulesets. Clicking the *Edit* item in the ruleset action menu (☰) on the YARA page opens the ruleset editor. The editor allows modifying an existing YARA ruleset, saving it to Spectra Intelligence, and managing different versions of the same ruleset. Users can also manually enable or disable a ruleset by selecting the *Enable ruleset* checkbox. Every time a ruleset is edited and saved, a new version of the ruleset is created. The versions are visible both in the Version filter on the ruleset's page and in the YARA ruleset editor. The statistics about matches of every ruleset version are preserved and displayed, allowing users to compare their efficacy. In the YARA ruleset editor, the user can restore any previous version of a ruleset to overwrite new changes. Click the *History* menu to open the pull-down list with ruleset versions. Click the **Load** link next to the ruleset version that should be restored, then click the *Save* button to apply changes and activate the selected version. Clicking **Save & Close** returns to the YARA page after saving changes, while clicking *Save* allows the user to remain in the ruleset editor and keep making changes. It is not possible to save an empty ruleset (a ruleset without any text in the *Rules* field), or a ruleset without a name. The **Save & Publish** button saves the ruleset and add it to the synchronization queue. This button is visible only to users with the appropriate [user role](/SpectraAnalyze/Administration/users-personalization/user-roles/). Optionally, [test the ruleset](#yara-test-run-page-testing-rulesets) before saving and/or publishing to check if it’s correctly matching the desired samples. ![YARA ruleset editor with highlighted History link](images/analyze-yara-rule-history-editor.png) The ruleset editor also displays information about the user who added the ruleset, and the user who last edited it. Editing a ruleset does not change its ownership. For example, if user *test1* creates a ruleset and another user *test2* saves changes to the same ruleset, user *test1* is still the owner of the ruleset. In this scenario, only the user *test1* can delete the ruleset. When a user account is deleted from the Spectra Analyze appliance, all YARA rulesets owned by that deleted user are automatically transferred to the administrator account that deleted the user. Validation indicators visible on the main YARA page are also displayed in the ruleset editor. Depending on the modifications made to the ruleset, the validation indicators can change (for example, when a ruleset is saved to Spectra Intelligence). ### YARA Test Run Page: Testing Rulesets When writing a new ruleset in the editor, you can test run a ruleset against a predefined set of samples to check if the ruleset matches everything it should. The YARA Test Run page can be accessed via the ruleset editor, or using the **Test Ruleset** option in the (☰) menu of a particular ruleset in the table. Samples can be added to the testing page either directly by hash, or using tags. You can select up to 5 tags, which will load 100 latest submitted samples from the selected tags. If unsure about a specific tag name, the tag menu also serves as a search box. Once the page is populated with samples, you can tweak the ruleset by clicking the **Edit Ruleset** button, or perform a YARA test run by clicking the **Run Test** button. After the test is complete, the samples list shows the test results in the far right column of the table. If you tweak the ruleset again and **Retest** it, new test results will show up alongside the old ones in the rightmost column. This way, you can compare the results of up to four previous runs. Test result icons, when clicked, open up a hex preview with the highlighted portion of the binary that was matched in the test run, as well as its ASCII representation. **Run New Test** clears the previous results. ### Enabling and Disabling Rulesets **Note: YARA rulesets cannot be enabled or disabled while their Cloud or Retro validation is pending, or while a Retro scan on the rulesets is in progress.** For individual rulesets: on the right side of the page, click the triple bar button (☰) to open the ruleset action menu. In the menu, click *Enable* or *Disable*. Alternatively, click the ruleset to open its page showing a list of matches, and disable the "Enable Ruleset" switch in the top right of the page. For multiple rulesets at once: select the checkboxes to the left of each ruleset name. When the rulesets are selected, open the main action menu by clicking the triple bar button (☰) in the toolbar, on the right side of the YARA page. In the menu, select the *Enable* or *Disable* option. The YARA page dynamically displays rulesets as enabled or disabled, one by one. ### Managing YARA Ruleset Matches Every ruleset on the YARA page can be clicked to show a list of files that matched one or more YARA rules in the selected ruleset. When a ruleset is opened, various statistics are displayed below the ruleset name and filters: Total Matched Samples, Most Matched Rule, Most Matched Format, Most Matched Threat Type. The top left of the page contains the ruleset name with a tooltip displaying the ruleset's creation and last updated date, commit ID and source repository information (where the ruleset was imported from). The top right of the page contains options to **Edit Ruleset**, and an **Actions** menu allowing the users to test, edit, export, disable or delete the ruleset, and to subscribe/unsubscribe. In addition to these buttons, there are switches to enable/disable the ruleset, and to run the ruleset in Spectra Intelligence cloud. To the right of the statistics boxes is an arrow button that expands each of the statistics into a graphical display. Colored sections on the graph can be clicked to actively filter out the results table below. The table can also be filtered using the dropdown menus at the top of the ruleset’s page: by ruleset version, by a specific rule, classification, format, threat type and the first seen date. Several different filters can be active at the same time. The filters can be cleared to their default values by clicking the **Clear** button. Ruleset matches can be filtered to display only those files that matched a particular rule. To do that, select the rule in the pull-down list under the ruleset name. ![YARA matches filtered by classification: Malicious.](images/analyze-yara-matches-filter-by-classification.png) The information about ruleset matches is divided into columns. All columns can be sorted in ascending or descending order by clicking the column name. From left to right, the columns are: - source indicator (local or cloud; if the file is not available for download from Spectra Intelligence, a light gray cloud icon is displayed), - status indicator (as explained in the [About Spectra Analyze](./index.md#color-coding-and-sample-status) section), - time when the file matched the selected ruleset, - time when the sample was first seen, - time when the sample was last seen, - threat name (if detected; applies to malicious and suspicious samples only), - file name (or SHA1 hash), - name of the YARA rule that triggered the match, - file format (for cloud matches, displays “Unknown” if file format information for a sample is not stored in the Spectra Intelligence cloud), - the number of extracted files, - the file size. The right side of the page contains two buttons with more options to filter the list of ruleset matches: On the right side of every match in the list, there is a triple bar button (☰) that activates the actions menu with different options. For local matches, the options include: - *Delete sample* - removes the submission. This action is visible to users only for files and URLs they have submitted themselves. Regular users cannot delete files and URLs submitted by other users. Administrators can see this menu item for all submissions, and delete all submissions regardless of who created them. - *Copy to my submissions* - available only if the sample was submitted by a different user. Selecting this adds the sample to the submissions for the current user. - *Download sample* – downloads the submission from Spectra Analyze to local storage (compatible with modern ZIP tools such as `unzip` 6.00+, Python's `zipfile`, 7-Zip, or `bsdtar`). - *Download container* - if the file is extracted, downloads its top-most parent from the appliance to the local storage - *Set classification* - allows users to manually override file classification - *Subscribe* and *Unsubscribe* - adds or removes the file from alert subscriptions - *Reanalyze* - allows users to reanalyze the file with static, dynamic, or Spectra Intelligence analysis services - *Download Extracted Files* - downloads files extracted during analysis to local storage For cloud and cloud retroactive hunting matches, the options include: - *Fetch & Analyze* - downloads the file to the appliance - *Reanalyze on Spectra Intelligence* - reanalyzes individual files using the Spectra Intelligence analysis services - *Subscribe* and *Unsubscribe* - adds or removes the file from alert subscriptions Clicking a row in the list of matches expands it to show additional information about the file, as described in the [Expanded Details](./search-page.md#expanded-details) section. Clicking the file name opens the local or the Spectra Intelligence version of the [Sample Details](./Sample%20Details/index.md) page for the selected file, depending on if the sample is locally available. #### Performing Bulk Actions on Ruleset Matches Multiple files can be selected in the list of ruleset matches, and supported bulk actions can be performed on them. Select the checkboxes on the left side of each individual file, or click the *Select all N YARA matches* link that appears above the list of matches when at least one sample is selected. It is also possible to select all matches by clicking the *Select All* checkbox on the left side of the *Match Time* column. When one or more files are selected, the bulk action menu (☰) appears on the right side of the list (next to the *Size* column). **For local matches, the supported bulk actions include:** - *Download samples* – downloads the submission from Spectra Analyze to local storage (compatible with modern ZIP tools such as `unzip` 6.00+, Python's `zipfile`, 7-Zip, or `bsdtar`). - *Reanalyze* - allows users to reanalyze the files with static, dynamic, or Spectra Intelligence analysis services - *Subscribe* and *Unsubscribe* - adds or removes the files from alert subscriptions - *Apply tags* - allows users to add custom tags to selected files - *Export data* - exports information about selected files as a CSV file - *Download Extracted Files* - downloads files extracted during analysis to local storage **For cloud and retro matches, the supported bulk actions include:** - *Fetch & Analyze* - downloads and analyzes the files to the Spectra Analyze appliance - *Remove* - removes selected files from the list of matches for that particular ruleset - *Subscribe* and *Unsubscribe* - adds or removes the files from alert subscriptions - *Download samples* – downloads the submission from Spectra Analyze to local storage (compatible with modern ZIP tools such as `unzip` 6.00+, Python's `zipfile`, 7-Zip, or `bsdtar`). - *Export data* - exports information about selected files as a CSV file - *Download Extracted Files* - downloads files extracted during analysis to local storage **The exported CSV file with YARA matches contains the following information:** - threat status, - risk score - threat name (if detected), - time when the file matched the selected ruleset, - file name, - YARA rule that triggered the match; - file format; - the number of extracted files, - the file size. ### Managing YARA Rulesets with Warnings Some YARA rulesets can slow down file processing on the Spectra Analyze appliance due to their length, scope, and other factors. When uploading, editing, enabling and disabling such rulesets, the Spectra Analyze appliance displays a notification. The ruleset(s) are marked with a warning icon on the YARA Hunting page. ![Section of the YARA page with highlighted ruleset warning message](images/analyze-yara-ruleset-warning.png) It is possible to filter the YARA Hunting page to display only rulesets with warnings by choosing the *Rulesets with warnings* item from the pull-down menu on the right. Those rulesets can then be disabled or edited to resolve the issue. It takes some time for the system to update the ruleset status, so the warning icon does not disappear immediately after modifying a problematic ruleset. Appliance administrators can access the list of rulesets with warnings from the *Scale* section of the [System Status](/SpectraAnalyze/Administration/usage-alerts/system-status/) page. If there are any rulesets with warnings, the *YaraWarnings* icon will not be green, and will show the number of warnings instead. ![Section of the System Status page with YARA ruleset warning indicator](images/analyze-yara-ruleset-warning-status.png) Clicking the icon redirects to the YARA Hunting page filtered to display only rulesets with warnings. Here the offending rulesets can be edited or disabled to improve the processing speed of the appliance. ## Hunting for Malware in the Spectra Intelligence cloud with YARA By default, rulesets only run on files processed on the Spectra Analyze appliance (through Continuous Local Matching). However, rulesets can be synchronized with Spectra Intelligence and run on a massive amount of files processed daily by ReversingLabs systems, greatly expanding the potential for YARA matches. If [YARA Retroactive Hunting](./yara-retro.md#cloud-retro-hunting) is enabled on the appliance, rulesets synchronized with Spectra Intelligence can also be run on files analyzed in the Spectra Intelligence cloud over the last 90 days. To synchronize a YARA ruleset with Spectra Intelligence: - Click the ruleset's name on the YARA page to open the page displaying a list of its matches. Enable the **Run Ruleset in Cloud** switch in the top-right corner of the page. - Open the ruleset editor by clicking the *Edit* item in the ruleset action menu (☰). In the editor, select the *Run ruleset continuously in Spectra Intelligence* checkbox and click the *Save* or *Save & Close* button. - Alternatively, the appliance administrator can enable automatic upload of YARA rulesets to Spectra Intelligence in the **Administration > Configuration > YARA Cloud Settings** configuration dialog. If this option is enabled on the appliance, the *Run ruleset continuously in Spectra Intelligence* option is automatically selected by default in the YARA ruleset editor. ![Section of the YARA ruleset editor with the options for saving the ruleset to Spectra Intelligence](images/analyze-yara-edit-ruleset.png) When synchronized with Spectra Intelligence, the ruleset displays a cloud status indicator on the YARA page. Additionally, an indicator of synchronization status is visible in the upper right part of the YARA page. The indicator displays the date and time of last synchronization, and allows skipping the synchronization until the next day. The number of matches stored for a ruleset synchronized with Spectra Intelligence is currently capped at 10 000 to limit the impact on Spectra Analyze. If a ruleset has hit this limit, its CLOUD status indicator on the YARA page gets a red X badge icon. New matches are no longer preserved for a ruleset that hits this limit. If the appliance administrator has enabled the *Automatic disabling of Cloud enabled YARA rulesets* option in the **Administration > Configuration > YARA Cloud Settings** configuration dialog, every ruleset that reaches the limit are automatically de-synchronized from Spectra Intelligence. To continue receiving new cloud matches for a ruleset that has reached the limit, users have to remove previous retro results from the ruleset’s action menu (☰) by clicking the Clear Retro Hunt option. Alternatively, open the ruleset’s page, configure the filters so no matches of importance are removed, select all samples in the table and pick the Remove Cloud Matches option from the upmost (☰) menu. To clear matches even more selectively, configure the filters, sort the table by source (click the first table column title so that the cloud matches are listed first) and select individual samples or entire pages of results and use the Remove Cloud Matches option on the selected samples. ## Synchronizing YARA Rulesets with Other Appliances **Note: Only appliance administrators can access the option to synchronize YARA rulesets.** If the Spectra Analyze appliance is connected to the ReversingLabs [Spectra Detect](/SpectraDetect/) Manager, it is possible to synchronize YARA rulesets to other appliances (Spectra Analyze, Spectra Detect Worker) connected to the same Spectra Detect Manager. The synchronization needs to be configured on Spectra Detect Manager first, and then enabled in the *Spectra Detect Manager* section of the Spectra Analyze [System Configuration](/SpectraAnalyze/Administration/configuration-update/configuration/) page. ![System Configuration page with highlighted option for YARA ruleset synchronization](images/analyze-yara-sync.png) Select the *Enable Syncing* checkbox to enable YARA rule synchronization to other appliances from the current Spectra Analyze instance. When synchronization is enabled, every new ruleset created on the appliance automatically gets sent to the Spectra Detect Manager, and from there to other appliances that support YARA rule synchronization. The synchronization is bidirectional, so any changes made to YARA rules on other appliances connected to Spectra Detect Manager also reach the current Spectra Analyze instance. New rulesets automatically appear on the Spectra Analyze YARA page. Rulesets retrieved from other appliances via Spectra Detect Manager synchronization are not visually distinguished from other rulesets on the YARA page. Ruleset synchronization includes deleting and disabling rulesets. Deleting a ruleset on one appliance also deletes it on the others, if they are synchronized via Spectra Detect Manager. When creating, deleting or editing rulesets, only users with the **YARA Publish and Delete** permission have the additional options to **Save & Publish / Delete & Publish** that add the ruleset action to the synchronization queue. Other users with the appropriate, **YARA**, permission are only able to manage rulesets locally. ### Important notes 1. Spectra Core rulesets are not synchronized. 2. If two users on different appliances make changes to the same ruleset at the same time, only the changes from the appliance that syncs with Spectra Detect Manager first will be applied. Synchronization with Spectra Detect Manager happens approximately every ten minutes. ### YARA Status If YARA syncing is enabled, the YARA page shows a YARA Status indicator to show the current synchronization status, and to indicate if any rulesets are out of sync. In case some rulesets are not synchronized, which is indicated by a red X badge, users can click the **Show Status** link to see which rulesets are not synchronized and **Republish** them. --- ## SpectraDetect Management API — SDM Authorization, Backup, and Appliance Setup **Tip: This is a companion guide for the Manager's API endpoints. For a full reference, including the OpenAPI specification, see the Manager API reference (available in the help menu on the Manager appliance).** Here are the steps covered in this guide: 1. [Obtain an authorization token](#api-token) 2. [Back up and restore the Manager](#back-up-and-restore-the-manager) 3. [Set up the Manager](#set-up-the-manager) 4. [Connect appliances to the Manager](#connect-appliances-to-the-manager) 5. [Configure appliances](#configure-appliances) 6. [Configure connectors](#configure-connectors) 7. [Configure central logging](#configure-central-logging) **Tip: For setups using a self-signed certificate, add `-k` or `--insecure` before the appliance URL in the curl examples.** ## Obtain an authorization token {#api-token} To use the Manager API, an authorization token needs to be included in all API calls. You can obtain this token using the authentication API or the web interface. The authorization token is a string of alphanumeric characters, between 30 and 60 characters in length. The authorization token should be provided in the HTTP `Authorization` header. The value of this field must be in this format: ``` Token exampletoken ``` where `exampletoken` should be replaced with the user's own authorization token string. Call the authentication API with your username and password using the `multipart/form-data` content type. ``` POST /api/v1/auth/token/ ``` Form parameters: - `username` - `password` Request example: ```bash curl -X POST 'https://c1000.example.com/api/v1/auth/token/' --form "username=user123" --form "password=pass123" ``` Response example: ```json { "token": "" } ``` This token has to be included in all further requests to the API in the *Authorization* header. For example: ```bash curl -H "Authorization: Token " https://c1000.example.com/api/configuration/appliances ``` ## Back up and restore the Manager In certain cases, you may want to perform a complete backup of the Manager before performing any other actions. Backup files contain [central configuration settings](../Config/ApplianceConfiguration.md), user accounts on the Manager instance, a database of connected appliances and all of their configuration files, appliance groups, and synchronized YARA rules. Uploading the backup archive to a clean machine restores the data stored in the Manager database, but configuration settings first need to be applied to configuration groups to take effect. The steps are: 1. [Create a backup file](#create-a-backup-file) 2. [Find out the backup status](#find-out-the-backup-status) 3. [Download backup file](#download-backup-file) 4. [Upload and restore the Manager from backup](#upload-and-restore-the-manager-from-backup) ### Create a backup file ``` POST /api/v1/backup/create/ ``` Creates a backup file that can be downloaded with the [`GET /api/v1/backup/download/`](#download-backup-file) endpoint. ### Find out the backup status ``` GET /api/v1/backup/ ``` Returns the current backup status and availability. Response: ```json5 { "status": "error", "message": "idle", // when the message is 'idle', the backup is ready "backup_filename": "example" } ``` ### Download backup file ``` GET /api/v1/backup/download/ ``` Downloads the backup file. ### Upload and restore the Manager from backup ``` POST /api/v1/backup/restore-upload/ ``` Uploads the backup file to the Manager. Form parameters: - `file`: the backup file ## Set up the Manager Perform the following actions to set up the Manager. This guide is a reference for an enterprise-ready deployment of Spectra Detect, with redundancy and password rotation requirements, for example. Skip the steps that are not relevant to your desired setup. If you have additional requirements that are not covered by this guide, you can always find the complete API reference on the Manager itself (Help menu). The steps for an initial setup are: 1. Set up authentication: 1. (Get authorization token - already covered in [the previous section](#api-token)) 2. [Change admin password](#change-admin-password) 2. [Configure network settings](#configure-network-settings): 1. Upload SSL certificate 2. Configure the application URL 3. List the allowed hosts 4. Set network time sync 3. [Configure user authentication](#configure-user-authentication) 1. Federated login 2. SSH access 4. [Configure Spectra Intelligence](#configure-spectra-intelligence) 5. [Update the Manager](#update-the-manager) 6. [License the Manager](#license-the-manager) 7. [Set up logging and alerting](#set-up-logging-and-alerting) 1. SNMP 2. SMTP 3. System alerting 4. Central logging 8. [Enable synchronization and redundancy](#enable-synchronization-and-redundancy) 9. [Set a timezone](#set-a-timezone) ### Change admin password First, you need to get the ID of the admin user. It's almost certainly `1`, but you can get all users and their IDs by calling this endpoint: ``` GET /api/v1/authentication/users/ ``` Returns all users on the appliance. Response example: ```json { "count": 1, "next": null, "previous": null, "results": [ { "id": 1, "username": "admin", "email": "hello@example.com", "first_name": "", "last_name": "", "directory_name": "", "directory_type": 0, "last_login": "2024-02-07T12:10:04.744401Z", "date_joined": "2024-01-03T03:01:30.166022Z", "is_superuser": true, "is_active": true } ] } ``` Then change the administrator password: ``` PUT /api/v1/authentication/users/{id}/password/ ``` Path parameters: - `id`: user ID Request body example: ```json5 { "password": "Correct Horse Battery Staple" } ``` ### Add or remove users #### Add a user ``` POST /api/v1/authentication/users/add/ ``` ##### Request body ```json5 { "username": "string", "email": "user@example.com", "first_name": "string", "last_name": "string", "password": "string" } ``` ##### Response example ```json { "id": 0, "username": "string", "email": "user@example.com", "first_name": "string", "last_name": "string", "directory_name": "string", "directory_type": 0, "last_login": "2019-08-24T14:15:22Z", "date_joined": "2019-08-24T14:15:22Z", "is_superuser": true, "is_active": true } ``` #### Remove a user ``` DELETE /api/v1/authentication/users/{id}/delete/ ``` ##### Path parameters - `id`: user ID #### Examples Example adding and deleting a user: ```bash curl --url https://{server}/api/v1/authentication/users/add/ --header 'Authorization: Token ' --json '{"username":"hello","email":"example@example.com","password":"example"}' curl --url https://{server}/api/v1/authentication/users/3/delete --header 'Authorization: Token ' --request DELETE ``` ### Configure network settings The Manager can operate without a valid SSL certificate, but to maximize security in communication between users and the appliance, you may want to first upload a valid SSL certificate to the Manager. #### Upload a certificate To upload the certificate to the Manager, use the Upload Certificate endpoint. ``` POST /api/v1/system/upload-certificate/ ``` ##### Form parameters - `file`: combined PEM file containing both private key and certificate (in standard PEM format) The file should contain both the private key and certificate in a single PEM file. You can create this combined file using: ```bash cat private.key certificate.pem > combined.pem ``` ##### Response codes - `200`: certificate uploaded successfully - `400`: validation error - `500`: server error `curl` code sample: ```bash # Add --insecure before the URL if you are using a self-signed SSL certificate curl -X POST https://{server}/api/v1/system/upload-certificate/ -H "Authorization: Token " -F "file=@combined.pem" ``` While the SSL certificate upload is optional, you *must* set up the base URL and allowed hosts. This request also allows completing your SSL setup. #### Configure the network ``` POST /api/v1/config/network/ ``` ##### Request body ```json5 { "base_url": "string", "allowed_hosts": "string", // allowlisted IP addresses or URLs, separated by a newline (`\n`) character "certificate_path_displayname": "string", "certificate_key_path_displayname": "string", "reset_ssl": true } ``` ###### Request body example ```json { "base_url": "c1000.example.com", "allowed_hosts": "worker1.example.com\nworker2.example.com", "certificate_path_displayname": "/my/default/path", "certificate_key_path_displayname": "/my/default/path", "reset_ssl": true } ``` #### Enable network time synchronization ``` POST /api/v1/config/ntp/ ``` ##### Request body ```json5 { "enable": true, "servers": "string", // must be separated by a newline (`\n`) character "timezone": "string" } ``` ### Configure user authentication After configuring the minimum necessary settings for proper networking of the Manager, continue with provisioning user access. In enterprise deployments, a federated (single sign-on) login is often used. This guide references only some of the available fields - for a complete overview, check the API reference on the Manager itself. ``` POST /api/v1/config/directory/ ``` Sends LDAP/OIDC configuration to the Manager. After sending the configuration with `POST /api/v1/config/directory/`, you can verify that the configuration is updated using `GET /api/v1/config/directory/`. ### Enable SSH root login **Warning: Dangerous! Some services like password rotation solutions require root SSH access to the devices they are managing. This opens up the attack surface considerably.** ``` POST /api/v1/system/config/sshd/ ``` Sends the SSH configuration to the appliance. ##### Request body ```json5 { "banner_content": "string", "macs": "string", "ciphers": "string", "kexalgorithms": "string", "authorized_keys_file": "string", "client_alive_interval": "string", "client_alive_count_max": "string", "permit_root_login": "yes" } ``` ### Configure Spectra Intelligence Next, configure the connection between Spectra Intelligence and the Manager. This is a prerequisite for licensing the Manager using Spectra Intelligence. In addition, connecting it to Spectra Intelligence allows it to download upgrade binaries. ``` POST /api/v1/config/ticloud/ ``` Sends the configuration for Spectra Intelligence to the Manager. ##### Request body example ```json5 { "enable": true, "url": "https://appliance-api.reversinglabs.com", "user": "u/myusername", "token": "exampleABC123", // your password "timeout": 10 // in seconds } ``` To verify the changes, send a GET request to the same endpoint: ``` GET /api/v1/config/ticloud/ ``` ### Update the Manager Get the list of available upgrades: ``` GET /api/v1/update-system/available-update/ ``` #### Response example ```json5 { "filename": "string", "sha1": "string", "version": "string", // version introduced by the update "requires": "string", // required version to use this update file "time": "2019-08-24T14:15:22Z" // build time } ``` Then update the Manager to the latest available version: ``` POST /api/v1/update/latest/ ``` #### Examples ```bash curl -X GET https://{server}/api/v1/update-system/available-update/ -H 'Authorization: Token ' curl -X POST https://{server}/api/v1/update/latest/ -H 'Authorization: Token ' ``` ### License the Manager The Manager can be licensed in two ways: 1. [Using Spectra Intelligence](#using-spectra-intelligence) 2. [Using a license file](#using-a-license-file) #### Using Spectra Intelligence To configure Spectra Intelligence on the Manager, use the following endpoint. ``` POST /api/v1/license/configure/cloud/ ``` Performs a licensing request using Spectra Intelligence. ##### Request body ```json5 { "url": "https://appliance-api.reversinglabs.com", "user": "string", "token": "string", "timeout": 1, "proxy_enabled": true, "proxy_host": "string", "proxy_port": 0, "proxy_user": "string", "proxy_password": "string" } ``` ##### Example ```bash curl -X POST 'https://c1000.example.com/api/v1/license/configure/cloud' --header 'Content-Type: application/json' --header 'Authorization: Token exampletoken' --data '{ "url": "https://appliance-api.reversinglabs.com", "user": "string", "token": "string", "timeout": 1, "proxy_enabled": true, "proxy_host": "string", "proxy_port": 0, "proxy_user": "string", "proxy_password": "string" }' ``` Conversely, perform a GET request to retrieve the configuration: ``` GET /api/v1/license/configure/cloud/ ``` #### Using a license file The alternative to Spectra Intelligence is manually uploading a license file. To do so, first create a unique machine ID: ``` POST /api/v1/license/machine-id/generate/ ``` ##### Example ```bash curl -X POST https://c1000.example.com/api/v1/license/machine-id/generate/ -H 'Authorization: Token exampletoken' ``` When the Manager responds with the machine ID, email it to [ReversingLabs Support](mailto:support@reversinglabs.com). We will respond with a license file that can be uploaded to the Upload License File endpoint: ``` PUT /api/v1/license/upload/ ``` ##### Form parameters - `file`: license file ##### Example ```bash curl -X PUT https://example.c1000.com/api/v1/license/upload -H 'Authorization: Token exampletoken' -F 'file=@example_license_file' ``` ### Set up logging and alerting This chapter covers methods to monitor performance of the Manager itself. All the endpoints listed below also accept the GET method, which returns the current configuration. #### SNMP Send SNMP configuration to the Manager. The Manager uses SNMP v2c. ``` POST /api/v1/config/snmp/ ``` ##### Request body ```json5 { "enable": true, "community": "string", "trap_sink_enable": true, // enable sending traps "trap_community": "string", // community string for traps "trap_sink": "string", // trap receiver address // 👇 The Manager sends a trap when this value is exceeded // (percentage of used CPU threads over 1, 5 and 15 minutes) "trap_avg_load_threshold_1min": 1, "trap_avg_load_threshold_5min": 1, "trap_avg_load_threshold_15min": 1, "trap_memory_threshold": 1, // percentage of used memory "trap_disk_threshold": 1 // percentage of used disk } ``` #### SMTP ``` POST /api/v1/config/smtp/ ``` Sends SMTP configuration to the Manager. ##### Request body ```json5 { "host": "string", // SMTP relay server "port": 65535, "user": "", "password": "", "password_set": false, "use_tls": false, "default_from_email": "user@example.com" } ``` #### System alerting Send configuration for syslog usage to the Manager. ``` POST /api/v1/config/systemalerting/ ``` ##### Request body ```json5 { "syslog_enable": true, "syslog_host": "string", "syslog_port": 65535, "syslog_transport_protocol": "TCP", "syslog_enable_audit_logs": true } ``` #### Central logging Configure central logging on the Manager. ``` POST /api/v1/config/central_logging/ ``` ##### Request body ```json5 { "central_logging__enable": true, "central_logging__retention_period": 1, // in days "central_file_storage__enable": true, "central_file_storage__ttl": 1, // how long to keep files on the Manager (in hours) "central_file_storage__file_size_limit": 1, // in MiB "central_storage_general__min_free_space": 10, // in GiB } ``` ### Enable synchronization and redundancy One last step before completing the setup of the Manager is to enable the synchronization of YARA rulesets, as well as setting up a redundant Manager that takes over in case of a failure of the primary. #### YARA sync ``` POST /api/v1/config/sync/ ``` Enables or disables the synchronization of YARA rulesets across connected appliances. ##### Request body ```json5 { "yararulesets_enable": true } ``` #### Set up a redundant cluster **Note: The prerequisite for this is to have another Manager already operational and accessible on the network.** The first step is to create a cluster and set up the basic configuration: ``` POST /api/v1/cluster/create/ ``` ##### Request body ```json5 { "local_ip": "192.168.0.1", // IP address of the primary Manager "remote_ip": "192.168.0.1", // IP address of the secondary Manager "vpn_network": "string", // virtual private network range in CIDR notation; subnet mask can only be `/29` or lower "virtual_ip": "192.168.0.1", // shared IP address to access the Manager when in redundant mode "failover_timeout": 3 // seconds between the unavailability of the primary and activation of the redundant Manager } ``` After this, you must perform a mutual authentication between the two appliances: ``` POST /api/v1/cluster/exchange_keys/ ``` ##### Request body ```json5 { "values": { "local_ip": "string", // primary Manager IP address "remote_ip": "string", // secondary Manager IP address "remote_url": "string", // secondary Manager URL "user": "string", // admin "pwd": "string" // admin password } } ``` Finally, verify that redundancy works: ``` POST /api/v1/cluster/verify/ ``` Performs checks between appliances. ##### Request body ```json5 { "remote_ip": "192.168.0.1", // IP address of the secondary Manager "section": "ssh" // aspect of redundancy to check (can be `ssh` or `versions`) } ``` ##### Response example ```json5 { "return_code": 0, // HTTP response status code "message": "string", // request-related message "messages": [ "string" // messages related to redundancy ], "reduser_ssh_connect": true, // is SSH connectivity established? "postgres_ssh_connect": true, // is the redundant database accessible? "rld_ssh_tunnel_connect": true, // can an SSH tunnel be established? "version_local": 0, // version of the primary Manager "version_remote": 0 // version of the secondary Manager } ``` ### Set a timezone This section covers listing and setting the timezone of the Spectra Detect Manager. #### List timezones List available timezones for the Spectra Detect Manager. The response contains a list of timezones that can be set. ``` GET /api/v1/system/config/list-available-timezones/ ``` ##### Response example ```json5 { "timezones": [ "string" ] } ``` #### Set the timezone of Spectra Detect Manager ``` POST /api/v1/system/config/timezone/ ``` ##### Request body ```json5 { "timezone": "string" } ``` ## Connect appliances to the Manager This section covers the following actions: 1. [Add appliances to the Manager](#add-appliances-to-the-manager) 2. [Authorize appliances](#authorize-appliances) 3. [Update appliances](#update-appliances) 4. [Upload SSL certificate to appliances](#upload-ssl-certificate-to-appliances) 5. [Check connected appliances](#check-connected-appliances) ### Add appliances to the Manager **Note: Appliances first must be up and running.** To configure appliances with the Manager, start with adding them: ``` POST /api/v1/appliances/create/ ``` #### Request body ```json5 { "name": "string", "type": "a1000", // possible values: a1000, tiscale-worker, tiscale-hub "url": "http://example.com", "snmp_community": "string", "haproxy_password": "string" // load balancer password (Hub) } ``` The appliance `type` accepts legacy names for the products: - `a1000` is Spectra Analyze - `tiscale-worker` is Spectra Detect Worker - `tiscale-hub` is Spectra Detect Hub The response object format is the same as the request object. ### Authorize appliances After being added to the Manager, appliances must be authorized. The authorization process differs, depending on the appliance. The API endpoint is the same, but two different methods must be used: **GET** for Hubs and Workers, and **PUT** for Spectra Analyze. **Tip: Use `GET /api/v1/configuration/appliances/` to get the IDs of all connected appliances.** #### Hub, Worker ``` GET /api/v1/appliances/{id}/authorization/ ``` ##### Path parameters - `id`: ID of the appliance #### Spectra Analyze ``` PUT /api/v1/appliances/{id}/authorization/ ``` ##### Path parameters - `id`: ID of the appliance ##### Request body ```json5 { "token": "string" } ``` ### Update appliances Get the list of available upgrades: ``` GET /api/v1/package-management/ ``` Returns the list of available update binaries for connected appliances. ##### Response body ```json5 { "bins": [ { "type": "string", "version": "string" } ], "used_space": 0, // how much space is used up on the appliance "allowed_space": 0, // how much space is left on the appliance "is_ticloud_data_available": true // is Spectra Intelligence data on upgrade binaries available } ``` After retrieving the list of available updates, use the appliance type and version from the `bins` object to download one of the available updates: ``` POST /api/v1/package-management/{appliance_type}/{version}/ ``` Downloads an update package from the Spectra Intelligence cloud. Path parameters: - `appliance_type`: what type of appliance? - `a1000` stands for Spectra Analyze - `c1000` stands for Spectra Detect Manager - `tiscale-hub` stands for Spectra Detect Hub - `tiscale-worker` stands for Spectra Detect Worker - `version`: version of the appliance Once the update downloads, the following request will upgrade a single appliance (upgrading multiple appliances at the same time isn’t supported). To upgrade an appliance, find its name using `GET /api/v1/configuration/appliances/` and include it in the request body (see example below). ``` POST /api/upgrade/start/ ``` Starts appliance upgrade. Request body: ```json5 { "appliance_name": "string" } ``` You can check the upgrade status at any point: ``` POST /api/upgrade/status/ ``` Request body: ```json5 { "appliance_name": "string" } ``` Response: The body displays the current status, which can be: - `idle`: there is no update happening - `pending`: an update is planned - `downloading`: an update is currently being downloaded to the appliance - `installing`: an update is installing - `error`: there was an error installing an update The body will also contain the entire upgrade log, if available. If not, it will contain the message "Update log not available." Here's a list of `curl` commands corresponding to the steps above: ```bash curl -X GET https://{server}/api/v1/package-management/ -H 'Authorization: Token ' curl -X POST "https://{server}/api/v1/package-management/tiscale-worker/3.4.0/" -H "Content-Type: application/json" -H 'Authorization: Token ' -d '{}' curl -X POST https://{server}/api/upgrade/start/ -H 'Authorization: Token ' -d '{"appliance_name":""}' curl -X POST https://{server}/api/upgrade/status/ -H 'Authorization: Token ' -d '{"appliance_name":""}' ``` ### Upload SSL certificate to appliances **Note: Hub and Worker only. Uploading SSL certificates to Spectra Analyze through the Manager is not possible.** ``` POST /api/v1/appliances/upload-certificate/{id}/ ``` Uploads an SSL certificate to an appliance. Path parameters: - `id` - ID of the appliance Form parameters: - `file` - certificate file The certificate file must be a combined file that includes both the private key and the certificate. You can generate it with a command such as: ``` cat private.key certificate.pem > combined.pem ``` ### Check connected appliances After adding appliances, verify their connectivity: ``` GET /api/v1/configuration/appliances/ ``` This endpoint returns a list of appliances connected to the Manager. In the response, the structure of the fields in the `results` array is as follows: ```json5 { "id": 0, "appliance_name": "string", "type": "a1000", "release": "string", "cluster": "string", // "standalone" if not in a group "state": "string", // whether primary or secondary "appliance_hostname": "string" } ``` ...and licensing status: ``` GET /api/v1/appliances/licensing/ ``` Returns the licensing status of all appliances connected to the Manager. ## Configure appliances This section describes how to centrally configure and manage connected appliances. ### Licensing connected appliances There are two ways to license connected appliances: using Spectra Intelligence and using license files. #### Using Spectra Intelligence To license connected appliances using the Spectra Intelligence cloud, configure it for that appliance, either through a group ([Standard configuration groups](#standard-configuration-groups), [Hub groups](#hub-groups)) or individually ([Individual configuration: Spectra Analyze](#individual-configuration-spectra-analyze)). The relevant portion required in the body of the request is: ```json5 { "content": { "cloud__enabled": true, "cloud__url": "data.reversinglabs.com", "cloud__user": "example_username", "cloud__token": "example_password", "cloud__token_set": true } } ``` #### Using license files To license connected appliances using license files, they all need to have a machine ID. Machine IDs can be generated for appliances without one, or generated for all appliances at once. If regenerating machine IDs for all appliances, those that already have a license will have to be licensed again. ``` GET /api/v1/license/ ``` Generates a machine ID only for those appliances that don't have it. The response contains individual objects (each of which represents an appliance) with the following fields: ```json5 { "status": "VALID", "product": "TISCALE_WORKER", "machineId": "string", "version": 0, "serialNumber": "148d7a56-176d-413b-9eb5-c62f6f5bb67a", "licenseType": "WILDCARD", "company": "string", "expiryDate": "2019-08-24", "creationDate": "2019-08-24" } ``` If, for some reason, **all** machine IDs have to be generated, use `GET api/v1/appliances/licensing/machine-id/`. Send one or more machine IDs to ReversingLabs support and, once we respond with license files, upload them to the Manager using the `PUT /api/v1/appliances/licensing/` endpoint. The Manager will automatically match the license to the appropriate appliance. ``` PUT /api/v1/appliances/licensing/ ``` Uploads a license file for one of the connected appliances. Form parameters: - `file` - license file to be uploaded ### Standard configuration groups This section describes actions related to creating and managing configuration groups for Spectra Detect Workers. Start with creating a configuration group: ``` POST /api/v1/config-group/create/ ``` Creates a Worker configuration group. ##### Request body ```json5 { "name": "string" } ``` Then, add the appliances and specify their configuration. Both PUT and PATCH methods are supported. **Note: In this guide, we display only some of the possible configuration options as there are almost 400 configurable fields. See the full API reference on the Manager for all options.** ``` PATCH /api/v1/config-group/content/{group_name}/ ``` Saves configuration (but doesn't apply it). The body must be a JSON object with two top-level fields: `appliances` and `content`. Path parameters: - `group_name`: name of the configuration group Request body: ```json5 { "content": { "snmp__enable": true, "snmp__community": "^$", "snmp__trap_sink_enable": true, "snmp__trap_community": "^$" // ... ``` Check that the configuration is correct: ``` GET /api/v1/config-group/retrieve/{group_name}/ ``` Returns the configuration for the group. Path parameters: - `group_name` - name of the configuration group Apply the configuration: ``` PUT /api/v1/config-group/update/{group_name}/ ``` Applies the configuration set in the previous step. Path parameters: - `group_name`: name of the configuration group Request body: ```json5 { "applyconfiguration": [ "string" // list of appliance names indicating which appliances in the group will receive group configuration ] } ``` - `applyconfiguration`: a list of strings where each string is the name of an appliance in the group. Group appliance configuration will be applied to the appliances listed. To verify that the configuration is applied, call the following endpoint: ``` GET /api/v1/appliances/ ``` Get a list of appliances with their configuration details. The response contains an array called `results`, with objects inside. Each object corresponds to one connected appliance. ```json5 { "id": 0, "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f", "ip": "string", "status": "string", "authorized": true, "online": true, "latency": 0.1, "maintenance": true, "last_seen_online": "2019-08-24T14:15:22Z", "system_data": {}, "haproxy_password": "string", "created": "2019-08-24T14:15:22Z", "updated": "2019-08-24T14:15:22Z", "group_name": "string", "redundancy_status": "string", "configured_redundancy_status": "string", "redundant_with": "string", "connected_workers": [], "application_url": "string", "available_update_version": "string", "has_update_log_endpoint": true, "last_applied_config_timestamp": "2019-08-24T14:15:22Z", "configuration_status": "changes_awaiting_apply", // check this field "configuration_status_details": {} } ``` If you have applied the configuration to all the appliances in a configuration group, then find an object that contains the `group_name` corresponding to your configuration group, and inspect its `configuration_status`. The possible values are: - `changes_awaiting_apply` - `pending` - `error` - `done` - `mismatch` - `unsupported` - `no_configuration_group` If you have several groups with different configurations, use this query to list them: ``` GET /api/v1/config-group/list/ ``` Lists configuration groups. Query parameters: - `page`: response page (if the contents are too large for a single response) - `page_size` - number of results to return per page If necessary, remove a group: ``` DELETE /api/v1/config-group/delete/{group_name}/ ``` Removes a group. Path parameters: - `group_name`: name of the group to remove Here's a list of `curl` commands corresponding to the steps above: ```bash # Add --insecure before the URL if you are using a self-signed SSL certificate curl -X POST https://{server}/api/v1/config-group/create/ -H 'Authorization: Token ' -H 'Content-Type: application/json' -d '{"name":"TestGroup1"}' curl -X PATCH https://{server}/api/v1/config-group/save/TestGroup1/ -H 'Authorization: Token ' -H 'Content-Type: application/json' -d '{"appliances":["Test"]}' curl -X PATCH https://{server}/api/v1/config-group/save/TestGroup1/ -H 'Authorization: Token ' -H 'Content-Type: application/json' -d '{"content": { "cloud__enabled":true, "cloud__url":"https://appliance-api.reversinglabs.com", "cloud__user":"Test", "cloud__token":"Test", "cloud__timeout":60, "cloud__proxy_enabled":false, "cloud__proxy_host":"", "cloud__proxy_port":null, "cloud__proxy_user":"", "cloud__proxy_password":"", "cloud__max_fetch_file_size":500 } }' curl -X PATCH https://{server}/api/v1/config-group/update/TestGroup1/ -H 'Authorization: Token ' -H 'Content-Type: application/json' -d '{"applyconfiguration":["Test"]}' ``` To reset the configuration of a group to default and apply the current group settings, use the following query: ``` POST /api/v1/config-group/resolve-mismatch/group_name/ ``` Resolves a mismatch in the configuration of a group by resetting it to default and applying the current group settings. Returns a `204` status code. ### Hub groups A special case of configuration groups is using **Hub groups**. These are just like regular configuration groups, but you can also create and manage Hubs, including all the functionalities that they offer. The prerequisite for the following actions is adding at least one Hub using `POST /api/v1/appliances/create/` (you can't have a Hub group without a Hub). The first step is to create a Hub group and add **all Hub-only configuration options**. These settings cannot be changed in a later request and must be set when creating the Hub group. ``` POST /api/v1/tiscale-hub-group/create/ ``` Creates a Hub group. ##### Request body ```json5 { "name": "string", // name of the Hub group "primary_router_id": "string", // ID for the primary Hub (arbitrary string) "primary_host": "string", // name, not URL "fallback_router_id": "string", // also arbitrary string "fallback_host": "string", // name, not URL "vrrp_enabled": false, // enable/disable sharing a virtual IP address "virtual_router_id": 1, // ID of the virtual Hub "auth_pass": "string", // VRRP authentication password "different_networks": true, // whether the Hubs are inside different networks "virtual_ipaddress": "string", // virtual Hub IP address "api_worker_token": "string", // file analysis token "srv_available_worker_token": "string", // service availability check token "srv_worker_token": "string", // operational stats token } ``` After creating a Hub group, the next step is to save and apply its configuration. The steps are the same as for regular configuration groups, and the only difference is the endpoint. **Hub-only parameters** (i.e. those relating exclusively to Hub groups, and not regular configuration groups) **must be set upon Hub group creation**. ``` PUT /api/v1/tiscale-hub-group/save/{group_name}/ ``` Save Hub group configuration. This endpoint will overwrite the existing Hub Group configuration with the values sent in the `content` request parameter, so any existing Hub Group settings will be discarded before the request values are written to the group. Path parameters: - `group_name`: name of the hub group Request body: ```json5 { "content": { "snmp__enable": true, "snmp__community": "^$", "snmp__trap_sink_enable": true, "snmp__trap_community": "^$" // ... } } ``` ``` PATCH /api/v1/tiscale-hub-group/save/{group_name}/ ``` This endpoint will override the existing Hub Group configuration with the values sent in the `content` request parameter, so any existing Hub Group settings not listed will be preserved, while those listed will be overwritten. Path parameters: - `group_name`: name of the hub group Request body: ```json5 { "content": { "snmp__enable": true, "snmp__community": "^$", "snmp__trap_sink_enable": true, "snmp__trap_community": "^$" // ... ``` ------------------------------------------------ ``` PUT /api/v1/tiscale-hub-group/update/{group_name}/ ``` Applies the configuration set in the previous step and resolves any settings mismatch by resetting any settings not configured in the Hub Group. This endpoint will not make any changes to the Hub Group appliance configuration stored on the Manager. This endpoint will apply Hub Group configuration from the Manager to the group appliances listed in the `applyconfiguration` request parameter, and resolve any configuration mismatch on those appliances. Path parameters: - `group_name`: name of the hub group Request body: ```json5 { "applyconfiguration": [ "string" // list of which group appliances will receive hub group configuration ], "primary_router_id": "string", // ID for the primary Hub (arbitrary string) "primary_host": "string" // hostname or IP address, not URL } ``` ------------------------------------------------ ``` PATCH /api/v1/tiscale-hub-group/update/{group_name}/ ``` Applies the configuration set in the previous step and resolves any settings mismatch by resetting any settings not configured in the Hub Group. This endpoint will not make any changes to the Hub Group appliance configuration stored on the Manager. This endpoint will apply Hub Group configuration from the Manager to the group appliances listed in the `applyconfiguration` request parameter, and resolve any configuration mismatch on those appliances. Path parameters: - `group_name` - name of the hub group Request body: ```json5 { "applyconfiguration": [ "string" // list of appliance names indicating which appliances in the group will receive hub group configuration ], "primary_router_id": "string", // ID for the primary Hub (arbitrary string) "primary_host": "string" // hostname or IP address, not URL } ``` The rest of the configuration steps (verifying, checking the timestamp etc.) are described in the chapter on [Standard configuration groups](#standard-configuration-groups). ### Individual configuration: Spectra Analyze It is also possible to configure only individual Spectra Analyze appliances (instead of through groups). ``` PUT /api/v1/appliances/content/{appliance_id}/save/ ``` Saves the entire configuration (but doesn't yet apply it). Path parameters: - `appliance_id`: ID of the appliance There are many configurable fields that you can send (the full list is in the reference on the Manager). While the request above uses PUT, it is also possible to use PATCH. All the parameters are the same, and the only difference is that PATCH allows partial saving (and PUT always replaces the entire configuration). After saving the configuration, you once again must apply it: ``` POST /api/v1/appliances/content/{appliance_id}/apply/ ``` Applies the uploaded configuration to Spectra Analyze. ### Set a timezone This section covers listing and setting the timezone of the Spectra Analyze, Hub, and Worker. List available timezones for the Spectra Analyze, Hub, and Worker. The response contains a list of timezones that can be set. ``` GET /api/v1/system/config/list-available-timezones/ ``` Response: ```json5 { "timezones": [ "string" ] } ``` Set the timezone of the Spectra Analyze, Hub, or Worker. ``` POST /api/v1/appliances/{appliance_id}/system/configure-timezone/ ``` Path parameters: - `appliance_id`: ID of the appliance ## Configure connectors ### Enable connector To enable a connector for a specific appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` POST /api/v1/appliances/{id}/connectors/{connector_name}/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector to enable. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` ### Get connector information To retrieve information about a specific connector of an appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` GET /api/v1/appliances/{id}/connectors/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). For a complete list of available information options, response fields, and examples for each connector service, refer to the API Reference Guide on the Manager. ### Get connector configuration To retrieve the configuration for a specific connector of an appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` GET /api/v1/appliances/{id}/connectors/{connector_name}/v1/config/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector whose configuration should be returned. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` For a complete list of available configuration options, response fields, and examples for each connector service, refer to the API Reference Guide on the Manager. ### Configure connector To configure a specific connector for an appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` POST /api/v1/appliances/{id}/connectors/{connector_name}/v1/config/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector that should be configured. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` #### Request body For a complete list of available configuration options, response fields, and examples for each connector service, refer to the API Reference Guide on the Manager. ### Test connector connection To test the connection to a connector, use the following endpoint. This action is supported only for appliances of type "Hub". ``` POST /api/v1/appliances/{id}/connectors/{connector_name}/test-connection/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector to test. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` #### Request body For a complete list of available configuration options, response fields, and examples for each connector service, refer to the API Reference Guide on the Manager. ### Disable connector To disable a connector for a specific appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` DELETE /api/v1/appliances/{id}/connectors/{connector_name}/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector to disable. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` ## Configure Central Logging Central Logging provides API endpoints for retrieving and managing data generated during file analysis. It supports access to antivirus scan results, classification outputs, detections overview, and processing statistics. The service enables exporting of classification and error data for analysis and reporting. It also provides operations for rescanning samples and retrieving file type group counts. In addition, Central Logging manages Deep Cloud Analysis hash allowlists and blocklists, allowing listing, adding, and removing SHA1 hashes that control which files are excluded from or forced into scanning. ### Deep Cloud Analysis Hash Allowlist The Deep Cloud Analysis Hash Allowlist is a list of SHA1 hashes that should be treated as safe and excluded from AV scanning. It provides endpoints to retrieve, add, and remove allowlisted hashes, ensuring known safe files are skipped during analysis. #### Retrieve SHA1 hashes on allowlist Retrieves SHA1 hashes currently on the Deep Cloud Analysis allowlist, which are considered safe and are excluded from scanning. ``` GET /api/central-logging/v1/multiscan/hashes/ ``` #### Add SHA1 hashes to allowlist Add one or more SHA1 hashes to the Deep Cloud Analysis hash allowlist. Files with these hashes are treated as safe and skipped from further scanning. Validates input and reports invalid hashes. ``` PATCH /api/central-logging/v1/multiscan/hashes/ ``` #### Remove SHA1 hashes from allowlist Remove one or more SHA1 hashes from the Deep Cloud Analysis hash blocklist. Hashes should be sent in the request body as an array of strings. The response reports which hashes were successfully deleted, not found, or invalid. ``` DELETE /api/central-logging/v1/multiscan/hashes/ ``` ### Deep Cloud Analysis Hash Blocklist The Deep Cloud Analysis Hash Blocklist is a list of SHA1 hashes that should be treated as unsafe and included in AV scanning. It provides endpoints to retrieve, add, and remove blocked hashes, ensuring known unsafe files are scanned during analysis. #### Retrieve SHA1 hashes on blocklist Retrieves SHA1 hashes currently on the Deep Cloud Analysis hash blocklist, which are excluded from further analysis. ``` GET /api/central-logging/v1/multiscan/hashes/ ``` #### Add SHA1 hashes to blocklist Add one or more SHA1 hashes to the Deep Cloud Analysis hash blocklist. Validates input and reports invalid hashes. ``` PATCH /api/central-logging/v1/multiscan/hashes/ ``` --- ## SpectraDetect Prometheus Metrics API — GET /metrics and honor_labels Setup The Metrics API provides an interface for collecting Prometheus-compatible metrics from Spectra Detect appliances. Users can scrape this endpoint to monitor the health and performance of the appliance. ## Prometheus `honor_labels` Configuration When configuring your Prometheus server to scrape metrics from Spectra Detect appliances, we recommend enabling the `honor_labels` setting in your Prometheus scrape configuration. By default, Prometheus renames conflicting labels scraped from targets to avoid collisions with its own internally generated labels (like `instance` and `job`). For example, a target's `job` label might be renamed to `exported_job`. However, Spectra Detect provides certain labels (e.g., `job`) within its exposed metrics that are essential for proper identification and data filtering. Setting `honor_labels: true` in your Prometheus scrape configuration for Spectra Detect targets will instruct Prometheus to preserve the original label names exposed by the appliance. Here's an example of a Prometheus scrape configuration snippet with `honor_labels` enabled: ```yaml scrape_configs: - job_name: 'spectra-detect' honor_labels: true static_configs: - targets: [':'] ``` ## Metrics API response status codes - 200 - The request has succeeded. - 404 - The server has not found anything matching the Request-URI. - 500 - The server encountered an unexpected condition which prevented it from fulfilling the request. - 503 - The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. ## Get Prometheus metrics ``` GET /metrics ``` ### Description Returns the current system and application metrics in a Prometheus-compatible format. This endpoint can be scraped by a Prometheus server to collect data about the appliance. ### Request format **Request examples** ```bash curl -X GET https:///metrics ``` ```python url = "https:///metrics" response = requests.request("GET", url) print(response.text) ``` ### Response format A successful request returns the `200 OK` status code with the metrics in the Prometheus exposition format in the response body. Each appliance returns metrics specific to the services it hosts. All appliances expose `node_*` metrics. Specifically, Hub appliances only provide `node_*` metrics. Manager appliances expose `node_*` and `rabbitmq_*` metrics. Worker appliances provide `node_*`, `rabbitmq_*`, and `rl_*` metrics. ### Aggregating metrics per appliance While the metrics endpoint exposes raw metrics for each internal service (e.g., `job="postprocessor"`, `job="receiver"`), individual service metrics may show a value of 0. To view meaningful metrics for a particular appliance, aggregate the metrics across all services on that appliance using the `sum()` function in your Prometheus queries. For example, to get the total processed bytes for a Worker appliance: ```promql sum(rl_processed_bytes_total{instance="10.200.185.215"}) ``` Replace `10.200.185.215` with the IP address of the target appliance. **Response examples** ``` node_memory_Mapped_bytes{job="node"} 4.29580288e+08 node_network_receive_compressed_total{device="ens192",job="node"} 0 node_network_receive_compressed_total{device="lo",job="node"} 0 node_disk_writes_merged_total{device="dm-0",job="node"} 0 node_disk_writes_merged_total{device="dm-1",job="node"} 0 node_disk_writes_merged_total{device="sda",job="node"} 87857 node_disk_writes_merged_total{device="sr0",job="node"} 0 node_disk_writes_merged_total{device="sr1",job="node"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_retry",job="rabbitmq"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_error",job="rabbitmq"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_input",job="rabbitmq"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_url",job="rabbitmq"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_result",job="rabbitmq"} 0 rl_processed_bytes_total{job="postprocessor"} 0 rl_processed_files_total{job="postprocessor"} 0 rl_submitted_bytes_total{job="postprocessor"} 0 rl_submitted_files_total{job="postprocessor"} 0 rl_unpacked_bytes_total{job="postprocessor"} 0 rl_unpacked_files_total{job="postprocessor"} 0 rl_processed_bytes_total{job="receiver"} 0 rl_processed_files_total{job="receiver"} 0 rl_submitted_bytes_total{job="receiver"} 0 rl_submitted_files_total{job="receiver"} 0 rl_unpacked_bytes_total{job="receiver"} 0 rl_unpacked_files_total{job="receiver"} 0 ``` --- ## SpectraDetect Worker Service API — System Status and Processing Queue The Service API provides an interface for controlling processing-related functionalities on the Worker. Users can check the status and availability of the Worker system, and retrieve file processing statistics to observe changes in system performance. ## Service API Authorization The Worker can be configured to require an authorization token to be present in every API request. The authorization token is a string of alphanumeric characters, between 30 and 60 characters in length. The authorization token should be provided in the HTTP `Authorization` header. The value of this field must be in this format: ``` Token exampletoken ``` where `exampletoken` should be replaced with the user's own authorization token string. If the Worker is configured not to require an authorization token, the `Authorization` header field will be ignored. ```bash curl https://tiscale-worker01/srv/tiscale/v1/sysinfo -H 'Authorization: Token exampletoken' ``` ## Service API Request Format The base URL for the service API is in the following format: ``` https://{server}/srv/tiscale/v1 ``` ## Service API response status codes - 200 - The request has succeeded. - 400 - The request could not be understood by the server due to malformed syntax or missing parameters. - 401 - The request requires authorization from the user. - 403 - Authorization has been provided in the request, but it is not valid. - 404 - The server has not found anything matching the Request-URI. - 500 - The server encountered an unexpected condition which prevented it from fulfilling the request. - 503 - The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. - In case the Spectra Intelligence upload limit quota was exceeded, the response will contain the "Paused - TiCloud quota exceeded" error message. To continue analyzing files, users can temporarily disable Spectra Intelligence, wait until the daily quota renews, or contact ReversingLabs support to increase their daily upload limit. When using the Quotas API endpoints on a Worker instance without a configured Spectra Intelligence account, the response code will be 503, and the error message "TiCloud is not configured." ## Get the current system information ``` GET /srv/tiscale/v1/sysinfo ``` ### Description Returns current system information, indicating the status of processing-related components on the Worker instance. ### Request format **Request examples** ```bash curl -X GET https://tiscale-worker01/srv/tiscale/v1/sysinfo -H 'Authorization: Token exampletoken' ``` ```python url = "https://tiscale-worker01/srv/tiscale/v1/sysinfo" headers = {'Authorization': 'Token exampletoken'} response = requests.request("GET", url, headers=headers) print(response.text) ``` ### Response format A successful request returns the `200 OK` status code with the `sysinfo` structure in the response body. **Response example** ```json { "load_average": [ 0.47, 0.27, 0.28 ], "queue_size": 0, "max_queue_name": "tiscale_hagent_input", "max_queue_size": 0, "hagent_running": true, "scratch_space": 3, "root_space": 3 } ``` **Response fields** | Field Name | Description | Data Type | | ---------------- | ------------------------------------------------------------ | --------- | | `load_average` | Average system load for the past 1, 5, and 15 minutes. | float | | `queue_size` | Amount of files in the input queue. | integer | | `max_queue_name` | Name of queue which currently holds the most files. | string | | `max_queue_size` | Number of files in the currently largest queue. | integer | | `hagent_running` | This field is no longer supported. For the time being, it always returns the `true` value. | boolean | | `scratch_space` | Indicates the percentage of used disk space on the partition holding files to be processed and reports for processed files. | float | | `root_space` | Indicates the percentage of used disk space on the root partition. | float | ## Control the Worker operating mode ### Check current operating mode status ``` GET /srv/tiscale/v1/mode ``` Returns whether the Worker is currently paused or not. Setting the `paused` flag to `true` will cause the Worker to decline all new file submissions until the flag is unset. The flag can be used to put a Worker into maintenance mode or to remove it from a pool of Workers. #### Request format **Request example** ```bash curl -X GET https://tiscale-worker01/srv/tiscale/v1/mode -H 'Authorization: Token exampletoken' ``` #### Response format The response indicates whether the Worker instance is paused (true) or not (false). If the response equals `"paused": true` the Worker is considered unavailable, and is not accepting any new file submissions. **Response example** ```json { "paused": false } ``` ### Change operating mode status ``` PUT /srv/tiscale/v1/mode ``` The operating mode of the Worker instance can be changed by sending a PUT request to the same Service API endpoint. The value of the `Content-Type` HTTP header field must be set to `application/json`. The data supplied in the request must be a JSON object with one or more fields. #### Request format **Request example** Set Worker operating mode to paused ```bash curl -X PUT https://tiscale-worker01/srv/tiscale/v1/mode -H 'Authorization: Token exampletoken' -H 'Content-Type: application/json' -d '{"paused": true}' ``` #### Response format A successful request returns the `200 OK` status code with an empty response body. If the `Content-Type` HTTP header in the request is not set to `application/json`, the response contains an error message "Invalid POST data: expected JSON object". Users experiencing this error message should submit a new request with the incorrect HTTP header properly configured. ## Check file processing availability ``` GET /srv/tiscale/v1/available ``` ### Description Checks whether the Worker instance is available for processing new files. ### Request format **Request example** ```bash curl -X GET https://tiscale-worker01/srv/tiscale/v1/available -H 'Authorization: Token exampletoken' ``` ### Response format If the Worker is available, it returns the status code `200 OK` with no message in the response body. If the instance is not available (`503 Service Unavailable`), it's either in an error state, or paused. There are two types of *paused* states: 1. Operating mode set to `paused`. In this case, the response body contains the following message: `paused`. See [control the Worker operating mode](#control-the-worker-operating-mode). 2. Paused due to an exceeded Spectra Intelligence quota. In this case, the response body contains the following message: `paused - TiCloud quota exceeded`. Depending on how your quota is configured (number of uploads vs. total byte size uploaded; daily vs. monthly), either wait for the quota to renew automatically or contact support to modify your quota. Alternatively, remove the Spectra Intelligence account altogether (this will disable AV scans in the cloud - only static analysis will be available). If the Worker is in an error state, for example due to a problem with system resources, the response body contains a `not healthy` message. ## Get file processing statistics ``` GET /srv/tiscale/v1/stats ``` ### Description Returns the file processing statistics for the Worker instance. ### Request format **Request example** ```bash curl https://tiscale-worker01/srv/tiscale/v1/stats -H 'Authorization: Token exampletoken' ``` ### Response format The format of the data returned is described by the `stats` structure. ``` stats := { "timeframes" : ["1m", "10m", "1h", "8h", "1d"], "files_submitted" : [, , , , ], "bytes_submitted" : [, , , , ], "files_processed" : [, , , , ], "bytes_processed" : [, , , , ], "files_unpacked" : [, , , , ], "bytes_unpacked" : [, , , , ], "mean_queue_size" : [, , , , ], } ``` Statistics information is accrued and displayed for multiple time windows. The `timeframes` field lists shorthand names for those time windows (1m - one minute, 1h - one hour, etc.). - `ųpload_success` shows the number of successful file uploads to the Worker. - `upload_failure` shows the number of failed file uploads to the Worker. - `s3_unpacked_success` shows how many children samples successfully unpacked to the S3 bucket. - `s3_report_upload` shows how many reports were successfully uploaded to the S3 bucket. - `s3_upload` shows the number of files that were successfully uploaded to the S3 bucket. - `s3_unpacked_failure` shows how many children samples failed to unpack to the S3 bucket. - `sns_failure` shows the number of Amazon SMS messages that failed to deliver. - `max_mean_queue_name` shows the names of queues with the largest mean amount of tasks in that timeframe. - `general_service_failure` shows the amount of service errors (5xx server errors). - `callback_failure` shows how many reports failed to upload to the Callback server. - `files_submitted` shows the number of files that were submitted for processing. - `mean_queue_size` shows the mean value of the input queue size for each of the time windows. - `general_user_failure` shows the number of errors caused by the user, for example by malformed API requests (4xx client errors). - `files_processed` and `bytes_processed` fields show the number of files/bytes that were successfully processed. - `splunk_failure` shows how many reports failed to upload to Splunk. - `s3_upload_failure` shows the number of unsuccessful uploads to the S3 bucket. - `bytes_submitted` shows the total number of bytes of all files that were submitted. - `splunk_success` shows how many reports were successfully uploaded to Splunk. - `callback_success` shows how many reports were successfully uploaded to the Callback server. - `not_available` is triggered when the Worker is in an unhealthy state (for example, disk usage is above the set threshold). - `sns_success` shows the number of successfully delivered Amazon SMS messages. - `max_mean_queue_size` shows the mean amount of tasks for queues listed in `max_mean_queue_name`. - `s3_report_upload_failure` shows how many reports failed to upload to the S3 bucket. - `files_unpacked` and `bytes_unpacked` fields show the number of files/bytes that were extracted (unpacked) from samples that were uploaded for processing. - `file_processing_failure` shows the number of files that the service failed to analyze. **Response example** A successful request returns the `200 OK` status code with the list of processing statistics fields and their values in the response. ```json { "upload_success": [ 7, 15, 15, 15, 15 ], "timeframes": [ "1m", "10m", "1h", "8h", "1d" ], "upload_failure": [ 0, 0, 0, 0, 0 ], "s3_unpacked_success": [ 0, 0, 0, 0, 0 ], "s3_report_upload": [ 0, 0, 0, 0, 0 ], "s3_upload": [ 0, 0, 0, 0, 0 ], "s3_unpacked_failure": [ 0, 0, 0, 0, 0 ], "sns_failure": [ 0, 0, 0, 0, 0 ], "max_mean_queue_name": [ "tiscale.hagent_input", "tiscale.hagent_input", "tiscale.hagent_input", "tiscale.hagent_input", "tiscale.hagent_input" ], "general_service_failure": [ 0, 0, 0, 0, 0 ], "callback_failure": [ 0, 0, 0, 0, 0 ], "bytes_processed": [ 142273712, 304872240, 304872240, 304872240, 304872240 ], "files_submitted": [ 7, 15, 15, 15, 15 ], "mean_queue_size": [ 0.625, 0.08908045977011493, 0.014432029795158285, 0.001795228167709057, 0.0008703953279424976 ], "general_user_failure": [ 0, 0, 0, 0, 0 ], "files_processed": [ 7, 15, 15, 15, 15 ], "splunk_failure": [ 0, 0, 0, 0, 0 ], "bytes_unpacked": [ 303859038, 651126510, 651126510, 651126510, 651126510 ], "s3_upload_failure": [ 0, 0, 0, 0, 0 ], "bytes_submitted": [ 142273712, 304872240, 304872240, 304872240, 304872240 ], "splunk_success": [ 0, 0, 0, 0, 0 ], "callback_success": [ 0, 0, 0, 0, 0 ], "not_available": [ 0, 0, 0, 0, 0 ], "sns_success": [ 0, 0, 0, 0, 0 ], "max_mean_queue_size": [ 0.625, 0.08908045977011493, 0.014432029795158285, 0.001795228167709057, 0.0008703953279424976 ], "s3_report_upload_failure": [ 0, 0, 0, 0, 0 ], "files_unpacked": [ 56, 120, 120, 120, 120 ], "file_processing_failure": [ 0, 0, 0, 0, 0 ] } ``` ## Download support logs ``` GET /api/tiscale/v1/support --output hub_support_logs.zip ``` ### Description Download the support logs from Hub or Worker appliances. The contents of the archive are identical to the contents of the archive generated by clicking the *Download Logs* button on the appliance status page. **Request example** ```bash curl https://detect-hub01/api/tiscale/v1/support --output hub_support_logs.zip ``` --- ## SpectraDetect Usage API — File Submission, Task Status, and Report Retrieval The usage API is the main interface for interacting with the file analysis engine. With this API, users can submit files for analysis, check the status of file processing tasks, retrieve analysis reports when file processing is complete, and delete file processing tasks. ## Worker API Authorization The Worker can be configured to require an authorization token to be present in every API request. The authorization token is a string of alphanumeric characters, between 30 and 60 characters in length. There are several different tokens, depending on the accessed endpoint. The authorization token should be provided in the HTTP `Authorization` header. The value of this field must be in this format: ``` Token exampletoken ``` where `exampletoken` should be replaced with the user's own authorization token string. If the Worker is configured not to require an authorization token, the `Authorization` header field will be ignored. ```bash curl https://tiscale-worker01/api/tiscale/v1/task/42 -H 'Authorization: Token exampletoken' ``` ## API Request Format The base URL for the Spectra Detect API is in the following format: ``` https://{server}/api/tiscale/v1 ``` ## API Response Status Codes | Code | Name | Description | | ---- | --------------------- | ------------------------------------------------------------ | | 200 | OK | The request has succeeded. | | 400 | Bad Request | The request could not be understood by the server due to malformed syntax or missing parameters. | | 401 | Unauthorized | The request requires authorization from the user. | | 403 | Forbidden | Authorization has been provided in the request, but it is not valid. | | 404 | Not Found | The server has not found anything matching the Request-URI. | | 500 | Internal Server Error | The server encountered an unexpected condition which prevented it from fulfilling the request. | | 503 | Service Unavailable | The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. In case the Spectra Intelligence upload limit quota was exceeded, the response will contain the "Paused - TiCloud quota exceeded" error message. To continue analyzing files, users can temporarily disable Spectra Intelligence, wait until the daily quota renews, or contact ReversingLabs support to increase their daily upload limit. When using the [Quotas API](#quotas-api) endpoints on a Worker instance without a configured Spectra Intelligence account, the response code will be 503, and the error message "TiCloud is not configured." | ## License appliances ### Return License Information Returns the current status and detailed licensing information for the appliance. If there is no machine ID, **it will be generated**, and the response will contain information associated with this newly generated machine ID. #### Request ``` GET /api/license/v1/ ``` ```bash curl https://tiscale-worker-01/api/license/v1/ -H 'Authorization: Token exampletoken' ``` #### Response ```json5 { "product" : "string", // always "tiscale-worker" "machineId": "string", // ID of the appliance "version": 1, // version of the license file "serialNumber": "string", // serial number of the license file applied to the Worker "licenseType": "string", // can either be to WILDCARD or REGULAR "company": "string", // company name that the license file is provided for "expiryDate": "string", // date when the license will expire (YYYY-MM-DDTHH:mm:ssZ) "creationDate": "string", // starting date of the license "status": "string" // licensing status } ``` Description of fields: - **licenseType** can either be equal to `WILDCARD` or `REGULAR`: - `WILDCARD` licenses are independent of the machine ID and can last for a maximum of 45 days; trial licenses belong to this category - `REGULAR` licenses depend on the machine ID and have a custom duration - **company** is the company name that the license file is provided for; it will be equal to the Spectra Intelligence username in case the cloud is used for licensing - status can be one of the following: - `VALID` - `ONLINE` - `INVALID` - `EXPIRED` - `NO_LICENSE` - `NO_MACHINE_ID` ### Return License Status Returns the current licensing status of the appliance. #### Request ``` GET /api/license/v1/status/ ``` ``` curl https://tiscale-worker-01/api/license/v1/status/ -H 'Authorization: Token exampletoken' ``` #### Response ```json { "status": "string" } ``` Status can be one of the following: - `VALID` - `ONLINE` - `INVALID` - `EXPIRED` - `NO_LICENSE` - `NO_MACHINE_ID` ### Generate Machine ID Creates a new machine ID. #### Request ``` POST http(s)://{server}/api/license/v1/machine-id/generate ``` The request body is empty. ``` curl -X POST https://tiscale-worker-01/api/license/v1/machine-id/generate -H 'Authorization: Token exampletoken' ``` #### Response - 200 OK `{ "machineId": "string" }` - 500 Internal Server Error `{ "message": "string" }` ### Upload License File Uploads a license file to the appliance. Not required for licenses configured using Spectra Intelligence. #### Request ``` PUT /api/license/v1/upload ``` Content type: `multipart/form-data` Example: ```bash curl -X PUT https://tiscale-worker-01/api/license/v1/upload -H 'Authorization: Token exampletoken' -F 'file=@example_license_file' ``` #### Response ```json { "message": "string" } ``` The `message` will differ depending on the response code: - 201: "License is applied" - 200: "License is already applied" - 400: returns an error message ## Upload a file for processing **Direct upload** File uploads can be performed synchronously or asynchronously. ``` POST /api/tiscale/v1/upload ``` Uploads a file for processing and immediately returns a `task_url` that can be queried later to retrieve the full analysis report once processing is complete. --- ``` POST /api/tiscale/v1/upload/sync ``` Uploads a file for processing and waits until processing is complete, returning the full analysis report directly in the response. Accepts the same parameters as the [Get information about a processing task](#get-information-about-a-processing-task-taskinfo) endpoint. **Upload from URL** ``` POST /api/tiscale/v1/submit-url ``` **Upload a container image** ``` POST /api/tiscale/v1/submit-container-image ``` ### Description Submits a file, a URL-based file, or container image registry reference to the Worker or Hub for analysis; container images are pulled directly from the specified registry. If the request is sent to a Hub with one or more connected Workers, the file(s) will be forwarded to the connected Workers using the round-robin scheduling algorithm, and the API response will include a link to the file processing task on the Worker. Analysis reports cannot be obtained from a Hub. To get the analysis report for each submitted file, users should send a request directly to the Worker using the link to the processing task returned in the API response. See [Get information about a processing task](#get-information-about-a-processing-task-taskinfo) for more information. If you are uploading a file directly, use the `file` parameter and set the request payload type to `multipart/form-data`. Else, if you are uploading a file via a URL, use the `url` parameter and set the request payload type to `application/json`. If you are uploading a container image, send JSON with an `image` field (e.g. `{ "image": "alpine:3.21"}`). Optional fields `image_platform` and `credential_helper_authentication` let you choose a specific architecture or supply registry credentials. Workers can be configured to require an authorization token in every API request. In that case, the `Authorization` header with a token must be included in the request. Users can provide an optional `X-TiScale-Token` header with a string that can be used to filter file processing tasks. Users can also submit optional fields in the file upload request: - `user_data` *type: string | format: JSON* This parameter can be used to store metadata in the analysis reports sent to Splunk, or to enable additional report-related options. The report doesn't include the contents of `user_data`. - `custom_data` *type: string | format: JSON* This parameter can be used to append any additional information about a file to its analysis report. This allows users to filter, reference, or distinguish samples and their analysis reports more easily. The value provided in the parameter is saved and returned in all reports on the Worker appliance - in the Worker API response and the Callback response, in reports sent to Splunk, and in the reports saved to AWS S3 (if enabled). Note that it is not included in reports when the file has not been processed yet. If an `integrity_check_hash` field is included in the `custom_data` field, an integrity check of the uploaded file is triggered on the Worker. If the SHA1 hash of the uploaded sample doesn’t match the `integrity_check_hash` field sent by the client, the sample will be discarded with the following error message: "ERROR: Upload: failed to accept upload integrity check failed, calculated hash XX differs from the one passed in custom data YY." This field is optional and is implemented in the S3 connector. If a container image is submitted, the Worker automatically appends an `image_metadata` object to the `custom_data` section of the analysis report. This metadata includes resolved values such as the image reference and platform. If `custom_data` already contains an `image_metadata` field, it will not be overwritten. If the structure of `custom_data` isn't valid JSON, it will not be included in the report. To get the analysis report for each submitted file, users should send a request to the [Get information about a processing task](#get-information-about-a-processing-task-taskinfo) endpoint. **Note: **Data retention**** All processed files are deleted immediately after processing. The reports are retained on the Workers for up to 90 minutes (default value), configurable by the "Task Age Limit" setting in the Hub Configuration Group under Worker Configuration - Cleanup. If a file is queued but not processed within the time configured by the "Unprocessed Task Limit", the processing task will be canceled, and the file will be deleted. However, the record of the unsuccessful task will still be present in the database for 1440 minutes (24 hours) before being deleted, as configured by the "File Age Limit" setting. ### Request format **Request parameters** | Parameter Name | Description | Required | |-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| | `file` | Path to the file that should be uploaded to the Worker for processing. Mutually exclusive with the `url` and `image` parameters. Used for the `upload` endpoint. | One of `file`, `url`, or `image` | | `url` | URL to the file that should be uploaded to the Worker for processing. Mutually exclusive with the `file` and `image` parameters. Used for the `submit-url` endpoint. | One of `file`, `url`, or `image` | | `image` | Container image reference (e.g. `busybox`, `alpine:3.21`). Mutually exclusive with `file` and `url`. Used for the `submit-container-image` endpoint. | One of `file`, `url`, or `image` | | `image_platform` | Target platform for the image (e.g. `linux/amd64`, `windows/amd64:10.0.26100.3476`). If not provided, the Worker uses its host platform. | Optional | | `credential_helper_authentication` | Registry credentials using the Docker credential helper format:`{ "Username": "your-user", "Secret": "your-token" }`. Only needed for private registries. | Optional | | `custom_data` | Accepts any user-defined data as a JSON-encoded payload. This data is included in all file analysis reports (Worker API, Callback, AWS S3 and Splunk, if enabled) in its original, unchanged form. The `custom_data` field will not be returned in the `taskinfo` endpoint response if the file has not been processed yet. | Optional | | `user_data` | Additional JSON-encoded payload can be supplied in a request in the `user_data` field. Portions of that data will be used in parts of the processing pipeline. The contents of `user_data` are not returned in the analysis report, except the `network` object in reports sent to Splunk (if configured). Supported objects are `network`, `callback`, `deep_cloud_analysis`, as well as the `password_list` array. | Optional | | `hash` | Optional parameter used for integrity verification of the downloaded file. The supported hash values are: `MD5`, `SHA1`, `SHA256`, or `SHA512`. The proper algorithm is determined automatically based on the hash length: `MD5` (16 bytes), `SHA1` (20 bytes), `SHA256` (32 bytes), `SHA512` (64 bytes). | Optional | | `custom_authorization_header` | Specifies a custom authorization header to be included in the HTTP request. Provide the header name and its corresponding value in the `custom_authorization_header` field. For example, to include an authorization token in the `Custom-Token` header, use `"custom_authorization_header": {"Custom-Token": "token_value"}`. If not provided, no custom header is added to the HTTP request. | Optional | `user_data.password_list` If the submission is a password-protected archive, provide a password list to be used to decrypt the archive. ```json { "password_list": ["pass", "pass2", "pass3"] } ``` `user_data.network` Optional network-related metadata can be supplied in the file upload request. If the Worker is configured to integrate with Splunk, this network metadata will also be sent to Splunk. ```json { "network": { "src_ip": "10.1.10.42", "src_port": 10621, "dst_ip": "10.1.10.212", "dst_port": 8080 } } ``` `user_data.callback` If the Callback feature is enabled and configured on the Worker instance, file analysis reports will be automatically sent to the configured Callback server. By default, only the medium (summary) report is sent to the Callback server. However, users can include an optional `callback` JSON object in the `user_data` field of the file upload request to have an extended analysis report delivered to the Callback server. The prerequisite is that the Callback feature is enabled on the Worker instance. ```json { "callback": { "report_type": "large", "view": "flat", "url": "http://example.com:8080" } } ``` By default, the report returned by the Worker is a **medium (summary) report**. Setting it to `large` will return the full report, whereas `small` and `medium` are default options that filter out certain fields. The `view` parameter can be used to apply a custom view to the report. For example, to enable sending an extended analysis report for the submitted file to the Callback server, provide the `view` parameter with the `extended` value. The `url` parameter specifies to which URL the report will be sent. If this is not specified, the report is sent to the default URL configured. If there is no `callback` object in the file upload request, the Worker uses preconfigured values. `user_data.deep_cloud_analysis` If the Deep Cloud Analysis is enabled in the **Administration > Spectra Detect Manager > Dashboard Configuration** section, the `deep_cloud_analysis` object can be included in the `user_data` field of the file upload request. The `deep_cloud_analysis` object contains optional parameters that can be used for Deep Cloud Analysis. ```json { "deep_cloud_analysis": { "full_scan": true, "skip_av_scan": true, "skip_wait_for_av_results": true, "wait_timeout_in_minutes": 30, "skip_unpacked_files": true, "rescan_enabled": true, "rescan_threshold_in_days": 0, "hash_block_list": [ "SHA1 hash", "SHA1 hash", "SHA1 hash" ] } } ``` - `full_scan`: Perform AV (re)scan of the container and unpacked samples ignoring the XREF record scan date. If set to `true`, it overrides any other configured setting except for the `hash_block_list`. The file size limit is 2.2 GB. - `skip_av_scan`: Skip the AV scan and process the sample. Sample upload needs to be enabled. - `skip_wait_for_av_results`: Skip waiting for the AV results. Has to be globally enabled in the **Central Configuration > Spectra Intelligence > Deep Cloud Analysis** section. - `wait_timeout_in_minutes`: Timeout in minutes for waiting for the AV results. - `skip_unpacked_files`: Skip AV analysis of unpacked files. Has to be globally enabled in the **Central Configuration > Spectra Intelligence > Deep Cloud Analysis** section. - `rescan_enabled`: Modify global rescan for current analysis. - `rescan_threshold_in_days`: Modify rescan threshold in days for current analysis. `0` indicates to always re-scan. - `hash_block_list`: An extension of the global hash block list defined via the Spectra Detect APIs. For more information on the Deep Cloud Analysis pipeline, refer to the [Deep Cloud Analysis Pipeline](../Usage/Analysis.md#deep-cloud-analysis-pipeline-diagram) diagram. **Request examples** ```bash curl -X POST https://tiscale-worker-01/api/tiscale/v1/upload -H 'Authorization: Token exampletoken' -F file=@Classification.zip -F 'user_data={"callback": {"view": "flat", "report_type": "large", "url": "https://example.com:8080"}, "password_list": ["pass","pass2","pass3"]}' curl -X POST https://tiscale-worker-01/api/tiscale/v1/upload -H 'Authorization: Token exampletoken' -F file=@Classification.pdf -F 'custom_data={ "file_source" : { "uploader" : "malware_analyst", "origin" : "example" }}' curl -X POST https://tiscale-worker-01/api/tiscale/v1/upload -H 'Authorization: Token exampletoken' -H 'X-TiScale-Token: Token custom' -F file=@Classification.pdf curl -X POST https://tiscale-worker-01/api/tiscale/v1/submit-url -H 'Authorization: Token exampletoken' -H 'Content-Type: application/json' -d '{"url": "https://example.com/file"}' curl -X POST https://tiscale-worker-01/api/tiscale/v1/submit-url -H "Content-Type: application/json" -d '{"url": "https://example.com/file", "custom_authorization_header": {"Custom-Token": "token_value"}}' # Upload a container image – Docker Hub, default tag (latest) curl -X POST https://tiscale-worker-01/api/tiscale/v1/submit-container-image \ -H 'Authorization: Token exampletoken' \ --json '{ "image": "busybox" }' # Upload with platform and tag curl --json '{ "image": "alpine:3.21.3", "image_platform": "linux/arm64/v8" }' \ https://tiscale-worker-01/api/tiscale/v1/submit-container-image # Upload from private registry with credentials curl --json '{ "credential_helper_authentication": { "Username": "", "Secret": "" }, "image": "private.registry.com/image:1.0.0", "image_platform": "linux/amd64" }' https://tiscale-worker-01/api/tiscale/v1/submit-container-image ``` For additional workflows, such as scanning entire directories, see our [SDK documentation and its cookbook](https://github.com/reversinglabs/reversinglabs-sdk-cookbook). ### Response format The response is a JSON-encoded object with a `task_url` field which contains the URL of the processing task started by the file or URL submission. If the task URL in the response is missing the `server` component, the same server value used for the file submission request should be used to access the task URL. The response will also contain a link to an S3 report if an output report bucket is configured. **Response example** ```json { "task_url": "https://tiscale-worker01:8080/api/tiscale/v1/task/1" } ``` ## List processing tasks ``` GET /api/tiscale/v1/task?age=700&token=token ``` ### Description Lists processing tasks generated by file submission requests. When a file is submitted for analysis to a Worker, a file processing task is created and queued on the Worker server. **Note: **Data retention**** All processed files are deleted immediately after processing. The reports are retained on the Workers for up to 90 minutes (default value), configurable by the "Task Age Limit" setting in the Hub Configuration Group under Worker Configuration - Cleanup. If a file is queued but not processed within the time configured by the "Unprocessed Task Limit", the processing task will be canceled, and the file will be deleted. However, the record of the unsuccessful task will still be present in the database for 1440 minutes (24 hours) before being deleted, as configured by the "File Age Limit" setting. This endpoint returns a list of processing tasks that have not yet expired; that is, tasks for which it is still possible to retrieve the analysis report. Every processing task is assigned a unique ID, returned in the `task_id` response field. This ID can be used in requests to other Worker API endpoints. If the optional `age` parameter is supplied in the request, only tasks older than the specified number of seconds are returned in the response. Task age is calculated as being the difference between the current system timestamp and the timestamp of the task submission. Timestamps are specified in the Unix time format, as the number of seconds since 1970-01-01 00:00:00. When submitting files for analysis, users can add an optional `X-TiScale-Token` header with a custom string to the file upload request. That custom string can be provided as the value for the optional `token` parameter in the request to this endpoint. The response will list only the tasks for files submitted with the specified custom string. **These parameters should not be confused with the mandatory authorization token that must be provided in the header of every request.** ### Request format **Request parameters** | Parameter Name | Description | Required | | -------------- | ------------------------------------------------------------ | -------- | | `age` | Specify the number of seconds to filter processing tasks by age. When provided in the request, the API returns only those processing tasks that are older than the specified number of seconds. Task age is the difference between the current system time and the time when the file processing task was created. | Optional | | `token` | Specify a custom string to filter processing tasks, if there are any files that were uploaded with that custom string in the `X-TiScale-Token` header. | Optional | **Request examples** Listing all available tasks for files uploaded with a custom X-TiScale-Token string: ```bash curl -X GET 'https://tiscale-worker-01/api/tiscale/v1/task?token=custom' -H 'Authorization: Token exampletoken' ``` Listing all available tasks older than 500 seconds: ```bash curl -X GET 'https://tiscale-worker-01/api/tiscale/v1/task?age=500' -H 'Authorization: Token exampletoken' ``` ```python url = "https://tiscale-worker-01/api/tiscale/v1/task" querystring = {"age":"500"} headers = {'Authorization': 'Token exampletoken'} response = requests.request("GET", url, headers=headers, params=querystring) print(response.text) ``` ### Response format If the request was successfully sent, the response includes a `200 OK` status code and the list of processing tasks for which it is still possible to retrieve the analysis report. **Response example** ```json [ { "submitted": 1481137280, "task_id": 27637, "processed": 1481137281 }, { "submitted": 1481137302, "task_id": 27638, "processed": 1481137303 } ] ``` **Response fields** | Field Name | Description | | ----------- | ------------------------------------------------------------ | | `submitted` | Timestamp indicating the time when the file was submitted for analysis, formatted as the number of seconds since 1970-01-01 00:00:00. | | `processed` | Timestamp indicating the time when the file was processed by the Worker, formatted as the number of seconds since 1970-01-01 00:00:00. | | `task_id` | Unique identification number assigned to each file processing task on the Worker instance. This task ID can be used in requests to other Worker API endpoints; for example, to retrieve the analysis report for a task, or to delete a task from Worker. To ensure backward compatibility with previous Worker versions, the task ID values are currently integers instead of strings. | ## Get information about a processing task {#get-information-about-a-processing-task-taskinfo} ``` GET /api/tiscale/v1/task/{task_id}?report_type={large}&view={flat} ``` ### Description Retrieves information about a completed file processing task. The `task_id` parameter is required, as it specifies which processing task the user is interested in. **Obtaining the task ID** The unique ID value for every processing task is returned in the response when the file is submitted for analysis, as the last part of the `task_url` field. For example, in `https://tiscale-worker01:8080/api/tiscale/v1/task/23`, the `task_id` value is 23. The task ID can also be obtained from the **List processing tasks** endpoint. ### Request format #### Request parameters | Parameter Name | Description | Type | | -------------- | ------------------------------------------------------------ | ----- | | `task_id` | Unique identification number assigned to each file processing task on the Worker instance. | path | | `report_type` | Applied report filter (see [Customizing Analysis Reports](../Usage/Analysis.md#customizing-analysis-reports)). If empty, a medium ("summary") report is returned. | query | | `view` | Applied report transformation. | query | | `top_container_only` | If set to `true`, the report will include metadata only for the top-level container. Reports for unpacked (child) files will be omitted. | query | | `malicious_only` | If set to `true`, the report will include only files and objects marked as malicious or suspicious. Benign content and clean children will be excluded from the report. | query | #### Deprecated and experimental parameters These parameters are maintained for backward compatibility or are experimental. - `include_fields` *experimental | query* Instead of creating a *report type* as a configuration file, you can pass the fields you want to include as a query parameter. The format is: `?include_fields=["metadata.media.image.exif","classification"]`. You can access nested fields using the dot (`.`) operator. All required fields should be passed as an array of strings in a **single query parameter**. - `exclude_fields` *experimental | query* Instead of creating a report type as a configuration file, you can pass the fields you want to exclude as a query parameter. The format is: `?exclude_fields=["metadata.media.image.exif","classification"]`. You can access nested fields using the dot (`.`) operator. All required fields should be passed as an array of strings in a **single query parameter**. - `full` *deprecated | query* Specifies whether the full (value 1; equals `true`) or summary (value 0; equals `false`) report should be returned. If this parameter is not provided in the request, and the `view` parameter is, the built-in summary transformation will be applied to the analysis report view. To avoid this transformation, the `full` parameter should be set to `true` if the `view` parameter is set to `extended`. - `v13` *deprecated | query* Specifies whether the report should be returned in the format used on Worker 1.3 (value 1, equals `true`) or in an improved format (value 0, equals `false`). Reports in version 1.3 represented enumerated values in their internal numeric value instead of as descriptive string values. For example, metadata for application library types was returned as numerical values 0 through 13 in version 1.3 instead of labels "ad", "social", "graphical" in the current version. This parameter should be used only for backward compatibility purposes because it does not take advantage of caching, which may slow down the API requests. **Request examples** ```bash curl 'https://tiscale-worker-01/api/tiscale/v1/task/7?report_type=large&view=flat' -H 'Authorization: Token exampletoken' ``` ```python url = "https://tiscale-worker-01/api/tiscale/v1/task/7" query_params = {"report_type": "large", "view": "flat"} auth = {'Authorization': 'Token exampletoken'} response = requests.get(url, headers=auth, params=query_params) print(response.text) ``` ### Response format The response returns information about the requested file processing task, and if the task has completed successfully, includes the analysis report. | Field Name | Description | |-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `submitted` | Unix timestamp indicating the time when the file was submitted for analysis. | | `task_id` | Unique identification number assigned to each file processing task on the Worker instance. This task ID can be used in requests to other Worker API endpoints; for example, to delete a task from Worker. To ensure backward compatibility with previous Worker versions, the task ID values are currently integers instead of strings. | | `processed` | Unix timestamp indicating the time when the file was processed by the Worker. If the file has not been processed yet, this field returns a null value until both the report and the file link are ready. | | `worker_ip` | List of IP addresses associated with the Worker that processed the file. | | `worker_address` | List of DNS records associated with the Worker that processed the file. | | `worker_hostname` | Hostname of the Worker that processed the file. | | `direct_sender` | IP address of the machine that sent the file to Spectra Detect. This is the immediate sender, the last hop in the forwarding chain (if the request was forwarded). | | `forwarded_for` | List of IP addresses that were part of the forwarding chain for that request, including the original sender. | | `tc_report` | Results of the static file analysis performed by the engine embedded in the Worker. If the report is already available, but the file has not yet been saved to S3, this field will return a null value. | | `file_link` | Returned only if the S3 integration is configured on the Worker, and if saving samples to S3 is enabled. When this field is present in the response, it includes a link to the sample stored in an S3 bucket. If Spectra Analyze integration and S3 are enabled on the Worker, the file link will point to the Spectra Analyze instance connected to the Worker. If report splitting is enabled, this link will point to the parent sample. | | `custom_data` | Returned only if the user submitted an optional `custom_data` parameter in the file upload request. This parameter can be passed to the `upload` endpoint to save arbitrary, user-defined data in the analysis report. The data must be submitted as a JSON-encoded payload, and it is returned in the report in its original, unchanged form. | | `deep_cloud_analysis` | Returned only if the Deep Cloud Analysis is enabled. When this field is present in the response, it includes the status of the Deep Cloud Analysis and the number of submitted (`submitted_av_scans`) and completed (`completed_av_scans`) AV scans. Available statuses of the Deep Cloud Analysis: `PREPROCESSING` - The file is still being unpacked (if the `Scan unpacked children` option is enabled), or the upload or re-scan of samples for that container is still in progress on Spectra Intelligence. `WAITING_FOR_AV_RESULTS` - Awaiting AV results, as not all AV scans are complete and the `Wait for AV scan` option is enabled for the analysis. `TIMED_OUT` - The wait timeout for AV scans has expired. Processing/reporting continues without all AV results being available. `FINISHED` - All AV scans have completed successfully. `NO_AV_ANALYSIS` - No AV analysis was triggered for this sample. For example, all samples may have been filtered out due to the `Rescan Interval` setting. `NO_WAIT` - Upload or re-scan was initiated on Spectra Intelligence, but the report will not wait for AV results to be included. `SUBSCRIBE_ONLY` - Deep Cloud Analysis is disabled, but the Cloud Classification Change feature is active. | ## Delete a processing task ``` DELETE /api/tiscale/v1/task/{task_id} ``` ### Description Deletes a processing task record from the system. All file processing results are automatically removed from the system 30 minutes after processing is completed. However, users can manually delete task records from the system at any time. When a task record is deleted, it is no longer listed in the response of the *List processing tasks* endpoint. The `task_id` parameter is required in the request, as it specifies which task record should be deleted. To bulk-delete task records, use the *Delete multiple processing tasks* endpoint. ### Request format **Request parameters** | Parameter Name | Description | Required | | -------------- | ------------------------------------------------------------ | -------- | | `task_id` | Unique identification number assigned to each file processing task on the Worker instance. The unique ID value for every processing task is returned in the response when the file is submitted for analysis, as the last part of the `task_url`. For example, in `"task_url": "https://tiscale-worker01:8080/api/tiscale/v1/task/23"`, the `task_id` value is 23. The task ID can also be obtained from the [List processing tasks](#list-processing-tasks) endpoint. | Required | **Request example** ```bash curl -X DELETE https://tiscale-worker01:8080/api/tiscale/v1/task/999 -H 'Authorization: Token exampletoken' ``` ### Response format If a task was successfully removed from the system, the API returns the `200 OK` status code with an empty response body. ## Delete multiple processing tasks ``` DELETE /api/tiscale/v1/task?{age=seconds} ``` ### Description Deletes task records from the system based on the time when they were submitted. All file processing results are automatically removed from the system 30 minutes after processing is completed. However, users can manually delete task records from the system at any time. When task records are deleted, they are no longer listed in the response of the [List processing tasks](#list-processing-tasks) endpoint. The `age` parameter is required, and it sets the limit for task age, expressed in seconds. Task age is calculated as being the difference between the current system timestamp and the timestamp of the task submission. Timestamps are specified in the Unix time format, as the number of seconds since 1970-01-01 00:00:00. All task records older than the number of seconds specified in the request will be deleted. To delete all tasks, set the `age` parameter to zero. ### Request format **Request parameters** | Parameter Name | Description | Required | | -------------- | ------------------------------------------------------------ | -------- | | `age` | Specify the number of seconds to delete processing tasks based on their age. Task age is the difference between the current system time and the time when the file processing task was created. The value must be specified in seconds. | Required | **Request examples** Delete tasks older than 500 seconds: ```bash curl -X DELETE 'https://tiscale-worker01:8080/api/tiscale/v1/task?age=500' -H 'Authorization: Token exampletoken' ``` Delete all tasks on the system: ```bash curl -X DELETE 'https://tiscale-worker01:8080/api/tiscale/v1/task?age=0' -H 'Authorization: Token exampletoken' ``` ### Response format The endpoint returns an empty body with one of the status codes, depending on the success of the request. | Code | Description | |------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 200 | The request has succeeded. | | 400 | The request could not be understood by the server due to malformed syntax or missing parameters. | | 401 | The request requires authorization from the user. | | 403 | Authorization has been provided in the request, but it is not valid. | | 404 | The server has not found anything matching the Request-URI. | | 500 | The server encountered an unexpected condition which prevented it from fulfilling the request. | | 503 | The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. In case the Spectra Intelligence upload limit quota was exceeded, the response will contain the "Paused - TiCloud quota exceeded" error message. To continue analyzing files, users can temporarily disable Spectra Intelligence, wait until the daily quota renews, or contact ReversingLabs support to increase their daily upload limit. When using the Quotas API endpoints on a Worker instance without a configured Spectra Intelligence account, the response code will be 503, and the error message "TiCloud is not configured." | ## Get the identifier for YARA rules currently on Worker ``` GET /api/tiscale/v1/yara ``` ### Description Retrieves an identifier for the current set of YARA rules on the Worker. If there are any changes to the set of YARA rules, the identifier will change. Therefore, this endpoint can be used to monitor changes to YARA rules by comparing the responses retrieved over multiple time intervals. Read more about [managing YARA rulesets on the Worker](../Config/YARASync.md). ### Request format **Request example** ```bash curl https://tiscale-worker-01/api/tiscale/v1/yara -H 'Authorization: Token exampletoken' ``` ### Response format **Response example** ```json { "id": "790473eae77e007cff9b88bc37a44596fa6a495a" } ``` ## Worker - Get YARA rule synchronization status. ``` GET /api/tiscale/v2/sync-status ``` ### Description Retrieves the current YARA rule synchronization status. Use the optional `errorOnly` query parameter to filter the response to include only rulesets with errors. If there are any changes to the set of YARA rules, the identifiers will change. Therefore, this endpoint can be used to monitor changes to YARA rules. ### Request format **Request example** ```bash curl https://tiscale-worker-01/api/tiscale/v2/sync-status -H 'Authorization: Token exampletoken' ``` ### Response format **Response example** ```json { "revision": 0, "rulesets": [ { "revision": 0, "key": "string", "hash": "string", "errors": [] } ] } ``` Appliances utilize revision numbers and keys as unique identifiers during ruleset synchronization. ## Worker - Resynchronize all YARA rules ``` POST /api/tiscale/v1/pull/all ``` ### Description Resynchronizes all YARA rules, ensuring the latest versions are retrieved and applied. This endpoint updates all rulesets regardless of their current revision status. ### Request format **Request example** ```bash curl -X POST https://tiscale-worker-01/api/tiscale/v1/pull/all -H 'Authorization: Token exampletoken' ``` ### Response format **Response example** If the request was successfully sent, the response includes a `200 OK` status code. In case the YARA synchronization is disabled, or the request fails, the response is `417 Expectation Failed`. ## Get information on usage quotas and limits {#quotas-api} The following endpoints can be used to return quota usage and quota limits for the Spectra Intelligence account configured on the Worker, or at the company level based on the Spectra Intelligence username. Products are grouped into one object if they share the usage quota and access rights. This means that the same users and products can appear multiple times in the response. Workers can be configured to require an authorization token in every API request. In that case, the `Authorization` header with a token must be included in the request. ``` GET /api/tiscale/v1/quota ``` ``` GET /api/tiscale/v1/quota/company ``` **Request Example** ```bash curl https://{server}/api/tiscale/v1/quota -H 'Authorization: Token exampletoken' ``` **Response Example** ```json { "limits": [ { "limit": 10, "limit_type": "daily", "limit_exceeded": false, "products": [ "TCA-0319 YARA Retroactive Hunting", "TCA-0303 YARA Hunting" ], "users": [ "u/company/user" ] }, { "limit": 500, "limit_type": "count", "limit_exceeded": false, "products": [ "TCA-0319 YARA Retroactive Hunting", "TCA-0303 YARA Hunting" ], "users": [ "u/company/user" ] } ], "usages": { "daily": { "date": "2023-11-23", "usage_report": [ { "product": "TCA-0103 Historic Multi-AV Scan Records", "number_of_queries": 6453 }, { "product": "TCA-9999", "number_of_queries": 87 }, { "product": "TCAI-0001 RCA2", "number_of_queries": 6415 }, { "product": "TCAI-0011 Sample Submission Counter", "number_of_queries": 17, "used_bytes": 171871643 } ] }, "monthly": { "month": "2023-11", "usage_report": [ { "product": "TCA-0103 Historic Multi-AV Scan Records", "number_of_queries": 7183 }, { "product": "TCA-0104 File Analysis - Hash", "number_of_queries": 6 }, { "product": "TCA-0201 File Download", "number_of_queries": 6 }, { "product": "TCA-0206 Alert on Reputation and Metadata Changes", "number_of_queries": 12 }, { "product": "TCA-0303 (Matches Feed)", "number_of_queries": 32642 }, { "product": "TCA-0319 (Matches Feed)", "number_of_queries": 32638 }, { "product": "TCA-0403 URL Threat Intelligence", "number_of_queries": 5 }, { "product": "TCA-0407 Network Reputation API", "number_of_queries": 20 }, { "product": "TCA-9999", "number_of_queries": 6273 }, { "product": "TCAI-0001 RCA2", "number_of_queries": 7403 }, { "product": "TCAI-0011 Sample Submission Counter", "number_of_queries": 64, "used_bytes": 352578582 }, { "product": "TCAI-0402 URI Statistics (RCA2)", "number_of_queries": 5 }, { "product": "TCF-0110 Data Change Feed", "number_of_queries": 427016 } ] } } } ``` --- ## SpectraDetect API — Manager, Service, Metrics, and Usage Endpoints --- ## SpectraDetect Authentication — LDAP, OAuth 2.0, OpenID Connect, and SAML ## General authentication settings *Administration > Spectra Detect Manager > Authentication* ### Session, cookies and passwords - Duration of login session How long an authenticated user session will remain active on the appliance, set in seconds, minutes, hours or days. Minimum: 1 minute; maximum: 90 days. The default is 7 days. - Session Inactivity Timeout When selected, the session for every logged-in user will expire after the configured period of inactivity in seconds, minutes, hours or days. ------ The remainder of this section describes federated (single sign-on) login options. ## LDAP *Administration > Spectra Detect Manager > Authentication > User Directory: LDAP* ### Connection - LDAP server host Host name or IP address of the server providing LDAP authentication. Example: *ldap.example.com*. Click the *Test* button to verify the connection to the server. - LDAP server port LDAP server host port. Defaults: 389 (LDAP) or 636 (LDAPS). - TLS Select to use a TLS (secure) connection when communicating with the LDAP server. - TLS require certificate Select to require TLS certificate verification when communicating with the LDAP server. - Bind DN or user User to use when logging in to the LDAP server for searches. DN stands for Distinguished Name. Examples: *user@example.com* or *cn=user,dc=example,dc=com* - Password Password for the Bind user account. ### User Schema - Base DN Root node in LDAP from which to search for users. Example: *cn=users,dc=example,dc=com* - Scope Scope of the user directory searches (base, one level, subordinate, subtree). - User Object Class The objectClass value used for when searching users. Example: *user* - User Name Attribute The user name field. Examples: *sAMAccountName* or *cn* ### Group Schema **The majority of fields in this section are the same as in the User Schema section, except the settings relate to groups.** - Group Type LDAP group membership attribute (Member, Unique Member) ### User attribute mapping - First name Field to map to a user’s first name. Example: *givenName* - Last name Field to map to a user’s last name. Example: *sn* - E-mail Field to map to email. Example: *mail* ### User access - Active flag group Group DN. Users will be marked as active only if they belong to this group. Example: *cn=active,ou=users,dc=example,dc=com* - Superuser flag group Group DN. Users will be marked as superusers only if they belong to this group. Example: *cn=admins,ou=groups,dc=example,dc=com* - Require group Group DN. Authentication will fail for any user that does not belong to this group. Example: *cn=enabled,ou=groups,dc=example,dc=com* - Deny group Group DN. Authentication will fail for any user that belongs to this group. Example: *cn=disabled,ou=groups,dc=example,dc=com* ### Select TLS CA Certificate file - Select a file to upload The dialog that opens when clicking **Choose File** allows the user to upload their own TLS certificate for verifying the LDAP host identity. The certificate must be in PEM file format. To apply the certificate, the options *TLS* and *TLS require certificate* must be enabled. It is also possible to upload certificates through the Central Configuration Management section on Spectra Detect Manager, if the appliance is connected and authorized on the Manager. ## OAuth 2.0 / OpenID Connect 👉 Described in the [OpenID guide](/General/SecurityAndAccessControl/OpenID). ## SAML 👉 Described in the [SAML guide](/General/SecurityAndAccessControl/SAML). --- ## SpectraDetect Email Alerting — Quota Notifications and Alert Thresholds The Email Alerting page allows users to configure Spectra Intelligence quota usage notifications. Users can enable email alerting, manage recipients, and define alert thresholds based on system quotas. To receive email alerts, users must configure Spectra Intelligence via **Central Configuration**. - **Enable Email Alerting**: Select this checkbox to enable sending system alerts via email. Once enabled, any alerts generated by the system, such as quota warnings or system issues, will be distributed to the specified recipients. - **Add All Users to the Recipients List**: Select this checkbox to add all system users to the recipients list for email alerts. - **Recipients**: Enter the email addresses of the individuals who should receive system alerts. Multiple email addresses can be added by pressing the Enter key after each one. If the **Add All Users to the Recipients List** option is checked, additional recipients cannot be manually added. - **Alert when quota has reached**: Choose the quota level at which email alerts will be triggered. You can select one or more thresholds. The available options are: `50%`, `75%`, and `90%`. - **Alert when quota has been exceeded**: Select this option to trigger an alert when the system's quota has been exceeded. --- ## SpectraDetect Manager Settings — Network, SSL, and System Configuration These are the general steps to configuring a new Manager: 1. Deploy the appliance and attach it to the network. 2. Configure network settings via the console to access the Web UI. 3. Configure installation-specific settings on the system configuration screen. 4. License the Manager ## Network Ports The Manager supports the following ports for inbound connections: - 80/TCP and 443/TCP for connecting to the Manager Web UI. - 22/TCP for maintenance purposes. - 161/UDP for SNMP monitoring Outgoing connections to the internet via the following ports are also supported: - 53/UDP for DNS - 123/UDP for NTP However, it is strongly recommended that the users configure the system to use their own DNS and NTP infrastructure (if necessary). For outgoing connections to the [Spectra Intelligence](/SpectraIntelligence/) database at `https://appliance-api.reversinglabs.com`, the destination port is 443/TCP. The DNS name is `appliance-api.reversinglabs.com` and the connection supports HTTPS only. ## Configuration via the Manager Web Interface After logging in, access the **Administration ‣ Spectra Detect Manager** page from the main Manager menu. The page contains dialogs with options for configuring the Manager. When done updating the settings in the configuration dialogs, click **Save**. The appliance will be restarted and begin using the new settings. ### General | | | | ----------------------------------- | ------------------------------------------------------------ | | **Network settings** | | | Application URL | The URL used to access the Web UI of the Manager. **The application URL must be configured to use the HTTPS protocol.** | | Allowed hosts | A list of strings, one per line, representing the host/domain names that this appliance installation can serve. Values in this list can be fully qualified names (e.g., "www.example.com"), in which case they will be matched against the request’s host header exactly (case-insensitive, not including port). A value beginning with a period can be used as a subdomain wildcard: ".example.com" will match "example.com", "www.example.com", and any other subdomain of "example.com". A value of "*" will match anything. Examples: *.reversinglabs.com, 89.201.174.154, 89.201.174.152*. **If the Spectra Detect Manager is configured as a redundant cluster, this list must include the hostname of the proxy server.** | | Select SSL certificate | Clicking **Browse** allows the user to upload a file containing a custom SSL certificate to replace the self-signed certificate generated by the Manager. | | Select SSL certificate key | Clicking **Browse** allows the user to upload a file containing the key that corresponds to the certificate uploaded above. | | **Synchronization** | | | Enable YARA ruleset synchronization | Select the checkbox to allow synchronizing YARA rulesets between the appliances connected to the Manager. This global setting affects all Spectra Analyze and Spectra Detect Worker appliances. For this functionality to work, YARA synchronization must also be enabled on connected Spectra Analyze appliances. See the [YARA Sync Page](../Config/YARASync.md) section for more details. | | **SSH** | | | Permit root SSH login | Select the checkbox to allow root SSH access to the Manager. This setting can be used for automated password management. | | **SWAP** | | | Disable SWAP memory | Checking this option will disable the usage of SWAP memory. Not applicable if appliance is deployed as a Docker image. Enabled by default. | ### SMTP | | | | ---------------------------- | ------------------------------------------------------------ | | SMTP hostname | The host to use for sending email. For the SMTP service to function properly, this field must not be empty. | | SMTP port | Port of the host used for sending email. For the SMTP service to function properly, this field must not be empty. | | Username; Password | SMTP username and password for authentication. | | Default "from" email address | The email address used by the appliance as the "from" address when sending email (for password resets, error alerts…). | | Use TLS | Select the checkbox to use a secure connection (TLS; Transport Layer Security) when communicating with the SMTP server. | ### SNMP & system alerting | | | | --------------------------------------------- | ------------------------------------------------------------ | | Enable SNMP service | Select the checkbox to enable Simple Network Management Protocol service. | | Community | Enter the name of an SNMP community list for authentication. Community is a list of SNMP clients authorized to make requests. The SNMP service will not function properly if this field is not configured. | | Enable trap sink | Select the checkbox to enable sending SNMP traps to the sink server. Traps are asynchronous, unsolicited SNMP messages sent by the SNMP agent to notify about important events on the appliances. | | Trap community | Enter the SNMP trap community string. If the *Enable SNMP service* and *Enable trap sink* checkboxes are selected, then this field is required. | | Trap sink server | Enter the host name or the IP address of the trap sink server. The sink server is the location to which SNMP traps will be sent. If the *Enable SNMP service* and *Enable trap sink* checkboxes are selected, then this field is required. | | SNMP trap thresholds | A set of configuration fields allowing the user to set the thresholds (values that will trigger an SNMP trap) for supported types of events. Thresholds can be configured for average system load in 1, 5, and 10 minutes (as a percentage), used memory and used disk space (as a percentage). | | **System Alerting** | | | Send system alert messages to syslog server | Select the checkbox to enable sending alerts about the status of critical system services on the connected appliances to the syslog server. | | Host | Host address of the remote syslog server to send alerts to. | | Port | Port of the remote syslog server. | | Protocol | Communication protocol to use when sending alerts to a remote syslog server. Options are TCP (default) and UDP. | | Enable audit logs to be sent to syslog server | Audit logs will be automatically sent to the syslog server in addition to other system messages. Enabling this will increase the traffic between the Manager and the syslog server. | #### Spectra Detect MIB Descriptions | MIB Module | Value | OID | Description | |----------------------|------------------------------|--------------------------------------|------------------------------------------------------------------| | RL-MIB | `device` | .1.3.6.1.4.1.48699.1.1 | | | RL-MIB | `tiscalehub` | .1.3.6.1.4.1.48699.1.1.6 | | | RL-MIB | `c1000` | .1.3.6.1.4.1.48699.1.1.3 | | | RL-MIB | `tiscaleworker` | .1.3.6.1.4.1.48699.1.1.5 | | | RL-TISCALEHUB-MIB | `tshMib` | .1.3.6.1.4.1.48699.1.1.6.1 | | | RL-TISCALEHUB-MIB | `tshMibObjects` | .1.3.6.1.4.1.48699.1.1.6.1.1 | | | RL-TISCALEHUB-MIB | `tshScalars` | .1.3.6.1.4.1.48699.1.1.6.1.1.1 | | | RL-TISCALEHUB-MIB | `tshVersionString` | .1.3.6.1.4.1.48699.1.1.6.1.1.1.1 | Product version string. | | RL-TISCALEHUB-MIB | `tshQueues` | .1.3.6.1.4.1.48699.1.1.6.1.1.2 | | | RL-TISCALEHUB-MIB | `tshPostFixQueue` | .1.3.6.1.4.1.48699.1.1.6.1.1.2.1 | | | RL-TISCALEHUB-MIB | `tshPFQueueName` | .1.3.6.1.4.1.48699.1.1.6.1.1.2.1.1 | Queue name. | | RL-TISCALEHUB-MIB | `tshPFQueueSize` | .1.3.6.1.4.1.48699.1.1.6.1.1.2.1.2 | Queue size. | | RL-TISCALEHUB-MIB | `tshPFQueueStatus` | .1.3.6.1.4.1.48699.1.1.6.1.1.2.1.3 | Queue status. | | RL-TISCALEHUB-MIB | `tshMibNotifications` | .1.3.6.1.4.1.48699.1.1.6.1.2 | | | RL-TISCALEHUB-MIB | `tshPFQueueThreshold` | .1.3.6.1.4.1.48699.1.1.6.1.2.1 | Queue size status based on configured threshold. | | RL-TISCALEHUB-MIB | `tshMibConformance` | .1.3.6.1.4.1.48699.1.1.6.1.3 | | | RL-TISCALEWORKER-MIB | `tswMib` | .1.3.6.1.4.1.48699.1.1.5.1 | | | RL-TISCALEWORKER-MIB | `tswMibObjects` | .1.3.6.1.4.1.48699.1.1.5.1.1 | | | RL-TISCALEWORKER-MIB | `tswScalars` | .1.3.6.1.4.1.48699.1.1.5.1.1.1 | | | RL-TISCALEWORKER-MIB | `tswVersionString` | .1.3.6.1.4.1.48699.1.1.5.1.1.1.1 | Product version string. | | RL-TISCALEWORKER-MIB | `tswQueueLength` | .1.3.6.1.4.1.48699.1.1.5.1.1.1.2 | Current input queue length. | | RL-TISCALEWORKER-MIB | `tswTables` | .1.3.6.1.4.1.48699.1.1.5.1.1.2 | | | RL-TISCALEWORKER-MIB | `tswStatsNumber` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.1 | The number of entries in tswStatsTable. | | RL-TISCALEWORKER-MIB | `tswStatsTable` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2 | A list of statistics entries. | | RL-TISCALEWORKER-MIB | `tswStatsEntry` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1 | An entry containing statistics information. | | RL-TISCALEWORKER-MIB | `tswStatsIndex` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.1 | A unique value, greater than zero, for each statistics interval. | | RL-TISCALEWORKER-MIB | `tswStatsDesc` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.2 | Description of a statistics interval. | | RL-TISCALEWORKER-MIB | `tswStatsFilesSubmitted` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.3 | Number of files submitted for processing. | | RL-TISCALEWORKER-MIB | `tswStatsFilesProcessed` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.4 | Number of files sucessfully processed. | | RL-TISCALEWORKER-MIB | `tswStatsBytesSubmitted` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.5 | Total size of files submitted for processing. | | RL-TISCALEWORKER-MIB | `tswStatsBytesProcessed` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.6 | Total size of files sucessfully processed. | | RL-TISCALEWORKER-MIB | `tswStatsFilesUnpacked` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.7 | Number of files unpacked. | | RL-TISCALEWORKER-MIB | `tswStatsBytesUnpacked` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.8 | Total size of files unpacked. | | RL-TISCALEWORKER-MIB | `tswStatsMeanQueueSize` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.9 | Mean input queue size. | | RL-TISCALEWORKER-MIB | `tswQueueNumber` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.3 | The number of entries in tswQueueTable. | | RL-TISCALEWORKER-MIB | `tswQueueTable` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4 | A list of queues. | | RL-TISCALEWORKER-MIB | `tswQueueEntry` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1 | An entry containing queue information. | | RL-TISCALEWORKER-MIB | `tswQueueIndex` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1.1 | A unique value, greater than zero, for each queue. | | RL-TISCALEWORKER-MIB | `tswQueueName` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1.2 | Queue name. | | RL-TISCALEWORKER-MIB | `tswQueueSize` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1.3 | Queue size. | | RL-TISCALEWORKER-MIB | `tswQueueStatus` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1.4 | Queue status. | | RL-TISCALEWORKER-MIB | `tswCPUNumber` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.5 | The number of entries in tswCPUTable. | | RL-TISCALEWORKER-MIB | `tswCPUTable` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6 | A list of CPU entries. | | RL-TISCALEWORKER-MIB | `tswCPUEntry` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1 | An entry containing CPU information. | | RL-TISCALEWORKER-MIB | `tswCPUIndex` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.1 | A unique value, greater than zero, for each CPU entry. | | RL-TISCALEWORKER-MIB | `tswCPUCores` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.2 | Number of CPU cores. | | RL-TISCALEWORKER-MIB | `tswCPUCoresStatus` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.3 | Status of CPU cores. | | RL-TISCALEWORKER-MIB | `tswCPUUtilization` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.4 | CPU utilization. | | RL-TISCALEWORKER-MIB | `tswCPUUtilizationStatus` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.5 | CPU utilization status. | | RL-TISCALEWORKER-MIB | `tswMibNotifications` | .1.3.6.1.4.1.48699.1.1.5.1.2 | | | RL-TISCALEWORKER-MIB | `tswQueueThreshold` | .1.3.6.1.4.1.48699.1.1.5.1.2.1 | Queue size status based on configured threshold. | | RL-TISCALEWORKER-MIB | `tswCPUCoreThreshold` | .1.3.6.1.4.1.48699.1.1.5.1.2.2 | CPU core status change notification. | | RL-TISCALEWORKER-MIB | `tswCPUUtilizationThreshold` | .1.3.6.1.4.1.48699.1.1.5.1.2.3 | CPU utilization status change notification. | | RL-TISCALEWORKER-MIB | `tswMibConformance` | .1.3.6.1.4.1.48699.1.1.5.1.3 | | | RL-CM-MIB | `cmMib` | .1.3.6.1.4.1.48699.1.1.3.1 | | | RL-CM-MIB | `cmMibObjects` | .1.3.6.1.4.1.48699.1.1.3.1.1 | | | RL-CM-MIB | `cmScalars` | .1.3.6.1.4.1.48699.1.1.3.1.1.1 | | | RL-CM-MIB | `cmMibNotifications` | .1.3.6.1.4.1.48699.1.1.3.1.2 | | | RL-CM-MIB | `redundancyTrigger` | .1.3.6.1.4.1.48699.1.1.3.1.2.2 | Failover on HA system | | RL-CM-MIB | `redundancyTriggerOk` | .1.3.6.1.4.1.48699.1.1.3.1.2.3 | HA System resumed operation | | RL-CM-MIB | `cmMibConformance` | .1.3.6.1.4.1.48699.1.1.3.1.3 | | Download the MIB file in CSV format. ### Authentication See the [Authentication](Authentication.md) section. ### Spectra Intelligence | | | | ------------------------------ | ------------------------------------------------------------ | | Enable Spectra Intelligence | Select the checkbox to enable the connection to Spectra Intelligence. Spectra Detect Manager needs to be connected to the Spectra Intelligence cloud in order to automatically retrieve system updates and appliance upgrades. When connected, the Manager polls the cloud once every 60 minutes. | | Username; Password | Username and password for authenticating to Spectra Intelligence. | | Timeout | Specify how long to wait before the Spectra Intelligence connection times out (in seconds; the maximum allowed value is 1000). | | Proxy host | Proxy hostname for routing requests from the appliance to Spectra Intelligence (e.g., 192.168.1.15). | | Proxy port | Proxy port number (e.g., 1080). | | Proxy username; Proxy password | Username and password for proxy authentication. | ### Dashboard configuration - Enable Central Logging Enabling central logging will completely change the home page to show statistics on the number of processed files and their classifications. This feature is also resource-intensive. Ensure at least 32 GB RAM and 1 TB disk for optimal performance. - Retention period How long to keep the collected logs on the Manager. - Enable Central File Storage Enables file storage on the Manager. If enabled, connected Workers will store samples on the Manager. Stored samples can later be analyzed with Spectra Analyze by clicking on "Analyze with Spectra Analyze" on the analytics page. Enabling this feature may require additional disk space. The required storage depends on the size of the samples coming from the connected Workers and their retention period. Samples larger than the file limit threshold will not be stored. - File Size Limit File size limit in MiB. Samples larger than the set threshold will not be stored. The default is 400, the maximum supported file size on Spectra Analyze. - Sample Retention Period Time, in hours, after which the uploaded samples will be removed from the Central File Storage. - Minimum Disk Space The minimum allowed free disk space in GiB. If the remaining disk space is below the configured threshold, new sample uploads will be rejected. For example, to use 900 GiB of space for central file storage on a 1000 GiB disk, set the value to 100. - Enable Deep Cloud Analysis Enabling Multi-Scanning instructs Workers to upload samples to the Cloud using their respective account and usage quota. Samples are uploaded only if they pass the filtering criteria: up to 2GB in size. If a sample already exists in the Cloud, the Manager monitors data changes in the data change feed and updates the dashboard accordingly. Enabling this feature impacts the final verdict - classification, risk score and threat name, resulting in increased detection rate and reduced remediation time. Additionally, up to 5 antivirus engine scanners can be selected to be listed on the dashboard. Deep Cloud Analysis parameters can be configured in the **Central Configuration > [Spectra Intelligence](../Config/ApplianceConfiguration.md#spectra-intelligence)** section under the **Deep Cloud Analysis** tab. For more information on the Deep Cloud Analysis pipeline, refer to the [Deep Cloud Analysis Pipeline](../Usage/Analysis.md#deep-cloud-analysis-pipeline-diagram) diagram. - Subscribe to Cloud Classification Changes Receive notifications about cloud classification changes from Spectra Intelligence directly on the Spectra Detect Dashboard. To configure additional alerts via UI, email, splunk, or syslog, navigate to the [Notifications](../Config/Notifications.md) section. Even if the *Scan Unpacked Files* option in the [Spectra Intelligence configuration](../Config/ApplianceConfiguration.md#spectra-intelligence) section is enabled, the notifications about cloud classification changes will be received only for top-level containers. ### System time - Enable network time synchronization Select the checkbox to enable clock synchronization via NTP (Network Time Protocol). - Timezone Select the timezone from the dropdown list. - NTP servers A list of server addresses, separated by a new line, to use for system clock synchronization. Click **Test connection** to verify that time synchronization functions properly. ### System Alerting If system alerting is enabled in the *System Alerting* configuration dialog, the following system operations and services will be monitored. Syslog notifications are sent when any of the services or operations meet the condition(s) defined in the table. | SYSTEM OPERATION OR SERVICE | NOTIFICATION TRIGGER | | ------------------------------- | ------------------------------------- | | RAM | usage is over 90% for 10 minutes | | CPU | usage is over 40% for 2 minutes | | CPU wait (waiting for IO) | over 20% for 2 minutes | | Disk usage | over 90% for 10 minutes | | UWSGI service | down for 2 minutes | | NGINX service | down for 2 minutes | | RABBIT-MQ service | down for 2 minutes | | POSTGRES service | down for 2 minutes | | MEMCACHED service | down for 2 minutes | | CROND service | down for 2 minutes | | SSHD service | down for 2 minutes | | SUPERVISORD service | down for 2 minutes | | SMTP | if enabled, but stopped for 4 minutes | | NTPD | if enabled, but stopped for 4 minutes | | Any of the SUPERVISORD services | if it has crashed | #### SNMP Trap Thresholds The Manager can receive notifications (traps) about important system events via the Simple Network Management Protocol (SNMP). The events are "trapped" and sent to the trap sink server when their configured threshold levels are triggered. The Manager uses the **DISMAN-EVENT-MIB::mteTriggerFired** SNMP trap and supports 3 different triggers. These triggers can be used to keep track of low disk space, high memory usage or high CPU load average over time. | TRIGGER IDENTIFIER | TRIGGER CONDITION | | -------------------------------------------------------- | ------------------------------------------------------------ | | `DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: dskTable` | disk usage is higher than the configured threshold (the default value is 90%) | | `DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: memoryFree` | memory usage is higher than the set threshold (the default value is 80%) | | `DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: laTable` | average system load in the specified time frame (1, 5 or 15 minutes) is higher than the set threshold | To enable SNMP traps and configure the address of the trap sink server, adjust the values in the **Settings ‣ Configuration ‣ SNMP & System Alerting** dialog on the Manager. The dialog also allows setting thresholds for supported types of events, which are described in more detail below. **Average system load** This trap is sent if the average load of the local system exceeds specified values (1-minute, 5-minute and 15-minute averages). Values should be provided as percentages, which are recalculated into appropriate thresholds as reported with uptime or top commands. The following examples show traps triggered by a high 1-minute, 5-minute and 15-minute system load average, respectively: ``` 2018-01-26 14:35:54 --- ## SpectraDetect Appendix — Reference Materials and Open Source Licenses --- ## SpectraDetect Analysis Input — Connector Configuration for Email and S3 **Connectors** allow users to automatically retrieve a large number of files from external sources such as email or S3 buckets. Users can configure this service from the Appliance status page. Select a connected Hub, then select the **Connectors** button. **Note: The Hub must belong to a Hub group with at least one Worker.** If a connector is disabled or if it has not been previously configured on the appliance, the dialog contains only the **Enable connector** button. Click the button to start configuring the connector. Never use the same folder for both input and output of files. ## Starting the Connector When the configuration is complete, click **Start connector** at the bottom of the page. This will initiate the connector service on the appliance. After starting, the connector service connects to the configured user account, automatically retrieves files from it, and submits them for analysis on the appliance. Each file retrieved via the connector has a set of User Tags automatically assigned to it. Those User Tags are based on the file metadata, and can contain information about the file source, the last modification time in the original location, file permissions, email subject, recipient and sender addresses, and more. If advanced options are not enabled, the connector service will not perform any additional actions on the files retrieved from the server after the appliance finishes analyzing them. The users can see the analysis results for each file on its Sample Details page. After providing the required information, click **Test connection** to verify that the appliance can access the configured service storage. When the button is clicked, the appliance attempts to connect and mount the service storage. To remove all configured settings for the current service storage, click **Remove item**. To add another service storage, click **Add item**. Up to five units of storage can be added this way. If there are already five units of storage connected to the appliance, at least one must be removed by clicking **Remove item** before adding a new one. Note that for S3 connector, this limit is 20. ## Pausing and Disabling the Connector While the connector service is active, the *Start connector* button changes into **Pause connector**. Clicking this button temporarily halts the connector service. The connector service records the last state and is able to resume scanning when **Start connector** is clicked again. While the connector is running, it is possible to modify its configuration and save it by clicking **Save changes** without having to pause or disable the connector. If the connector service is active during a scheduled or manually executed Purge action, the system will automatically stop the service before performing the Purge action, and start it after the Purge action is complete. To disable the entire connector service on the appliance, click **Disable connector** at the bottom of the page. When the connector is disabled, it will not be possible to reconfigure, start, or pause it until the service is enabled again. Note that the current connector configuration will be preserved and restored when the service is enabled again. Likewise, all files that have been retrieved and analyzed by Spectra Analyze will remain on the appliance. All files retrieved from the server and analyzed on the appliance are accessible to Spectra Analyze users from the Submissions page. They are distinguished from other files by a unique username specific for each connector. Spectra Detect Workers will follow the default data retention policy. All processed files are deleted immediately after processing. If a file is queued but not processed within 9 hours, the processing task will be canceled (and the file deleted) but the record of the unsuccessful task will still be present in the database for 24 hours. All file processing results are retained until deleted by the user, or for 9 hours after processing (whichever comes first). ## IMAP - MS Exchange - AbuseBox Connector The IMAP - MS Exchange AbuseBox connector allows connecting to a Microsoft Exchange server and analyzing retrieved emails on the appliance. ### Requirements - IMAP must be enabled on the Exchange server. - A new user account must be configured on the mail server and its credentials provided to the connector in the configuration dialog. - A dedicated email folder must be created in the Exchange user account, and its name provided to the connector in the configuration dialog. All emails forwarded to that folder are collected by the connector and automatically sent to the appliance for analysis. When the analysis is complete, email samples with detected threats will get classified as malicious and, if the automatic message filing option is enabled, moved to the specified *Malware* folder. Emails with no detected malicious content do not get classified. They can optionally be moved to the specified *Unknown* folder on the configured Exchange user account. To improve performance and minimize processing delays on Spectra Analyze, each email sample will get analyzed and classified only once. When the *Automatic message filing* option is enabled, each email sample is moved only once, based on its first available classification. Because of that, it is recommended to enable classification propagation and allow retrieving [Spectra Intelligence](/SpectraIntelligence/) classification information during sample analysis instead of after. Administrators can enable these two options in the **Administration ‣ Configuration ‣ Spectra Detect Processing Settings** dialog. This will improve classification of emails with malicious attachments. Workers do this by default and no configuration is necessary. The connector can be configured to automatically sort emails after analysis into user-defined email folders in the configured Exchange user account. ### Configuring the Exchange user account To configure the connection with the Exchange user account: - make sure the connector is enabled - fill in the fields in the *Exchange setup* section of the Email AbuseBox Connector dialog. | | | | | ---------------- | --------- | ------------------------------------------------------------ | | Server domain | Mandatory | Enter the domain or IP address of the Exchange server. The value should be FQDN, hostname or IP. This should not include the protocol (e.g., http) | | Email folder | Mandatory | Enter the name of the email folder from which the email messages will be collected for analysis. This folder must belong to the same Exchange user account for which the credentials are configured in this section. The folder name is case-sensitive. | | Connection Type | Mandatory | Supports IMAP (Basic Authentication) and Exchange (OAuth 2) methods of authentication. Depending on the selection, the next section of the form will ask for different user credentials: Basic Authentication asks for a username and password, OAuth 2 asks for a Client ID, Client Secret and Tenant ID. | | Email address | Mandatory | Enter the primary email address of the configured Exchange user account. | | Access Type | Mandatory | Delegate is used in environments where there’s a one-to-one relationship between users. Impersonation is used in environments where a single account needs to access many accounts. | | Connect securely | Optional | If selected, the connector will not accept connections to Exchange servers with untrusted or expired certificates. | Note that the connector will operate and analyze emails even if these advanced options are disabled. They only control the post-analysis activities. ## Network File Share Connector The Network File Share connector allows connecting up to five shared network resources to the appliance. Once the network shares are connected and mounted to the appliance, it can automatically scan the network shares and submit files for analysis. After analyzing the files, the appliance can optionally sort the files into folders on the network share based on their classification status. The Network File Share connector supports SMB and NFS file sharing protocols. Currently, it is not possible to assign a custom name to each network share. The only way to distinguish between configured network shares is to look at their addresses. If there are 3 configured network shares, and the network share 2 is removed, the previous network share 3 will automatically "move up" in the list and become network share 2. ### Configuring Network Shares To add a new network share to the appliance, expand the *Shares* section in the Network File Share Connector dialog and fill in the relevant fields. | | | | | ------------ | ------------------ | ------------------------------------------------------------ | | Address | Mandatory | Enter the address of the shared network resource that will be mounted to the appliance. The address must include the protocol (SMB or NFS). Leading slashes are not required for NFS shares (example: *nfs:storage.example.lan*). The address can point to the entire network drive, or to a specific folder (example: *smb://storage.example.lan/samples/collection*). When the input folder and/or sorting folders are configured, their paths are treated as relative to the address configured here. **Note:** If the address contains special characters, it may not be possible to mount the share to the appliance. The comma character cannot be used in the address for SMB shares. Some combinations of ? and # will result in errors when attempting to mount both the SMB and the NFS shares. | | Username | Optional, SMB only | Enter the username for authenticating to the SMB network share (if required). Usernames and passwords for SMB authentication can only use a limited range of characters (ASCII-printable characters excluding the comma). | | Password | Optional, SMB only | Enter the password for authenticating to the SMB network share (if required). Usernames and passwords for SMB authentication can only use a limited range of characters (ASCII-printable characters excluding the comma). | | Input folder | Optional | Specify the path to the folder on the network share containing the files to be analyzed by the appliance. The folder must exist on the network share. The path specified here is relative to the root (address of the network file share). If the input folder is not configured, the root is treated as the input folder. | ### Folder Naming Restrictions Folder names used in the **Address**, **Input folder** and **Automatic File Sorting** folder fields (described below) can include: - Alphanumeric characters (`A–Z`, `a–z`, `0–9`) - Spaces - The following special characters: `_`, `-`, `.`, `(`, `)` To ensure maximum compatibility across network file systems, avoid using special characters beyond underscore and hyphen. The following characters are **not allowed**: `/`, `\`, `:`, `*`, `?`, `"`, `<`, `>`, `|`, and the null byte (`\0`). When specifying subfolders in the **Input folder** or within the **Address**, each folder name must conform to these rules. --- The service will continually scan the network shares for new files (approximately every 5 minutes). If any of the existing files on the network share has changed since the last scan, it will be treated as a new file and analyzed again. ## Microsoft Cloud Storage: Azure Data Lake The Azure Data Lake connector allows connecting up to 20 Azure Data Lake Gen2 containers to the appliance. When the containers are connected and mounted to the appliance, it can automatically scan them and submit files for analysis. The files can be placed into the root of each container, or into an optional folder in each of the containers. **Important: This connector is not compatible with containers that have the Blob Soft Delete feature enabled.** After analyzing the files, the appliance can optionally sort the files into folders on the container based on their classification status. Currently, it is not possible to assign a custom name to each data lake input. The only way to distinguish between configured containers is to look at their names. If there are three configured data lake inputs, and input 2 is removed, the previous input 3 will automatically "move up" in the list and become input 2. ### Configuring Azure Data Lake containers To add a new Azure Data Lake container: - make sure the connector is enabled - expand the *Azure Data Lake Inputs* section in the Azure data lake dialog and fill in the relevant fields. | | | | | -------------------- | --------- | ------------------------------------------------------------ | | Storage account name | Mandatory | The name of the storage account. | | Storage access key | Mandatory | The access key used for Shared Key Authentication. This value should end in `==` | | Container | Mandatory | Specify the name of an existing Azure Data Lake container which contains the samples to process. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters. | | Folder | Optional | The input folder inside the specified container which contains the samples to process. All other samples will be ignored. | ## Microsoft Cloud Storage: OneDrive / SharePoint Online The Microsoft Cloud Storage connector allows connecting up to five OneDrive or SharePoint storages to the appliance. When the storages are connected and mounted to the appliance, it can automatically scan them and submit files for analysis. The files can be placed into the root of each storage, or into an optional subfolder, After analyzing the files, the appliance can optionally sort the files into folders based on their classification status. ### Configuring File Storage Inputs To add a new File Storage Input: - make sure the connector is enabled - expand the *File Storage Inputs* section in the Microsoft Cloud Storage dialog and fill in the relevant fields. | | | | --------------- | ------------------------------------------------------------ | | Host | Host of the OAuth2 authentication server. | | Client ID | Identifier value of the OAuth2 client. | | Client Secret | The value used by the service to authenticate to the authorization server. | | Scopes | When the user is signed in, these values dictate the type of permissions Microsoft Cloud Storage connector needs in order to function. Provide one or more OAuth2.0 scopes that should be requested during login. These values should be separated by comma. | | Auth URL | Authentication endpoint for OAuth2. | | Token URL | Link to access token for OAuth2 after the authorization. | | Resource | The server that hosts the needed resources. | | Source | A choice between **OneDrive** and **Online Sharepoint**. | | Drive ID | Identifier value for the drive when **OneDrive** source option is chosen. | | Sharepoint Site | A dropdown which appears when the **Online Sharepoint** source option is chosen. | | Folder | The input folder which contains samples to be processed, while all other samples will be ignored. | ## AWS S3 Connector The S3 connector allows connecting up to 20 S3 buckets to the appliance. When the buckets are connected and mounted to the appliance, it can automatically scan the buckets and submit files for analysis. The files can be placed into the root of each bucket, or into an optional folder in each of the buckets. After analyzing the files, the appliance can optionally sort the files into folders on the S3 bucket based on their classification status. Currently, it is not possible to assign a custom name to each S3 file storage input. The only way to distinguish between configured buckets is to look at their names. If there are 3 configured S3 file storage inputs, and input 2 is removed, the previous input 3 will automatically "move up" in the list and become input 2. To ensure that all files in AWS S3 buckets can be reprocessed, users can use the **Clear All Processed Files** button. This option resets the connector’s tracking of previously processed files. As a result, all files in the buckets will be treated as new and reprocessed when the connector is restarted. No files in the buckets are deleted. ### Configuring S3 Buckets To add a new S3 bucket: - make sure the connector is enabled - expand the *S3 File Storage Inputs* section in the S3 dialog and fill in the relevant fields. | | | | | --------------------------- | --------------------------------------------------- | ------------------------------------------------------------ | | AWS S3 access key ID | Mandatory | The access key ID for AWS S3 account authentication. In cases where the appliance is hosted by ReversingLabs and Role ARN is used, this value will be provided by ReversingLabs. | | AWS S3 secret access key | Mandatory | The secret access key for AWS S3 account authentication. In cases where the appliance is hosted by ReversingLabs and Role ARN is used, this value will be provided by ReversingLabs. | | AWS S3 region | Mandatory | Specify the correct AWS geographical region where the S3 bucket is located. This parameter is ignored for non-AWS setups. | | Enable Role ARN | Optional | Enables or disables authentication using an external AWS role. This allows the customers to use the connector without forwarding their access keys between services. The IAM role which will be used to obtain temporary tokens has to be created for the connector in the AWS console. These temporary tokens allow ingesting files from S3 buckets without using the customer secret access key. If enabled, it will expose more configuration options below. | | Role ARN | Mandatory and visible only if `Role ARN` is enabled | The role ARN created using the external role ID and an Amazon ID. In other words, the ARN which allows the appliance to obtain a temporary token, which then allows it to connect to S3 buckets without using the customer secret access key. | | External ID | Optional, visible only if `Role ARN` is enabled | The external ID of the role that will be assumed. Usually, it’s an ID provided by the entity which uses (but doesn’t own) an S3 bucket. The owner of that bucket takes the external ID and creates an ARN with it. In non-production or test environments, you can enter a placeholder value for the External ID if your use case doesn't require a real one. This is useful when you do not want to enforce the External ID requirement while testing configurations. However, it is strongly recommended to use a valid External ID in production environments to maintain security. | | Role session name | Mandatory and visible only if `Role ARN` is enabled | Name of the session visible in AWS logs. Can be any string. | | ARN token duration | Mandatory and visible only if `Role ARN` is enabled | How long before the authentication token expires and is refreshed. The minimum value is 900 seconds. | | AWS S3 bucket | Mandatory | Specify the name of an existing S3 bucket which contains the samples to process. The bucket name can be between 3 and 63 characters long, and can contain only lower-case characters, numbers, periods, and dashes. Each label in the bucket name must start with a lowercase letter or number. The bucket name cannot contain underscores, end with a dash, have consecutive periods, or use dashes adjacent to periods. The bucket name cannot be formatted as an IP address. | | Processing Priority | Mandatory | Assign a priority for processing files from an S3 bucket on a scale of 1 (highest) to 5 (lowest). Multiple buckets may share the same priority. The default value is 5. | | AWS S3 folder | Optional | The input folder inside the specified bucket which contains the samples to process. All other samples will be ignored. The folder name can be between 3 and 63 characters long, and can contain only lower-case characters, numbers, periods, and dashes. Each label in the folder name must start with a lowercase letter or number. The folder name cannot contain underscores, end with a dash, have consecutive periods, or use dashes adjacent to periods. The folder name cannot be formatted as an IP address. If the folder is not configured, the root of the bucket is treated as the input folder. | | S3 endpoint URL | Optional | Enter a custom S3 endpoint URL. Specifying the protocol is optional. Leave empty if using standard AWS S3. | | Server Side Encryption Type | Optional | Specify the server-side encryption method managed by AWS for your S3 bucket. You can choose either “AES256” to enable Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3) or “aws:kms” to enable Server-Side Encryption with AWS Key Management Service (SSE-KMS). This setting should be left blank unless your bucket policy requires SSE headers to be sent to S3. It is mutually exclusive with the Customer Encryption settings, meaning you should not configure this option alongside Customer Encryption Algorithm or Customer Encryption Key. | | Customer Encryption Algorithm | Optional | Defines the encryption algorithm used when you provide your own encryption keys. The only valid value for this field is “AES256”. It must be used in conjunction with the Customer Encryption Key field and cannot be used simultaneously with the Server Side Encryption Type. This option is intended for users who prefer to manage their own encryption keys rather than relying on AWS-managed keys. | | Customer Encryption Key | Optional | Provide a customer-managed encryption key for encrypting and decrypting objects in your S3 bucket. The key must be a valid Base64-encoded AES256 key. It must be used together with the Customer Encryption Algorithm field and is mutually exclusive with the Server Side Encryption Type. | | Connect securely | Optional | If selected, the connector will not accept connections to S3 buckets with untrusted or expired certificates. This setting only applies when a custom S3 endpoint is used. | | Enable Selection Criteria Using Metadata | Optional | If selected, the connector will fetch and process samples that either match the specified metadata or have no associated metadata. This option can be used to target specific samples for further ingestion and processing via Spectra Analyze. | | Classification | Optional | Specify classifications to ensure that only samples matching the selected classification criteria, or samples that have no associated metadata are considered for processing. The available classifications are `Unknown`, `Goodware`, `Suspicious`, `Malicious`. | | Threat Name | Optional | Specify only the samples containing any of the specified threat names in their metadata for further processing. Multiple threat names can be specified using the enter or tab key. | ## SMTP Connector The SMTP connector allows analyzing incoming email traffic on the appliance to protect users from malicious content. When enabled, the connector service collects emails (with attachments) and uploads them to the appliance for analysis. Each email message is saved as one file. If email uploading fails for any reason, the connector automatically retries to upload it to the appliance. When the analysis is complete, each email message receives a classification status from the appliance. In this operating mode, the connector acts as an SMTP Relay. Therefore, the connector should not be used as a front-end service for accepting raw email traffic, but only as a system inside an already established secure processing pipeline for SMTP email. To allow the SMTP connector to inspect and collect email traffic, users must ensure that the SMTP traffic in their network is diverted to port 25/TCP prior to configuring the connector on the appliance. **Warning: Additional port configuration may be required on the appliance. Because it involves manually modifying configuration files, this action can cause the appliance to malfunction. Contact [ReversingLabs Support](mailto:support@reversinglabs.com) for instructions and guidance.** **Profiles** There are two profiles for this connector: *Default* and *Strict*. These two profiles correspond to different Postfix configuration files. Clicking the underlined **See How It Affects Postfix Config** text will display a pop-up modal with a detailed *Default* and *Strict* profile Postfix configuration. In the *Default* profile, you don’t enforce TLS traffic and you accept any SMTP client. This corresponds to the following Postfix configuration: ``` mynetworks = 0.0.0.0/0 [::]/0 smtpd_tls_security_level = may smtp_tls_security_level = may ``` In the *Strict* profile, you do enforce TLS and you can also specify trusted SMTP clients (highlighted line 1 in the example below; see [Postfix docs](https://www.postfix.org/postconf.5.html#mynetworks) for the specific syntax). The relevant portion of the configuration looks like this in *Strict* mode: ``` mynetworks = 0.0.0.0/0 [::]/0 smtpd_tls_security_level = encrypt smtp_tls_security_level = encrypt smtpd_tls_mandatory_ciphers = high smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5 ``` **Starting the connector** After the connector is enabled, click the **Start connector** button. This will initiate the connector service on the appliance. **Pausing and disabling the connector** While the connector service is active, the *Start connector* button changes into **Pause connector**. Clicking this button temporarily halts the connector service, which in turn stops receiving and analyzing new email traffic. The connector service records the last state and is able to resume scanning when **Start connector** is clicked again. If the connector service is active during a scheduled or manually executed Purge action, the system will automatically stop the service before performing the Purge action, and start it after the Purge action is complete. To disable the entire connector service on the appliance, click **Disable connector** at the bottom of the page. When the connector is disabled, it will not be possible to reconfigure, start, or pause it until the service is enabled again. ### Use Case: Accept Email from Any Email Client or Server To configure the service to accept email from any email client or server, follow these steps: 1. **Enable the SMTP connector** 2. **Contact [ReversingLabs Support](mailto:support@reversinglabs.com)** with: - Your intended email forwarding address range - A formal request to enable email forwarding 3. After receiving your request, ReversingLabs will: - Create an MX DNS record for your hosted service - Configure receiver restrictions to accept email only from your domain 4. Once ReversingLabs has added the MX record to the DNS for your hosted service, **create email forwarding rules** in your email system to automatically forward emails to the hosted service for processing. ## Citrix ShareFile Connector The Citrix ShareFile connector allows integration with ShareFile to scan and classify files stored on the platform. It supports advanced options for sorting files based on classification (`Goodware`, `Malware`, `Suspicious`, `Unknown`) into designated folders on ShareFile. Additionally, users can enable automatic deletion of source files post-analysis. ### Configuring Citrix ShareFile To add a new Citrix ShareFile input: - make sure the connector is enabled. - expand the *Citrix ShareFile Inputs* section in the Citrix ShareFile dialog and fill in the relevant fields. | Input | Description | |---------------|---------------------------------------------------------------------------------------------------------------------------------------------| | Hostname | The hostname used to access ShareFile servers. Usually `.sharefile.com`. | | Client ID | Identifier value of the OAuth2 client. | | Client Secret | The value used by the service to authenticate to the authorization server. | | Username | The username for authenticating to the ShareFile servers. | | Password | The password for authenticating to the ShareFile servers. | | Root Folder | Root folder used for scanning, needs to be defined as GUID/Item Id in format `foxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`, and available on Citrix ShareFile. If left empty, the connector will scan all the shared folders assigned to the user account, including the account's home folder, if it has one. | ## ICAP Server **The ICAP Connector acts as an ICAP server** and processes files sent through ICAP-enabled systems. It applies rules based on classifications, file size, and processing time, ensuring files meet the specified criteria before being analyzed by Spectra Detect. ### Supported Topologies The ICAP Server Connector is designed to integrate seamlessly in any ICAP deployment model, including but not limited to: - Application Delivery Controllers (ADCs) - Forward Proxy - Ingress Controller - Intrusion Prevention System - Load Balancer - Managed File Transfer - Next-Generation Firewall - Protecting Enterprise Storage - Reverse Proxies - Secure Remote Access - SSL Inspection and Termination - Web Application Firewall (WAF) - Web Gateway - Web Traffic Security ### Configuration Guides - [Spectra Detect integration with Kiteworks](/Integrations/ICAP/kiteworks/). ### Request and Response Modes The ICAP Server Connector supports two workflows: request mode (REQMOD) and response mode (RESPMOD). Configure your ICAP client to use one mode depending on your requirements: #### Request Mode (REQMOD) - Processes outgoing client requests before they reach the destination server. - Example: Validating uploaded files to ensure they meet security requirements. #### Response Mode (RESPMOD) - Processes server responses before they are delivered to the client. - Example: Scanning downloaded files for malware or sanitizing content before delivery. ### ICAP Response Codes The table below lists the ICAP response codes and their meanings: | ICAP Response Code | Description | |--------------------|-------------| | **100 Continue** | Signals the client to continue sending data to the ICAP server after the ICAP Preview message. | | **200 OK** | The request was successfully processed. This status is returned whether the file is blocked or not. If the file was blocked, the `X-Blocked` header will be set to `True`. In RESPMOD, a blocked file will result in a 403 HTTP status code. | | **204 No modification needed** | Returned when the file was not blocked and the client sent an `Allow: 204` header in the request. | | **400 Bad request** | Indicates a failure during file submission, analysis processing, or delivering results back to the client. | | **404 Not found** | The ICAP service could not be found. | | **405 Method Not Allowed** | The requested method is not supported on the endpoint. | | **500 Internal Server Error** | A generic server-side error occurred. | ### Configuring ICAP To configure the ICAP Server connector, ensure that it is enabled and adjust the settings in the ICAP Server configuration section. | | | | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Max File Size | Specify the maximum file size (in megabytes) that the ICAP Connector will process. Files exceeding this size will not be analyzed. Default: `0 (unlimited)`. | | Allow Classifications | Select which classifications to allow. Available options: `goodware`, `unknown`, `suspicious`, `malicious`. Files not matching the selected classifications will be blocked. | | Service Alias | The base service name is `spectraconnector` and cannot be changed. Here you may define any additional aliases your ICAP client can use to connect. Do not re-enter `spectraconnector` here. | | Timeout | Set the timeout period (in seconds) for processing requests. If a file is not processed within this time, the request will be terminated. Default: `300`. | | REQMOD Block Page URL | Enter the full URL your ICAP clients will fetch when a request is blocked. **To use the default block page, set it to `https://{RL_APPLIANCE_IP}/icap-block-page` and replace `{RL_APPLIANCE_IP}` with your appliance’s actual IP or hostname.** If you host a custom block page, enter its URL here and ensure it's accessible to ICAP clients. **Note:** In some deployments, `RL_APPLIANCE_IP` may refer to a virtual IP or a load-balancer IP. | | RESPMOD Block Page | Upload a custom file that will replace the blocked HTTP response content. This page will be served to the client instead of the original response. **Maximum file size:** 0.5 MB. | | Use TLS | Select the checkbox to use a secure connection (TLS v1.3) when communicating with the ICAP server. | | ICAP server listen port | Specify the network port on which the ICAP server listens for incoming requests. The default is `11344` when TLS is enabled, and `1344` when TLS is disabled. | | Scan Raw Data | Enable this to extract the raw HTTP message body and send it to the ReversingLabs analysis engine unmodified. | ## SFTP Connector The SFTP Connector allows the appliance to automatically retrieve files from a remote SFTP server for analysis. Users can configure authentication using either a password or a public key. Once started, the connector continuously scans the specified folder and submits new files for analysis. Classification results can be viewed on the appliance after processing. | | | |---------------------|---------------------------------------------------------------------------------------------------| | Host/Server Address | Hostname or IP address of the target SFTP server. | | Port | Port used for the SFTP connection. Defaults to `22`. | | Username | Account name used to authenticate with the SFTP server. | | Authentication Type | Method used to authenticated with the SFTP server. Supported methods: `Password` or `Public Key`. | | Password | Password associated with the specified username. | | Input Folder | Path to remote directory from which files will be retrieved. Example: `/incoming/data` | **Tip: Key pairs can also be created using APIs. Refer to the API documentation on the appliance, specifically the **Appliances > Generate key pair** and **Appliances > Download public key** endpoints.** ## Using Advanced Connector Options In addition to main connector options for every connector service, users can set advanced options. Advanced options for a connector refer to actions that the connector service can perform on files after the appliance finishes analyzing them. Specifically, the connector can be configured to automatically sort files into user-defined sorting folders on the connector user account. Files are sorted into folders based on the classification status they receive during analysis (malicious, suspicious, known, no threats found). For Azure Data Lake, Network File Share, and S3 Connectors, advanced options can be configured for every storage unit individually. This means that the sorting criteria, folder names, and folder paths can be different on each configured storage unit. ### IMAP - MS Exchange - AbuseBox Connector | | | | ------------------------------- | ------------------------------------------------------------ | | Enable automatic message filing | Selecting the checkbox will allow the connector to move analyzed emails and sort them into email folders in the configured Exchange email user account. This checkbox toggles the availability of other options in the Advanced Options section. | | Malware folder | Specify the name of the email folder into which the connector will store emails classified as "Malicious" (malware). This folder will be created if it doesn’t exist. This field is mandatory when *Enable automatic message filing* is selected. | | Unknown folder | Specify the name of the email folder into which the connector will store emails with no malicious content detected (classified as Known, or not classified at all = Unknown). This folder will be created if it doesn’t exist. This field is mandatory when *Enable automatic message filing* is selected. | | Allow suspicious | When selected, emails classified as "Suspicious" will be moved to the configured Unknown folder. If this checkbox is not selected, files classified as "Suspicious" will by default be sorted into the configured Malware folder. | ### Azure Data Lake, Microsoft Cloud Storage, S3 Connectors | | | | ------------------------------------------------ | ------------------------------------------------------------ | | Enable Same Hash Rescan | Selecting the checkbox will force the connector to rescan samples that share the same hash. **This checkbox can only be enabled for the S3 connector.** | | Delete source files | Selecting the checkbox will allow the connector to delete source files on the connector storage after they have been processed. | | Enable automatic file sorting | Selecting the checkbox will allow the connector to store analyzed files and sort them into folders based on their classification. Usually, the connector skips already uploaded files. If this option is enabled and some files have already been uploaded, they will be uploaded to the Worker again. | | Sort Malware detected by Microsoft Cloud Storage | If enabled, the samples which are identified as Malware by Microsoft Cloud Storage will be moved to the Malware folder. These samples are not processed by Spectra Detect. **This checkbox can only be enabled for Microsoft Cloud Storage connector.** | | Goodware folder | Specify the path to folder into which the connector will store files classified as "Known" (goodware). This field is mandatory when *Enable automatic file sorting* is selected. The path specified here is relative to the address of the connector storage unit. If the folder doesn’t already exist on the container, it will be automatically created after saving the configuration. | | Malware folder | Specify the path to folder into which the connector will store files classified as "Malicious" (malware). This field is mandatory when *Enable automatic file sorting* is selected. The path specified here is relative to the address of the connector storage unit. If the folder doesn’t already exist on the container, it will be automatically created after saving the configuration. | | Unknown folder | Specify the path to folder into which the connector will store files without classification ("Unknown" status). The path specified here is relative to the address of the connector storage unit. If the folder doesn’t already exist on the container, it will be automatically created after saving the configuration. Files stored in the Unknown folder are regularly rescanned.| | Suspicious folder | Specify the path to folder into which the connector will store files classified as "Suspicious". The path specified here is relative to the address of the connector storage unit. If the folder doesn’t already exist on the container, it will be automatically created after saving the configuration. | ### Network File Share Connector | | | | ----------------------------- | ------------------------------------------------------------ | | Delete source files | Selecting the checkbox will allow the connector to delete source files on the network share after they have been processed. | | Enable automatic file sorting | Selecting the checkbox will allow the connector to store analyzed files and sort them into folders on every configured S3 bucket based on their classification status. This checkbox toggles the availability of other options in the Advanced Options section. | |Rescan Unknowns|If enabled, the connector rescans samples previously classified as unknown in intervals defined by the Rescan Unknowns Interval value (in days). If Allow Unknown is enabled, unknown files will be stored in the Goodware folder, and will not be rescanned.| | Goodware folder | Specify the path to folder into which the connector will store files classified as "Known" (goodware). This field is mandatory when *Enable automatic file sorting* is selected. The path specified here is relative to the address of the network file share. If the folder doesn’t already exist on the network share, it will be automatically created after saving the configuration. | | Malware folder | Specify the path to folder into which the connector will store files classified as "Malicious" (malware). This field is mandatory when *Enable automatic file sorting* is selected. The path specified here is relative to the address of the network file share. If the folder doesn’t already exist on the network share, it will be automatically created after saving the configuration. | | Unknown folder | Specify the path to folder into which the connector will store files without classification ("Unknown" status). The path specified here is relative to the address of the network file share. If this field is left empty, unknown files will be stored either to the Goodware or to the Malware folder, depending on the "Allow unknown" setting. | | Known threshold | Files classified as "Known" (goodware) with the trust factor value higher than the one configured here will be stored into the configured Malware folder. "Known" files with the trust factor less than or equal to the value configured here will be stored into the configured Goodware folder. Supported values are 0 to 5. Default is 5 (saves all to the Goodware folder). This field is mandatory when *Enable automatic file sorting* is selected. | | Allow unknown | When selected, files with the "Unknown" classification status are stored into the configured Goodware folder. If this checkbox is not selected, files with the "Unknown" status are either stored into the Unknown folder (if the Unknown folder is configured), or to the Malware folder (if the Unknown folder is not configured). | | Allow suspicious | When selected, files classified as "Suspicious" will be stored into the configured Goodware folder. If this checkbox is not selected, files classified as "Suspicious" will be stored into the configured Malware folder. | ## Global Configuration In addition to every connector service having specific configuration settings, there is a **Global Configuration** section at the bottom of every connector page. These settings apply to all configured connectors. | | | | -------------------------------------------------------- | ------------------------------------------------------------ | | Save files that had encountered errors during processing | Original files that were not successfully uploaded will be saved to `/data/connectors/connector-[CONNECTOR_SHORTNAME]/error_files/` | | Max upload retries | Number of times the connector will attempt to upload the file to the processing appliance. Upon reaching the number of retries, it will be saved in the `error_files/` destination or be discarded | | Max upload timeout | Period (in seconds) between upload attempts of the sample being re-uploaded. | | Upload algorithm | The algorithm used for managing delays between attempting to reupload the samples. In **Exponential backoff**, the delay is defined by multiplying the *Max upload timeout* parameter by 2, until reaching the maximum value of 5 minutes. **Linear backoff** will always use the *Max upload timeout* value for the timeout period between reuploads. | | Max upload delay | In case the Worker cluster is under high load, this parameter is used to delay any new upload to the cluster. The delay parameter will be multiplied by the internal factor determined by the load on the appliance. If set to 0, the delay will be disabled. | | Database cleanup period | Specifies the number of days for which the data will be preserved. | | Database cleanup interval | Specifies the time (in seconds), in which the database cleanup will be performed. | | Max File Size (MB) | The maximum file size the connector will transfer to the appliance for analysis. Setting it to 0 disables the option. (Available for AWS S3 and Azure Data Lake Connectors) | ## Unique Usernames | Connector | Unique Username | | ---------------------------- | ------------------------- | | Email AbuseBox Connector | abusebox_connector | | Azure Data Lake Connector | azure-data-lake_connector | | Network File Share Connector | fileshare_connector | | Microsoft Cloud Storage | graph-api-connector | | S3 Connector | s3_connector | | SMTP Connector | smtp_connector | | SFTP Connector | sftp_connector | --- ## SpectraDetect Appliance Configuration — Central Configuration and Hub Groups Spectra Detect Manager allows users to modify configuration settings on Spectra Detect appliances directly from the Manager interface. The Central Configuration feature makes it easier to configure appliances remotely, and to ensure that the settings are consistent and correct across multiple appliances. The appliances must first be connected and authorized on the Manager instance. **Note: **Spectra Analyze** appliances can be managed using the Spectra Detect Manager APIs.** To start working with this feature, access the Central Configuration page by selecting **Central Configuration** in the upper right corner of the Manager interface. The Central Configuration page contains different configuration modules that users can enable. Different combinations of modules and their configuration values can be saved as **configuration groups** or **Hub groups**. For example, users can create a configuration group for Worker appliances that should be connected to [Spectra Intelligence](/SpectraIntelligence/), and a Hub group for Hub and Worker appliances that should be connected to T1000. In addition to options described below, appliance groups containing a Hub instance provide more configuration options (such as Connector services) if they are configured as a [Spectra Detect Hub group](./Redundancy.md#hub-groups) rather than a regular configuration group. When appliances are added to a group, all settings configured in the modules are applied to them, overwriting their previous settings. Generally, the Central Configuration workflow includes the following steps: 1. Create a configuration group or edit an existing one. 2. Select which appliances should be in the group. 3. Modify settings in configuration modules. 4. Save changes 5. Apply modified settings to all appliances in the group. ## Central Configuration Page The Central Configuration page contains the *Select configuration group* pull-down list at the top, allowing users to switch between existing groups. There are also buttons for creating a new group and deleting the currently selected group. If there are no configuration groups created on the Manager instance, the `default` group is displayed on the Central Configuration page. Users can manage appliances and modify settings in the default group, or create their own groups. All configuration modules supported by the Manager are listed in the sidebar on the left. Selecting a module opens its configuration dialog on the right side of the page. ![Central Configuration page with configuration group selection and modules sidebar](../images/central-configuration.png) If the selected group is a [Spectra Detect Hub group](./Redundancy.md#hub-groups), an additional section is present at the top of the page. The section indicates which Hub instance in the group is the primary, and which is the fallback node. Clicking **Details** in this section displays more information about both Hub instances, such as their router IDs and configured priority values. To see the list of appliances that can be added to the currently selected configuration group, select *Appliances* in the sidebar. Appliances that are already in the current group have a tick in the checkbox next to their name. Appliances that are in other configuration groups have an indicator next to the appliance name. Users can save and/or apply configurations to appliances in the group by clicking on the **Save** button. This opens a pop-up dialog with the options to **Save** or **Save and Apply** the configuration to all appliances in the group. To apply the configurations to specific appliances, select their checkboxes in the appliance list below, and click the **Apply** button at the top of the list. Note that applying configuration removes any configurations that aren’t defined in the current configuration group, preventing mismatches caused by values from previous groups or manual changes. ### Configuration status The configuration status of appliances can be one of the following: - Applied - Not Applied - Pending - Error - Out of Sync **Note: Older appliances (i.e. before Spectra Detect v5.2) will show different status messages.** ## Adding Appliances to a Configuration Group Appliances that can be added to the current configuration group are listed in the *Appliances* section. Select the checkbox next to the appliance(s) that should be added to the group, and click **Save**. This opens a dialog with the options to save the selected appliances to the group, and to optionally apply the current group configuration. An appliance cannot be in two configuration groups at the same time. If an appliance is already in another configuration group, the Manager displays a warning message after clicking **Save**. Confirming the change will move the appliance from the previous configuration group to the current one. When an appliance is successfully added to a configuration group, the group’s configuration has to be manually applied to it either by clicking the **Save** button and selecting the **Save and Apply** option, or by selecting its checkbox in the Apply Configuration list and clicking the **Apply** button. The appliance will restart and start using the new configuration. The configuration dialogs on the appliance will indicate that the settings are being managed by the Manager. Although the configuration values modified in the group will still be editable in the appliance’s configuration dialogs, any changes saved on the appliance will not be applied as long as the appliance is managed through Central Configuration. If an appliance is added to a group and the configuration is applied, but the appliance is offline or unreachable by the Manager at that time, its settings will be modified when it becomes reachable. **Warning: Moving a Worker between Hub groups can result in configuration mismatches if the Worker’s settings do not align with the target Hub group. Applying configuration changes to the affected Worker, either through the GUI or the API, is expected to resolve the mismatches.** ## Creating Configuration Groups To create a new configuration group, click **Add new group** at the top of the Central Configuration page. **Tip: It’s also possible to create a configuration group by clicking **Add new group** on the Appliance Management tab on the Dashboard page.** Group names are case-sensitive, so "example" and "Example" are treated as two different groups. Supported characters for group names are `a-z`, `A-Z`, `0-9`, and the underscore ( `_` ). If the group name contains an unsupported character, an error message is displayed. Likewise, a warning is displayed when trying to create a configuration group with a name that is already taken by another group. The dialog also requires selecting the group type. Two types are supported: 1. Configuration group (for Spectra Detect Worker appliances without a Hub), 2. Hub group (for [setting up a high-availability cluster](./Redundancy.md#hub-groups)). Select the first type ("Configuration Group") and click **Add** to confirm changes and create a new configuration group. The newly created group will not contain any appliances, and there won’t be any active configuration modules. **Important: Some configuration modules and options apply only to specific appliance types or versions. For example, the "Splunk" configuration module and its options apply only to the Worker. Read more in the [configuration modules](#configuration-modules) section.** To enable a configuration module, select it in the sidebar on the Central Configuration page and modify the options in it. The indicator in the sidebar warns about unsaved changes in the module. Unsaved changes are lost if the user navigates away from the Central Configuration page without clicking **Save** first. Configuration modules that are not enabled do not have any indicators in the sidebar. Those that are enabled and properly configured have a green indicator. If there are issues with the configuration of a module, the indicator changes to red. Save changes in the module by clicking **Save**. The indicator in the sidebar shows whether the module is configured properly. Repeat this procedure for every configuration module that needs to be enabled in the configuration group. To disable a configuration module, select it in the sidebar and click **Reset to Default**. This action removes all current configuration entries and restores the default settings in the UI. The changes are temporary until you **save** and then click **Apply**. Applying pushes the new settings to all connected appliances. The full list of supported configuration modules and options for all appliance types is available in the [configuration modules](#configuration-modules) section. ## Managing Configuration Groups The following changes can be made to any configuration group on the Manager: - enable and disable configuration modules - change options in enabled configuration modules - add and remove appliances from a group - move appliances from one group to another - delete the entire group (does not apply to the *default* group, which cannot be deleted) **Depending on the type of change, appliances may be automatically restarted. Only applying new configurations to an appliance will trigger a restart of that specific appliance. Adding an appliance to a group, removing it from a group, moving it between groups, or deleting a group will not restart the appliances.** Depending on the type of appliance, the process of restarting and reloading configuration values might take some time. Spectra Detect Worker appliances generally take longer to restart. ## Configuration modules The configuration modules listed in this section can be enabled on the Central Configuration page, and their options can be modified. Some configuration modules support all versions of Spectra Detect appliances, but specific options within the modules apply only to specific versions. Such options are indicated by a comment in the Manager interface. ### General Root SSH login can be enabled for use with password management systems. These checkboxes are not available by default. To enable them, do the following: 1. Log in via SSH to the Manager. 2. Run `sudo rlapp configure --sshd-control-enable`. This will enable the checkboxes on the Manager. 3. In the browser, go to *Spectra Detect Manager > Central configuration*, select the Hub group which will have root SSH login enabled, then go to *General > SSH* 4. Enable *Permit SSH configuration* 5. Enable *Permit root SSH login* Note that this can only be applied to Hub groups. For SSH credentials, contact [ReversingLabs Support](mailto:support@reversinglabs.com) . This section also includes the option to disable the use of swap memory. Swap memory is disk space used as RAM. Note that this option isn’t applicable if the appliances are deployed as Docker images. ### [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) This section lists configuration settings related to static analysis performed by Spectra Core. #### Processing Settings This setting determines which file formats will be unpacked by Spectra Core for detailed analysis. "Best" fully processes all formats supported by the appliance. "Fast" processes a limited set of file formats. Fast option does not support unpacking and/or validation of several file formats, providing only minimal information for: - Archives (ActiveMimeMSO, ARC (.arc, .ark), ARSC, BLZ, CGBI, CRTF, DICOM (.dicom, .dcm, .dic), PE .Net Resources, LZ4, LZIP, LZMA, LZOP, MAR, NuGet, PAK, PCAP (http, smtp), PYZ, SQX, TIFF, WARC, XAR, ZOO) - Documents (bplist, Certutil (.crt, .cert, .pem), CHM, HTML (.html, .htm, .xhtml, .xht), IQY, SettingContent (.xml), SYLK, URL) - Mobile Applications (Android (.apk), iOS (.ipa), Windows Phone (.xap), APPX) - Multimedia (BMP, GIF, JPEG, PNG, SWF) - File System/Firmware (cramfs, HFSP, jffs2, squashfs, yaffs) - Web Applications (Google Chrome (.crx), Opera (.oex), Mozilla FireFox (.xpi)) - Quarantine formats (KasperskyKLQ, SymantecQBD, SymantecVBN) - Emails (UUE, YENC) - Disk Images (VHD, WIM (.wim, .swm)) - ...and others (CxMacro, Docker, PyInstaller, SMTP, sqlite3 (.db, .sqlite), VBE(.vbe, .jse)). Additionally, the report metadata will no longer include overlay and resources hashes, storyteller descriptions, Spectra Intelligence XREF data, Mitre ATT&CK mappings, IoC reasons, as well as mobile, browser and media details. #### CEF Messages Configuration Spectra Detect can log events using the Common Event Format (CEF) to ensure compatibility with security information and event management software products (SIEMs). CEF is an extensible, text-based logging and auditing format that uses a standard header and a variable extension, formatted as key-value pairs. Select the checkbox to enable sending CEF messages to a syslog receiver. **CEF Message Hash Type**: The hash type to use for CEF messages. #### String Extraction Configuration **Note: Changing the strings configuration can impact classification.** Spectra Core can extract information from binaries in the form of strings. While useful in some contexts, this metadata can also be very extensive. **Minimum String Length**: The minimum length of extracted strings that make it into the analysis report. Default is 4. **Maximum String Length**: The maximum length of extracted strings that make it into the analysis report. Default is 32768. A value of 0 is interpreted as unlimited length, and can increase processing memory requirements. Additionally, the following options can be enabled: Unicode Printable, UTF-8 Encoding, UTF-16LE Encoding, UTF-16BE Encoding, UTF-32LE Encoding, UTF-32BE Encoding. #### MWP-related settings **MWP goodware factor**: The value configured here determines the threshold at which the KNOWN classification for a file (from the Malware Presence algorithm) will change to the Spectra Core Goodware classification. By default, all KNOWN classifications are converted to Goodware. Lowering the value reduces the number of files classified as goodware. Files with a trust factor higher than the configured value are considered to have no threats. Supported values are 0 - 5. The default is 2. **Extended MWP Metadata**: Select the checkbox to include extended malware presence metadata in Worker analysis reports for files analyzed with AV engines in the Spectra Intelligence system. Spectra Detect Worker must be connected to Spectra Intelligence, and the user account must have appropriate access rights for this feature to work. Note that extended metadata is displayed in the report only for those files that have been analyzed by AV engines at some point. #### Decompression configuration Decimal value between 0 and 999.9. Used to protect the user from intentional or unintentional archive bombs, terminating decompression if size of unpacked content exceeds a set quota. The maximum allowed decompression ratio is calculated as: ``` MaximumDecompressionFactor * (1000 / ln(1 + InputFileSize * pow(10, -5))) ``` The `InputFileSize` must be in bytes. To calculate the maximum decompressed file size, multiply this ratio by the `InputFileSize`. Unpacking will stop once the size of all extracted content exceeds the theoretical maximum of the best-performing compression algorithm. Setting it to 0 will disable decompression management. ReversingLabs recommend against disabling decompression management. #### Propagation When propagation is enabled, files can be classified based on the content extracted from them. This means that files containing a malicious or suspicious file will also be considered malicious or suspicious. Goodware overrides ensure that any files extracted from a parent file and whitelisted by certificate, source or user override can no longer be classified as malicious or suspicious. Additionally, this goodware classification can be propagated from extracted files to their parent files in order to prevent and suppress possible false positives within highly trusted software packages. Goodware overrides will apply to all files with the trust factor value equal to or lower than the value configured here. Trust factor is expressed as a number from 0 to 5, with 0 representing the best trust factor (highest confidence that a file contains goodware). #### Enable Classification Scanners Fine-tune which scanners are used in the static analysis performed by Workers. - Images: heuristic image classifier - PECOFF: Heuristic Windows executable classifier - Documents: Document format threat detection - Certificates: Checks whether the file certificate passes the certificate validation in addition to checking white and black certificate lists - Hyperlinks: Embedded hyperlink threat detection - Emails: Phishing and email threat detection #### ML Model Configuration Configure the behavior of machine learning models used in static analysis. Each model can be set to one of the following options: - **Ignored**: The model's output is not considered in the classification decision. - **Disabled**: The model is not executed during analysis. - **Suspicious**: Detections from this model result in a suspicious classification. - **Malicious**: Detections from this model result in a malicious classification. The default option for all models is **Malicious**. **Important: General models are early warning detectors that can identify novel malware. Being both predictive and more aggressive than specialized models, they can result in unwanted detections for legitimate software interacting with low-level system functions.** ##### Scripts | Option | Description | Example Detection | | ------ | ----------- | ----------------- | | Python | ML model for detecting malicious Python scripts. | Script-Python.Malware.Heuristic | | Visual Basic | ML model for detecting malicious Visual Basic scripts. | Script-Macro.Malware.Heuristic | | PowerShell | ML model for detecting malicious PowerShell scripts. | Script-PowerShell.Malware.Heuristic | | AutoIT | ML model for detecting malicious AutoIT scripts. | Script-AutoIt.Malware.Heuristic | | Excel | ML model for detecting malicious Excel macros. | Script-Macro.Malware.Heuristic | ##### Windows | Option | Description | Example Detection | | ------ | ----------- | ----------------- | | General | General ML model for detecting malicious Windows executables. | Win[32\|64].Malware.Heuristic | | Backdoor | ML model for detecting backdoor threats in Windows executables. | Win[32\|64].Backdoor.Heuristic | | Downloader | ML model for detecting downloader threats in Windows executables. | Win[32\|64].Downloader.Heuristic | | InfoStealer | ML model for detecting information stealer threats in Windows executables. | Win[32\|64].Infostealer.Heuristic | | Keylogger | ML model for detecting keylogger threats in Windows executables. | Win[32\|64].Keylogger.Heuristic | | Ransomware | ML model for detecting ransomware threats in Windows executables. | Win[32\|64].Ransomware.Heuristic | | Riskware | ML model for detecting riskware threats in Windows executables. | Win[32\|64].PUA.Heuristic | | Worm | ML model for detecting worm threats in Windows executables. | Win[32\|64].Worm.Heuristic | ##### Linux | Option | Description | Example Detection | | ------ | ----------- | ----------------- | | General | General ML model for detecting malicious Linux executables. | Linux.Malware.Heuristic | #### Ignore the Following Threat Types Selected threat types will be excluded from final classification decision. The classification returned will be Goodware with reason Graylisting. - Adware - Packer - Riskware - Hacktool - Spyware - Spam #### Password List Appliances use these passwords to decrypt password-protected compressed files submitted for analysis. Prior to submitting password-protected compressed files to the appliance, users can add the password for each file to this list (one password per line). Passwords can also be provided on each upload using the optional `user_data` request field. ### Spectra Intelligence **Applies to Spectra Detect Worker** | Option | Description | | -------------------------------------------------- | ------------------------------------------------------------ | | Enable Spectra Intelligence | Receive additional classification from the Spectra Intelligence cloud. By default, it is false. | | Spectra Intelligence URL | The host address for the Spectra Intelligence cloud. Click the *Test connection* button to test the connectivity. The default URL is *https://appliance-api.reversinglabs.com* | | Username | Spectra Intelligence username | | Password | Spectra Intelligence password | | Timeout | Default Spectra Intelligence connection timeout in seconds (maximum 1000). | | Enable proxy | Enables the configuration of an optional proxy connection. By default, it is false. | | Proxy host | Proxy host name for routing requests from the appliance to Spectra Intelligence (e.g., 192.168.1.15). | | Proxy port | Proxy port number (e.g., 1080). | | Proxy username | User name for proxy authentication. | | Proxy password | Password for proxy authentication. | Cache Spectra Intelligence results to preserve quota and bandwidth when analyzing sets of samples containing a lot of duplicates or identical extracted files. | Option | Description | |---------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Enable | Enable or disable the caching feature. Default: False | | Cache max size (%) | Maximum cache size expressed as a percentage of the total allocated RAM on the Worker. Default: 6.25, Range: 5 - 15 | | Cache cleanup window | How often to run the cache cleanup process, in minutes. It is advisable for this value to be lower, or at least equal to the TTL value. Default: 10, Range: 5 - 60 | | Maximum number of idle upstream connections | The maximum number of idle upstream connections. Default: 50, Range: 10 - 50 | | Cache entry TTL | Time to live for cached records, in minutes. Default: 240, Range: 1 - 3600 | Deep Cloud Analysis enables uploading files to Spectra Intelligence for scanning, making the results available in real-time on the Spectra Detect Dashboard. You can enable or disable Deep Cloud Analysis in the **Administration > Spectra Detect Manager > Dashboard Configuration** section by clicking the "Enable Deep Cloud Analysis" checkbox. | Option | Description | |--------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Wait for Deep Cloud Analysis Results | Delays report generation until the latest AV scan completes, ensuring updated AV metadata is included. This option is disabled by default. | | Deep Cloud Analysis Timeout | Maximum time in minutes AV waits for a scan to complete before timing out, ensuring system performance isn't impacted by long delays. Default: 240. Minimum value: 1. Maximum value: 1440. | | Scan Unpacked Files | Sends unpacked files to Deep Cloud Analysis for scanning. | | Rescan Files on Submission | Rescans files upon submission based on the configured interval to include the latest AV results in the reports. This option is enabled by default. | | Rescan Interval | Set the interval in days for triggering an AV rescan. If the last scan is older than the specified value, a rescan will be initiated. A value of 0 means files will be rescanned with each submission. Default: 3. | | File type filter | Select which file types should be excluded from Deep Cloud Analysis. `PE` maps to `PE16`, `PE`, `PE+`, `ELF` maps to `ELF32 Little`, `ELF32 Big`, `ELF64 Little`, `ELF64 Big`, `MachO` maps to `MachO32 Little`, `MachO32 Big`, `MachO64 Little`, `MachO64 Big`, `Package/JAR` maps to `Package/Java/JAR` and `Binary/Archive/JAR`. | | File name filter | Define which file name patterns should be excluded from Deep Cloud Analysis. Use wildcard (*) to match multiple characters. | ### T1000 File Reputation Appliance **Applies to Spectra Detect Worker** | Option | Description | | -------------- | ------------------------------------------------------------ | | Enable T1000 | When enabled, an integration with ReversingLabs T1000 instance to receive additional classification information is configured. By default, it is false. | | T1000 URL | The host address for the on-premises T1000 File Reputation appliance. | | Username | T1000 user name for authentication. Note: this user name needs to be created via the T1000 Web administration application. | | Password | T1000 password for authentication. | | Timeout | Default T1000 service connection timeout in seconds (maximum 60). | | Enable proxy | Enables the configuration of an optional proxy connection. By default, it is false. | | Proxy host | Proxy host name for routing request from the appliance to T1000 (e.g., 192.168.1.15). | | Proxy port | Proxy port number (e.g., 1080). | | Proxy username | User name for proxy authentication. | | Proxy password | Password for proxy authentication. | ### SNMP **Applies to Spectra Detect Worker** | Option | Description | | -------------------- | ------------------------------------------------------------ | | Enable SNMP service | Select the checkbox to enable the Simple Network Management Protocol service. | | Community | Enter the name of an SNMP community list for authentication. Community is a list of SNMP clients authorized to make requests. The SNMP service will not function properly if this field is not configured. | | Enable trap sink | Select the checkbox to enable sending SNMP traps to the sink server. Traps are asynchronous, unsolicited SNMP messages sent by the SNMP agent to notify about important events on the appliances. | | Trap community | Enter the SNMP trap community string. If the *Enable SNMP service* and *Enable trap sink* checkboxes are selected, then this field is required. | | Trap sink server | Enter the host name or the IP address of the trap sink server. The sink server is the location to which SNMP traps will be sent. If the *Enable SNMP service* and *Enable trap sink* checkboxes are selected, then this field is required. | | SNMP trap thresholds | A set of configuration fields allowing the user to set the thresholds (values that will trigger an SNMP trap) for supported types of events. Thresholds can be configured for average system load in 1, 5, and 10 minutes (as a percentage), used memory and used disk space (as a percentage), the size of Spectra Detect queues (maximum value is 20000) and the size of the classifications queue (maximum value is 20000). | ### System Time **Applies to Spectra Detect Worker** | Option | Description | | ----------------------------------- | ------------------------------------------------------------ | | Enable network time synchronization | Select the checkbox to enable server clock synchronization via NTP, which uses port 123. | | NTP servers | A list of servers, one per line, to use for system clock synchronization. | ### Spectra Detect Worker Configuration #### General ##### Limits It is possible to set up limits on file processing: - maximum file size - number of daily uploads File size is in MB, and the daily limit includes files uploaded through a connector. It also resets at midnight. **Large Report Size Limit (MB)** - Reports over this size will be handled by optimizing memory consumption (RAM), which may result in longer processing and post-processing times. Views are not supported for the full report; they can only be used with the split report option. Use this option when minimizing memory usage is important. Setting to 0 disables this option. ##### Health Monitoring **Processing and Postprocessing Service Status Check** *Processing* and *postprocessing* service status fields can be used to configure how often the services will be checked for timeouts. If any issues are detected, the process will be restarted. The default for both fields is 720 minutes. Setting to 0 will disable this option. **Monit Memory Threshold** *Monit Memory Threshold* is the percentage of memory, between 50 and 100, that services can use. If memory usage reaches the number configured here, the system will restart services. **If this number is set to 100, the restart will be disabled.** **Health Thresholds** Set the health thresholds to true or false to enable or disable the thresholds functionality. - **Disk High Threshold**: Specify the highest allowed percentage of hard disk usage on the system. If it exceeds the configured value, the appliance will start rejecting traffic. - **Queue High Threshold**: Specify the maximum number of items allowed in the queue. If it exceeds the configured value, the appliance will start rejecting traffic. Default: 100. ##### Cleanup *All values are in minutes* - File age limit How long an unprocessed file is present on the appliance before being deleted. Processed files are deleted immediately after processing. Default: 1440. - Task age limit How long before the record of a completed processing task is deleted. Default: 90. - Unprocessed task limit How long before an incomplete processing task is cancelled. Default: 1440. ##### Spectra Analyze Configuration - **Spectra Analyze IP address or FQDN**: Specify the hostname or IP address of Spectra Analyze appliance associated with the Worker. This address will be referenced in Splunk reports to enable retrieving additional processing information. #### File Processing ##### Processing - **Processing Mode**: Choose the processing mode of the Worker instance to improve pressure balancing. Supported modes are *standard* and *advanced*. In advanced mode, files larger than the threshold specified below are processed individually. - **Large File Threshold**: If advanced mode is selected, files larger than the threshold specified here will be processed individually, one by one. If standard mode is enabled, this parameter is ignored. The threshold value is expressed in MB. Default is 1000. Limit is 5000. - **Unpacking Depth**: Select how "deep" a file is unpacked. For example, if a file contains other files, each of those containing other files etc., by default (when this value is set to zero), Workers will unpack everything until no more files can be unpacked. Setting this value to something else than zero specifies the depth of recursion, which can be useful for quicker (but shallower) analyses. - **Processing Timeout**: Specify how many seconds Worker should wait for a file to process before terminating the task. The default is 28800 seconds (8 hours). The minimum allowed value is 1. ##### Caching - **Enable caching**: When caching is enabled, the SHA1 of file contents is used to determine if there have been recent analysis reports for the same file, and if those reports can be reused instead of processing the file again. - **Cache Timeout**: If file processing caching is enabled, this parameter can be used to specify for how long the analysis reports should be preserved in the cache and reused before they expire (in seconds). Restarting the Worker or changing configuration will clean the cache. Setting the value to 0 will use the timeout of 600 seconds. ##### Scaling - Processing Specify how many copies of Spectra Core instances to run. Changing this setting from the default is not recommended. - Post-processing Specify how many report post-processing instances to run. These instances will then modify and save reports as specified by the user. Increasing this value can increase throughput for servers with extra available cores. Default: 1. - Preprocessing Unpacker Specify how many copies of Spectra Core is used to unpack samples for Deep Cloud Analysis. This setting only has effect if Deep Cloud Analysis is enabled with the Scan Unpacked Files capability. *Applies only to Spectra Detect Worker v5.4.1 and higher* - Load size Defines the maximum number of individual files that can simultaneously be processed by a single instance of Spectra Core. When one file is processed, another from the queue enters the processing state. Default is zero (0), which sets the maximum number of files to be processed to the number of CPU cores on the system. - Concurrency Count Defines the number of concurrent threads per Spectra Detect instance that should be used for processing. Default is zero (0), which sets the number of threads to the number of CPU cores on the system. Modifying this option may cause issues with the system. Consult with ReversingLabs Support before making any changes to the parameter. #### Analysis Report ##### Default Report Settings - **Strings**: Select the checkbox to enable extracting strings from files during Spectra Detect file analysis. - **Relationships**: When enabled, the relationships section of the report lists hashes of files that are found within a given file. - **Relationships for First Report Only**: If disabled, the reports for samples that contain children files will include the relationships of all their descendants. This can lead to a lot of redundant information in the report. If enabled, relationship metadata will be included only for the root parent file. - **Network Reputation Report**: If enabled, Spectra Detect Worker (4.1+) file analysis reports will contain a new section, `network_reputation` , with reputation information on any network resources found within the file. This feature is unavailable if **Spectra Core > Processing Settings** is set to `Fast`, as it relies on interesting strings extracted during analysis. ##### API Report Settings This section configures the default report view applied to a Spectra Detect report if no other view has been applied elsewhere. Specify the report type that should be applied to the Worker analysis report. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. - **Report Type**: Available report types are *extended_small*, *small*, *medium*, and *large*, as well as *classification*, *classification_tags*, *extended*, *mobile_detections* and *short_cert* which contain metadata equivalent to views with the same name. Click the *Upload* button to submit a custom report type to the appliance. - **Report View**: Apply a view for transforming report data to the *large* report type to ensure maximum compatibility. See **Spectra Detect Product Documentation > Analysis and Classification > Customizing Analysis Reports** for detailed information about how report types and views work. Enable the **Top Container Only** option to only include metadata for the top container. Reports for unpacked files will not be generated. Enable the **Malicious Only** option for the report to contain only malicious and suspicious children. - **Timeout (Seconds)**: Specify the timeout for API reports in seconds. The limit is 432000. Applies only to synchronous API calls. - **Number of Concurrent Connections**: Specify the number of concurrent connections for API reports. The limit is 5000. Applies only to synchronous API calls. ##### Additional hashes - CRC32 - MD5 - SHA384 - SHA512 - SSDEEP Spectra Core calculates file hashes during analysis and includes them in the analysis report. Select which additional hash types should be calculated for files analyzed on connected Worker appliances. MD5 is selected by default. SHA1 and SHA256 hashes are always included, and therefore aren’t configurable. Note that selecting additional hash types may cause the report to generate slower. #### Authentication ##### Tokens Specify tokens required for authorizing to the listed Spectra Detect Worker endpoints. Every token must be a string of alphanumeric characters between 16 and 128 characters in length. ### Egress Integrations After analysis, Spectra Detect can save: - original files - unpacked files - file analysis reports These are forwarded to one or more external storage providers: - AWS S3 - Network file share (SMB, NFS) - Microsoft Cloud storage (Azure Data Lake, OneDrive, SharePoint) #### Spectra Analyze Integration *Applies only to Spectra Detect Worker v5.4 and higher* Spectra Analyze integration allows Spectra Detect to upload processed samples to a configured Spectra Analyze instance. This can be used for sharing samples between Spectra Detect and Spectra Analyze, and for further analysis within a configured Spectra Analyze instance. - Enable Spectra Analyze Integration A checkbox to enable the Spectra Analyze integration. - Spectra Analyze Instance An instance of Spectra Analyze to which the Worker will upload samples. - Enable Global Filter A checkbox to enable the global filter set in the [Filter Management](../Config/FilterManagement.md) section. When enabled, the Worker will only upload samples that pass the global filter criteria to Spectra Analyze. - Filter Name A filter that will be used to determine which samples are uploaded to Spectra Analyze. Samples uploaded to Spectra Analyze will have tags and comments visible on the Spectra Analyze Sample Summary page. - Supported tags: `filter_name`, `source_address`, `connector_name`, and `hostname`. - Supported comments: `full_file_path`. **Note:** If the upload is performed via the API, `source_address`, `connector_name`, and `full_file_path` will not be shown on the Spectra Analyze Sample Summary page. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Management](./FilterManagement.md) workflow. #### AWS S3 There are two ways to connect to an output bucket: 1. Using your own S3 credentials. 2. Using an **IAM role**. The **AWS S3 Access Key ID** and **AWS S3 Secret Access Key** (in the *General* tab) must be provided in **both cases**. If ReversingLabs hosts the appliance and you use an IAM role, we will provide the access key and secret key. **Bucket naming conventions**: regardless of the type of storage (files, unpacked files, reports), input fields for S3 buckets expect the bucket names to conform to specific rules. The bucket name can be between 3 and 63 characters long, and can contain only lowercase characters, numbers, periods, and dashes. Each label in the bucket name must start with a lowercase letter or number. The bucket name cannot contain underscores, end with a dash, have consecutive periods, or use dashes adjacent to periods. The bucket name cannot be formatted as an IP address. ##### General - AWS S3 Access Key ID Specify your access key ID. - AWS S3 Secret Access Key Specify your secret access key. - AWS S3 Endpoint URL Enter S3 endpoint URL if you want to use S3 over HTTP. Only required in non-AWS setups in order to store files to an S3-compatible server. When this parameter is left blank, the default value is used ([https://aws.amazonaws.com](https://aws.amazonaws.com/)). Supported pattern(s): https?://.+ - SSL Verify This checkbox enables SSL verification in case of an `https` connection. - CA Path Path to the certificate file for SSL verification. If this parameter is left blank or not configured, SSL verification is disabled. By default, it is set to **/etc/pki/tls/certs/ca-bundle.crt**. - AWS S3 Region The default value is `us-east-1`. - AWS S3 Signature Used to authenticate requests to the S3 service. In most AWS regions, only Signature Version 4 is supported. For AWS regions other than `us-east-1` , the value `s3v4` must be configured here. (*Deprecated*) - AWS S3 Number of Upload Retries Maximum number of retries when saving a report to an S3-compatible server. ##### AWS IAM Role Settings Using S3 storage in a way where the customer secret key isn’t shared with the Spectra Detect system requires setting up an IAM role for Spectra Detect in the AWS console. This requires setting up the Amazon Resource Name (ARN) for Workers, which they can use to obtain temporary tokens. These temporary tokens allow saving files to S3 buckets without the customer secret access key. For this setup, an **external ID** is also required. This is provided by the entity which owns an S3 bucket. The owner of that bucket takes the AWS Account ID of the account that owns the appliance and builds an ARN with it (in the hosted Spectra Detect deployment, we provide our AWS Account ID). - ARN Session Name Name of the session visible in AWS logs. - Token duration How long the authentication token lasts before it expires and is refreshed. The minimum value is 900 seconds. - Refresh Buffer Number of seconds defined to fetch a new ARN token before the token timeout is reached. This must be a positive number, and the default value is 5. ##### File Storage This section configures how **original** files are stored. - Enable S3 File Storage A checkbox to enable storing files in an S3 bucket. - Store metadata This option stores the analysis metadata to the uploaded S3 object as key-value pairs. Metadata in S3 is used to attach additional information to objects. It can also be used to forward the data to another connector, appliance group, or Spectra Analyze for further processing. For example, you can use this metadata as a filter for only retrieving malicious files. By default, this option is enabled. For more details, see [AWS S3 Using Metadata](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html). - Enable Global Filter A checkbox to enable the global filter set in the [Filter Management](../Config/FilterManagement.md) section. When enabled, the Worker will only upload samples that pass the global filter criteria. - Filter Name A filter that will be used to determine which samples are uploaded. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Managemenet](./FilterManagement.md) workflow. - Output Bucket Destination There are two "modes" for setting the output bucket destination: 1. **Only Default Bucket**: storing files in the default bucket by inputting the bucket name. 2. **Map to Input Bucket**: storing files in buckets that are mapped to specific input buckets. - **Input Bucket**: A name of the input bucket that will be mapped to the output bucket. - **Output Bucket**: A name of the output bucket where the samples will be stored. If the output bucket is empty, any sample uploaded from the specified input bucket will be ignored. - **Mapping Filter name**: A filter that will be used to determine which samples are uploaded. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Managemenet](./FilterManagement.md) workflow. After configuring the output to input bucket mappings, a table will be displayed with the columns *Input Bucket*, *Output Bucket*, *Filter*, and *Actions* (edit or delete), where users can view and manage the mappings. - Connection Method for Output Buckets Used to set individual AWS connection methods for target buckets. Clicking the `Add New Mapping` button opens a dialog where users can set up a new mapping. Multiple mappings can be set up. The dialog contains the following fields: - Buckets A list of input buckets that will be mapped to the output bucket. - Connection Strategy - **Standard AWS**: requires setting the *AWS S3 Access Key ID* and *AWS S3 Secret Access Key*, with additional options for setting the *AWS S3 Endpoint URL*, *AWS S3 Region*, *AWS S3 Signature* (deprecated in Spectra Detect version 5.4.0+), *AWS S3 Number of Upload Retries*, *Spectra Detect Worker AWS S3 Server-Side Encryption Algorithm*, verifying the HTTPS connection against the CA bundle, and setting the *CA Path*. - **ARN Connection**: requires selecting a strategy for role assumption: *Default AWS* or *Custom AWS Connection to Assume ARN Role*. Based on the selected role assumption strategy, input the configurations. - **Default AWS**: *Role ARN*, *External ID*, *ARN Session Name*, *Token Duration*, *Refresh Buffer*, *Spectra Detect Worker AWS S3 Servier-Side Encryption Algorithm*, *verify the HTTPS connection against the CA bundle*, and the *CA Path*. - **Custom AWS Connection to Assume ARN Role**: *AWS S3 Access Key ID*, *AWS S3 Secret Access Key*, *AWS S3 Endpoint URL*, *AWS S3 Region*, *AWS S3 Signature* (deprecated in Spectra Detect version 5.4.0+), *AWS S3 Number of Upload Retries*, *Role ARN*, *External ID*, *ARN Session Name*, *Token Duration*, *Refresh Buffer*, *Spectra Detect Worker AWS S3 Server-Side Encryption Algorithm*, *verify the HTTPS connection against the CA bundle*, and the *CA Path*. After configuring the output to input bucket mappings, a table will be displayed with the columns *Input Bucket*, *Output Bucket*, *Filter*, and *Actions* (edit or delete), where users can view and manage the mappings. Clicking the `Delete` button removes the selected mapping. - Server-Side Encryption Algorithm The server-side encryption algorithm can be any server-side encryption configured on the target default bucket (such as `aws:kms` or `AES256`). Clicking the `Test Connection` button will attempt to verify the selected server-side encryption. - Folder Folder where samples will be stored on given S3 bucket. The folder can be up to 1024 bytes long when encoded in UTF-8. It must not start or end with a slash or contain leading or trailing spaces. Consecutive slashes ("//") are not allowed. The folder key containing relative path elements ("../") are valid if, when parsed left-to-right, the cumulative count of relative path segments never exceeds the number of non-relative path elements encountered. **Note: Storage age policy** Spectra Detect retains data stored in customer S3 buckets for **12 months**. After this period, the data is automatically deleted from the S3 storage. ##### Unpacked Files Storage This section configures how **unpacked** files are stored. There is just one **output mode**: a default bucket that needs to be specified in the *Bucket Name* field (with an optional *Folder* name). Unpacked files are saved in **two possible formats**: 1. Individual unpacked files are saved in subfolders, which can be named in three different ways: | Method | Format | | ------------------- | ---------------------------------------------- | | Date-based | `YYYY/mm/dd/HH` | | Date-and-time-based | `YYYY/mm/dd/HH/MM/SS` | | SHA-1-based | first four characters of a sample’s SHA-1 hash | 2. Unpacked files are saved as ZIP archives, which can optionally be password-protected. ##### Report Storage This section configures how **analysis reports** are stored. - Enable S3 Report Storage A checkbox to enable storing reports in an S3 bucket. - Enable Global Filter A checkbox to enable the global filter set in the [Filter Management](../Config/FilterManagement.md) section. When enabled, mapping filters will be automatically disabled. - Filter Name A filter that will be used to determine which samples are uploaded. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Managemenet](./FilterManagement.md) workflow. - Output Bucket Destination There are three "modes" for setting the output bucket destination: 1. **Map to Input Bucket**: storing reports in buckets that are mapped to specific input buckets. - **Input Bucket**: A name of the input bucket that will be mapped to the output bucket. - **Output Bucket**: A name of the output bucket where the samples will be stored. If the output bucket is empty, any sample uploaded from the specified input bucket will be ignored. - **Mapping Filter name**: A filter that will be used to determine which samples are uploaded. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Managemenet](./FilterManagement.md) workflow. After configuring the output to input bucket mappings, a table will be displayed with the columns *Input Bucket*, *Output Bucket*, *Filter*, and *Actions* (edit or delete), where users can view and manage the mappings. 2. **Same as Input Bucket**: storing reports in the same bucket as the input bucket. 3. **Only Default Bucket**: storing reports in the default bucket by inputting the bucket name. - Connection Method for Output Buckets Used to set individual AWS connection methods for target buckets. Clicking the `Add New Mapping` button opens a dialog where users can set up a new mapping. Multiple mappings can be set up. The dialog contains the following fields: - Buckets A list of input buckets that will be mapped to the output bucket. - Connection Strategy - **Standard AWS**: requires setting the *AWS S3 Access Key ID* and *AWS S3 Secret Access Key*, with additional options for setting the *AWS S3 Endpoint URL*, *AWS S3 Region*, *AWS S3 Signature* (deprecated in Spectra Detect version 5.4.0+), *AWS S3 Number of Upload Retries*, *Spectra Detect Worker AWS S3 Server-Side Encryption Algorithm*, verifying the HTTPS connection against the CA bundle, and setting the *CA Path*. - **ARN Connection**: requires selecting a strategy for role assumption: *Default AWS* or *Custom AWS Connection to Assume ARN Role*. Based on the selected role assumption strategy, input the configurations. - **Default AWS**: *Role ARN*, *External ID*, *ARN Session Name*, *Token Duration*, *Refresh Buffer*, *Spectra Detect Worker AWS S3 Servier-Side Encryption Algorithm*, *verify the HTTPS connection against the CA bundle*, and the *CA Path*. - **Custom AWS Connection to Assume ARN Role**: *AWS S3 Access Key ID*, *AWS S3 Secret Access Key*, *AWS S3 Endpoint URL*, *AWS S3 Region*, *AWS S3 Signature* (deprecated in Spectra Detect version 5.4.0+), *AWS S3 Number of Upload Retries*, *Role ARN*, *External ID*, *ARN Session Name*, *Token Duration*, *Refresh Buffer*, *Spectra Detect Worker AWS S3 Server-Side Encryption Algorithm*, *verify the HTTPS connection against the CA bundle*, and the *CA Path*. After configuring the output to input bucket mappings, a table will be displayed with the columns *Input Bucket*, *Output Bucket*, *Filter*, and *Actions* (edit or delete), where users can view and manage the mappings. Clicking the `Delete` button removes the selected mapping. - Folder Folder where reports will be stored on given S3 bucket. The folder can be up to 1024 bytes long when encoded in UTF-8. It must not start or end with a slash or contain leading or trailing spaces. Consecutive slashes ("//") are not allowed. The folder key containing relative path elements ("../") are valid if, when parsed left-to-right, the cumulative count of relative path segments never exceeds the number of non-relative path elements encountered. - Report Type / View Upload, manage and configure report types. See **Spectra Detect Product Documentation > Analysis and Classification > Customizing Analysis Reports** for detailed information about how report types and views work. - Top Container Only Enable to only include metadata for the top container. Reports for unpacked files will not be generated. - Malicious Only For the report to contain only malicious and suspicious children. - Split Report Split reports of extracted files into individual reports. - Archive and Compress Split Report Files Enable sending a single, smaller archive of split report files to S3 instead of each file. Relevant only when the "Split report" option is used. - Archive Password If set, enables encryption of the archive file using this value as the password. Relevant only when the "Archive and compress split report files" option is used. - Subfolder Reports can be saved into subfolders, with specific naming formats: | Subfolder naming | Format | |---------------------|------------------------------------------------| | Date-based | `YYYY/mm/dd/HH` | | Date-and-time-based | `YYYY/mm/dd/HH/MM/SS` | | SHA-1-based | first four characters of a sample’s SHA-1 hash | - Enable Filename Timestamp This refers to the naming of the report file itself (and not the subfolder in which it is stored). A timestamp is appended to the SHA1 hash of the file. The timestamp format must follow [the strftime specification](https://strftime.org/) and be enclosed in quotation marks. If not specified, the ISO 8601 format is used. ##### SNS You can enable publishing notifications about file processing status and links to Worker analysis reports to an Amazon SNS (Simple Notification Service) topic. The configured AWS account must be given permission to publish to this topic. The topic name is limited to 1000 characters. #### Network Share Configuration Specify the protocol and the address of the network share. The supported protocols are NFS and SMB, so for an NFS share, the address would start with `nfs://`. The address should include the IP address or URL of the share, and should also include the full path to the shared directory. Note: the *username* and *password* fields are not required for NFS, only for SMB. #### Microsoft Cloud Storage Configuration ##### Azure To set up saving on Azure, specify the storage account name and storage account key. In order to use a custom server, specify the endpoint suffix for the address of your Azure Data Lake container, which defaults to **core.windows.net**. The Azure integration can save analyzed files, unpacked files, as well as file analysis reports. The integration **doesn’t** support saving unpacked files or split reports into a single ZIP archive. Regardless of the use, the container name must be a valid DNS name, conforming to the following naming rules: 1. Container names must start or end with a letter or number, and can contain only letters, numbers, and the dash (-) character. 2. Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not permitted in container names. 3. All letters in a container name must be lowercase. 4. Container names must be from 3 through 63 characters long. ##### OneDrive / SharePoint Online Microsoft Cloud Storage allows saving of reports and analyzed files on either SharePoint Online or OneDrive storage types. After configuring the *Client ID*, *Client Secret*, and the application’s *Custom Domain* (as configured in the Azure portal), select the desired storage type and fill in the appropriate additional credentials: OneDrive storage requires the OneDrive account *Username*, while SharePoint requires the *Site Hostname* and *Site Relative Path* (for example, `sharepoint.com` and `sites/test-public`). #### Saving Files and Reports Different integrations have different options available for saving analyzed files and reports, but all will require configuring the storage location (S3 bucket or buckets, Microsoft Cloud storage, Azure container, or network share) and a folder on that storage (optional). These settings relate to where Spectra Detect Workers appliances save files. ##### Report customization It’s possible to change the look of the report to further customize it. See **Spectra Detect Product Documentation > Analysis and Classification > Customizing Analysis Reports** for detailed information about how report types and views work. ##### Splitting reports All storage options allow splitting the report into several JSON files, one report per extracted file. In addition to that, S3 and network share sections also allow saving these individual reports together in an archive (optionally password-protected). ##### Report name format Reports are saved with their timestamp at the end of their file name. By default, they will end with an ISO 8601 datetime string (YYYY-MM-DDTHH:mm:ss.SSSSSS) but this can be modified following the Python `strftime()` syntax. For example, to save reports only with their year and month, set the *Filename Timestamp Format* to `%Y%m`. This field is editable only if the **Enable Filename Timestamp** option is turned on. ### Saving unpacked files During analysis, a Worker can extract "children" from a parent file (the file initially submitted for analysis). Such child files can be saved to one of the external storage options (S3 bucket, Azure container, or network share). It’s also possible to sort them into subfolders based on date, date and time, or based on the first four characters of the sample’s SHA-1 hash. S3 and network share sections also allow saving these unpacked files as a ZIP archive instead of each individual file (*Archive and Compress Unpacked Files*). #### Callback Select the checkbox to enable sending file analysis results to an HTTP server ("callback"), and optionally return the analysis report in the response. Specify the full URL that will be used to send the callback POST request. Only HTTP is supported. If this parameter is left blank, no requests will not be sent. Additionally, specify the number of seconds to wait before the POST request times out. Default is 5 seconds. In case of failure, the Worker will retry the request up to six times, increasing waiting time between requests after the second retry has failed. With the default timeout set, the total possible waiting time before a request finally fails is 159 seconds. ##### CA path If the Callback URL parameter is configured to use HTTPS, this field can be used to set the path to the certificate file. This automatically enables SSL verification. If this parameter is left blank or not configured, SSL verification will be disabled, and the certificate will not be validated. ##### Report options and views It’s possible to change the look of the report to further customize it. Click the *Upload* button to submit a custom report type, or select one of the default options. Names for views can only include alphanumerical characters, underscores and dashes. See **Spectra Detect Product Documentation > Analysis and Classification > Customizing Analysis Reports** for detailed information about how report types and views work. Enable the **Top Container Only** option to only include metadata for the top container. Reports for unpacked files will not be generated. Enable the **Malicious Only** option for the report to contain only malicious and suspicious children. Enable the **Split Report** option to split reports of extracted files into individual reports. Enable the **Include Full Report** option to retrieve the full report. By default, only the summary report is provided in the callback response. ### Archiving *only for S3 and Azure* Files can be stored either as-is or in a ZIP archive. This archive can further be password-protected and customized: - Zip Compression Level: 0 (no compression) to 9 (maximum compression). The default is 0. - Maximum Number of Files: Specify the maximum allowed number of files that can be stored in one ZIP archive. Allowed range: 1 … 65535 ### File filters **Tip: You can also configure [advanced file filters](../FilterManagement).** The file filter is used by the Worker to control which files won't be stored after processing. You can filter out files based on their classification, factor, file type, and file size. For a file to be filtered out by the Worker, at least one of the filters has to match. To enable the feature, select the *File filters* checkbox in the **Central Configuration ‣ Egress Integrations ‣ File Filters** dialog. Then, use the **Add new filter** button to create custom file filters. Every filter can be individually enabled or disabled by selecting or deselecting the *Active* checkbox, or the checkbox to the right of the filter name. All created filters are listed in the dialog. Every filter can be expanded by clicking the arrow to the left of the filter name. When a filter is expanded, users can modify any of the filtering criteria, or remove the entire filter by clicking **Delete**. #### File Filtering Criteria | CRITERIA | DESCRIPTION | | -------------- | ------------------------------------------------------------ | | Classification | Allows the user to filter out files by their classification. Supported values are "Known" and "Unknown". Both values can be provided at the same time. Malicious and suspicious files cannot be filtered out. | | Factor | Allows file filtering based on threat factor. When a file is processed, it is assigned a threat factor value, represented as a number from 0 to 5, with 5 indicating the most dangerous threats (highest severity). Enter one value from 0 to 5. The filter program will filter out files with the threat factor of N (entered value) or less. | | File Type | Spectra Detect Worker can identify the file type for every analyzed file. To filter out files by type, select one or more file types, or select the "All" checkbox. | | File Size | To filter out files by size, specify the file size in any of the supported units, and the file size condition (greater than or less than). The file size value should be provided as an integer; if it is not, it will automatically be rounded down to the nearest whole integer. | ### Splunk **Applies only to Spectra Detect Worker** | Option | Description | | ------------------------- | ------------------------------------------------------------ | | Enable | Select the checkbox to enable Splunk integration. | | Host | Specify the hostname or IP address of the Splunk server that should connect to the Worker appliance. | | Port | Specify the TCP port of the Splunk server’s HTTP Event Collector. Allowed range(s): 1 … 65535 | | Token | Specify the API token for authenticating to the Splunk server. Not mandatory. | | Use HTTPS | Select the checkbox to use the HTTPS protocol when sending information to Splunk. If it’s not selected, non-encrypted HTTP will be used. | | SSL require certificate | If HTTPS is enabled, selecting this checkbox will enable certificate verification. The Worker host needs to have correct certificates installed in order to successfully pass verification. | | Timeout - **Worker only** | Specify how many seconds to wait for a response from the Splunk server before the request is considered failed. If the request fails, the report will not be uploaded to the Splunk server, and an error will be logged. Default is 5 seconds | | Report type | Specify the report type that should be applied to the Worker analysis report before sending it to Splunk. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Click the *Upload* button to submit a custom report type to the appliance. Report types are stored in the `/etc/ts-report/report-types` directory on each Worker. | | Report view | Specify the name of an existing transformation view that should be applied to the Worker analysis report before sending it to Splunk. Views can be used to control the presentation format and the contents of the analysis report; for example, to flatten the JSON hierarchy, or to preserve only selected parts of the report. Allowed characters are alphanumeric characters, underscore and dash. Views are stored in the `/usr/libexec/ts-report-views.d` directory on each Worker. | | Top Container Only | Whether or not the report sent to Splunk should contain the reports for child files. | ### System Alerting **Applies to Spectra Detect Worker** | Option | Description | | ----------------------------------------- | ------------------------------------------------------------ | | **Syslog receiver** | | | Enable | Select the checkbox to receive alerts about the status of critical system services to a syslog server. Read more about which services are supported in the table below. | | Host | Host address of the remote syslog server to send alerts to. | | Port | Port of the remote syslog server. | | Protocol | Communication protocol to use when sending alerts to remote syslog server. Options are TCP (default) and UDP. | **System Alerting: Supported Services** Syslog notifications are sent when any of the services or operations meets the condition(s) defined in the table. | SYSTEM OPERATION OR SERVICE | NOTIFICATION TRIGGER | | ------------------------------- | ------------------------------------------- | | RAM | usage is over 90% for 10 minutes | | CPU | usage is over 40% for 2 minutes | | CPU wait (waiting for IO) | over 20% for 2 minutes | | Disk usage | over 90% for 10 minutes | | UWSGI service | down for 2 minutes | | NGINX service | down for 2 minutes | | RABBIT-MQ service | down for 2 minutes | | POSTGRES service | down for 2 minutes | | MEMCACHED service | down for 2 minutes | | CROND service | down for 2 minutes | | SSHD service | down for 2 minutes | | SUPERVISORD service | down for 2 minutes | | SMTP | if enabled, but stopped for 4 minutes | | NTPD | if enabled, but stopped for 4 minutes | | Any of the SUPERVISORD services | if it has crashed | | SCALE socket | not detected/does not exist for 4 minutes | | SCALE INPUT queue | receiving over 500 messages for 10 minutes | | SCALE RETRY queue | receiving over 100 messages for 10 minutes | | COLLECTOR queue | receiving over 1000 messages for 10 minutes | | CLASSIFICATION queue | receiving over 5000 messages for 10 minutes | In addition, the Manager sends syslog alerts for files that haven’t been processed in 24 hours and for file processing failures. ### Log Management **Applies to Spectra Detect Worker** Users can configure the [level](https://en.wikipedia.org/wiki/Syslog#Severity_level) of events that are sent to a syslog receiver (*Syslog log level*) or that are saved in internal logs (*TiScale log level*). Users cannot save high-severity events only but send lower-severity events to a syslog receiver. To send events, they first need to be saved, which is why the *TiScale log level* must always be equal or lower-severity than the value in *Syslog log level*. --- ## SpectraDetect Certificate Management — Root CA Trust Store *Administration > Certificates* The **Root CA Trust Store Management** page enables administrators to manage Root CA certificates for Spectra Detect and [Spectra Analyze](/SpectraAnalyze/) appliances. Users can add, remove, trust, and distrust Root CA certificates through the web interface to customize the appliance's certificate validation behavior for secure communications. ## Certificate Scope Certificates managed through the Spectra Detect interface apply to: - **Spectra Detect Manager** - **Spectra Detect Hub** - **Spectra Detect Worker** - **Spectra Analyze** ## Accessing Certificate Management To access the certificate management interface: 1. Log into the appliance web interface as an administrator. 2. Navigate to **Administration > Certificates**. 3. The certificate management page displays all Root CA certificates currently in the Trust Store. ## Certificate List Overview The certificate management page displays a table with the following information for each certificate: | Column | Description | | -------------- | ------------------------------------------ | | **ID** | Number of certificate. | | **Subject** | Name of certificate. | | **Issuer** | Certificate issuer. | | **Valid From** | Certificate validity start date. | | **Valid To** | Certificate validity end date. | | **Filename** | Name of .pem file. | | **Trusted** | Whether the certificate is currently trusted. | | **Actions** | Available operations for the certificate. | ## Adding Root CA Certificates To add a new Root CA certificate to the Trust Store: 1. Under **Administration > Certificates**, click **Add Certificate**. 2. Click **Browse Files**, and select a valid certificate file in `.pem` format. 3. Click **Upload** to add the certificate to the Trust Store. 4. The certificate appears in the list with the **Trusted** status set to *Yes*. ## Managing Certificates To distrust or blocklist a certificate: 1. Locate the certificate in the list. 2. Under **Actions**, click **Distrust**. 3. Confirm the change in the modal dialog. 4. The **Trusted** status changes to *No*. To re-trust a certificate: 1. Locate the certificate in the list. 2. Under **Actions**, click **Trust**. 3. Confirm the change in the modal dialog. 4. The **Trusted** status changes to *Yes*. To remove a certificate: 1. Locate the certificate in the list. 2. Under **Actions**, click **Remove**. 3. Confirm the change in the modal dialog. 4. The certificate is deleted from the Trust Store. **Info: Some certificates may be greyed out because you don't have the necessary permissions to modify them.** To reset the Trust Store: 1. Click **Remove All Added Certificates**. 2. Confirm the action in the modal dialog. 3. All user-added certificates are removed from the Trust Store. **Warning: Removal of certificates works for both trusted and distrusted certificates and completely deletes them from the system. ** Clicking **Remove All Added Certificates** removes all certificates added by users and cannot be undone. --- ## Spectra Detect Filter Management — Advanced Egress Filtering Rules In addition to regular [file filtering on egress integrations](../ApplianceConfiguration#file-filters), you can set up an advanced filter. Advanced filters have more options and therefore allow more granularity. **Note: Advanced filters are currently available for [Spectra Analyze](/SpectraAnalyze/) Integration (Central Configuration > Egress Integrations > Spectra Analyze Integration), AWS S3 (Central Configuration > Egress Integrations > AWS S3 > File Storage/Unpacked Files Storage/Report Storage > Enable Filter), and Callback (Central Configuration > Egress Integrations > Callback > Enable Filter)** ### Types of filters Filters can be either **inclusive** or **exclusive**. If a filter is inclusive, that means that all files that match you criteria will be saved to an S3 bucket. If it's exclusive, you will save everything *except* the files that match your criteria. When creating a filter, you can also specify that it should only apply to top-level parent files: **Filter Applies Only to Container**. In this case, extracted files will not be saved. ### Conditions Conditions allow you to specify rules based on various attributes such as file type, file size, classification, and more. Conditions are grouped into categories: - **File**: filters related to one of the recognized file types and file size - **Classification**: filters related to [how we classify files](/General/AnalysisAndClassification/Classification) - **Identification**: filters for file format identification - **Behavior**: filters for network calls made by a file (see the [report schema](/SpectraDetect/report-schema#uri)) - **Document**: if a file is identified as a document, these filters allow you to narrow down what you save based on the document attributes, such as number of pages or word count - **Unpacking**: similar to how the "Identification" category of conditions relates to file format identification, this category contains filters based on file unpacking - **File statistics**: filters operating on statistics produced by an analysis (for example, the `count` of files of a particular type found in an analyzed file) - **Capabilities**: different capabilities detected in the analyzed file - **YARA matches**: see description of the filters [in the report schema](/SpectraDetect/report-schema/#yara) - **Indicators**: see [report schema](/SpectraDetect/report-schema/#indicator) - **Mitre**: one of the Mitre [techniques](https://attack.mitre.org/techniques/enterprise/) - **Tags**: the full list is available in the [report schema appendix](/SpectraDetect/report-schema/#appendix-c-spectra-core-tags) --- ## Spectra Detect Notifications — Classification Alerts and Delivery Rules Users can access the notifications page from the header by clicking the notifications icon, which will display unread notifications, providing a quick overview of alerts that require attention. Clicking the **See all notifications** link redirects users to the notifications page, where they can view all notifications. A table on the notifications page displays all notifications, separated in columns by *Type*, *Time*, and *Notification*. The *Type* column indicates the type of notification. The *Time* column displays the timestamp of the notification, indicating when the event occurred. The *Notification* column provides a brief description of the event, such as a classification change from unknown (no threats found) to malicious. Filtering options are available to help users quickly find relevant information. Notifications can be [filtered by period](#filter-by-period), allowing users to view alerts from the last hour, day, week, month, or all time. Users can also [filter by read status](#filter-by-read-status), distinguishing between read and unread notifications, [filter by notification type](#filter-by-notification-type), including Cloud Classification Changes and Classification Detection, or [filter by classification](#filter-by-classification), narrowing the results to cloud classification changes where a sample was marked as unknown (no threats found), malicious, suspicious, or goodware. Clicking on the hash value within the alert redirects users to the **Dashboard > Analytics > Detections Overview** table, providing additional context and information about the sample. The **Mark All as Read** button allows users to clear unread notifications by marking them as read. This can be used to quickly clear the notification list and focus on new alerts as they arrive. ## Filter by Period The filter by period option allows users to view notifications from the last hour, day, week, month, or all time. This can be used to quickly identify recent alerts and track changes in the classification status. ## Filter by Read Status The filter by read status option allows users to distinguish between read and unread notifications. This can be used to quickly identify new alerts that require attention or to review previously read notifications for additional context. ## Filter by Notification Type The filter by notification type option allows users to distinguish notifications from Cloud Classification Changes and Classification Detection. ## Filter by Classification The filter by classification option allows users to narrow results based on the classification of the sample, including samples marked as Unknown (No Threats Found), Malicious, Suspicious, or Goodware. ## Notification Settings The notification settings page allows users to configure and manage custom notification rules for tracking cloud classification changes. The page provides an overview of existing notifications, displaying their *Name*, *Type*, associated *Alert* type, *Description*, and *Action*. Users can navigate through the list using pagination controls and adjust the number of rows displayed per page. If no notifications are configured, the table remains empty. A button labeled **Add Notification** in the upper-right corner allows users to create new notification rules. ### Adding a Notification To add a new notification, users must first specify a *name*, *description*, and select a *notification type*. When choosing cloud classification changes, users can define the conditions by selecting the original classification (*cloud classification changes from*) and the new classification (*cloud classification changes to*) that will be used to trigger the notifications. Users can also choose the delivery method, including *E-mail*, *Splunk*, or *Syslog*, to ensure alerts are sent through the appropriate channels. - *E-mail* delivery method requires users to enter the recipients' *email addresses* and select the desired *notification frequency*. - *Splunk* delivery method requires users to enter the Splunk *protocol* (`http` or `https`), *host*, *port*, and *token*. - *Syslog* delivery method requires users to enter the Syslog *server*, *port*, *protocol* (`UDP` or `TCP`), *tag*, and *priority* level. ### Manage Profiles The Profile section allows users to manage their personal information and credentials. It includes fields for *First Name*, *Last Name*, and *Email Address*. Users can update their password by entering a new password, repeating the new password, and providing their current password for verification. These options ensure users can securely manage their account settings within the notification system. --- ## Spectra Detect Redundancy — Manager Clustering and Failover Spectra appliances are designed for full redundancy. If one component fails, another automatically takes over using the same configuration settings. This chapter describes how redundancy works for **Spectra Detect Manager**. ## Creating Manager Redundancy **Important: A proxy or Load balancer is REQUIRED for Spectra Detect Manager to function properly in Redundancy.** Use the following endpoint to identify the active/primary Spectra Detect Manager: `GET /api/cluster/check/` Before clustering, add the active/primary manager (the one you will use to initiate redundancy) to the proxy or load balancer. After redundancy setup completes, update the proxy to include both managers (this is described in [step 7](#7-update-the-proxy-or-load-balancer)). ### 1. Configure Allowed Hosts As admin, go to [Administration > Spectra Detect Manager > General > Network Settings](../Admin/ManagerSettings.md#general). - Verify that the **Application URL** is set to the proxy IP or domain. - In **Allowed Hosts**, ensure the following are listed: 1. Current Detect Manager IP address and domain 2. Proxy IP and domain 3. Secondary IP and domain 4. localhost and 127.0.0.1 ![Allowed Hosts](./images/redundancy_1.png) - Click **Save** on the primary manager. - Repeat for the secondary manager, but **only update the Allowed Hosts section**. Do not modify the Application URL on the secondary manager. It will be automatically replaced during clustering. **Note:** Changing the Application URL updates the configuration on all connected appliances (Hub, Workers, and Analyze). ### 2. Create Redundant Cluster Go to **Administration > Redundancy**, then click **Create redundant cluster**. ![Create Redundant Cluster](./images/redundancy_2.png) ### 3. Establish Connection ![Establish Connection](./images/redundancy_3.png) - Fill out the required fields. - A VPN is automatically established between redundant managers. - The user on the secondary manager must be an admin. - Set **Failover Timeout (seconds)** between 3 and 600; 30 is recommended. - Usually, the machines in a redundant cluster share the same TLS certificate. However, if you need individual TLS certificates for each machine, you can *Disable TLS Certificate Sharing*. - Click **Next**. ### 4. Check Prerequisites ![Check Prerequisites](./images/redundancy_4.png) - The system validates if your environment supports redundancy. - When all checks pass, the message "All checks successfully passed" appears. - Click **Next** to continue. ### 5. Run the Configuration Process ![Run the Configuration Process](./images/redundancy_5.png) - Click **Start Configuration**, then confirm in the popup. - **Do not refresh or navigate away** during setup; leaving the page interrupts the process and it cannot be reopened. - The system automatically switches to the maintenance screen while redundancy is configured. - When setup completes, both managers reboot and return online automatically. Wait until both are reachable before proceeding. **Important: When initiating redundancy on a Spectra Detect Manager with Central logging enabled and a large volume of stored sample data, the initial sync to the secondary manager may take longer than 30 minutes.** If you refresh the screen after 30 minutes, you may see a "Configuration failed" status with a "Rollback Configuration" button, even though clustering is still in progress. Do not roll back while data is syncing. To verify sync progress, SSH to each SDM and run the `top` command. If the `rsync` process is using ~70% CPU or more, the sync is still in progress. Wait until the `rsync` process on both the primary and secondary is under ~10% CPU before refreshing the GUI. - When the "Configuration finished successfully" message appears, click **Finish**. ![Finish the Configuration Process](./images/redundancy_6.png) ### 6. Verify Status - Open the **Cluster Configuration** tab to see which manager is primary and which is secondary. - The secondary manager is read-only while redundancy is active. - The **Status** tab provides redundancy health and logs. - You can initiate a switchover or remove the cluster configuration here. - Check the [Manage and View the Status of Redundancy](#manage-and-view-the-status-of-redundancy) section for more details. ### 7. Update the Proxy or Load Balancer - After the redundancy setup is complete, update the proxy or load balancer to include both manager IPs. - Access the Spectra Detect GUI via the proxy or load balancer; it automatically directs users to the primary node. - When you later remove redundancy, follow the guidance in the [Post De-clustering Actions](#post-de-clustering-actions) section below to keep only one manager active. ## Manage and View the Status of Redundancy ### Redundancy Status Go to **Administration > Redundancy > Cluster Configuration**. ![Cluster Configuration](./images/redundancy_7.png) - The **Cluster Configuration** tab shows which manager is primary or secondary. - Click the **Status** tab for detailed health checks and logs. - Three log views are available: RL Cluster, RL Daemon, and Corosync. ![Cluster Status](./images/redundancy_8.png) ### Managing Switchovers ![Manual Switchover](./images/redundancy_9.png) 1. Go to **Administration > Redundancy > Cluster Configuration**. 2. Click **Manual Switchover** to make the secondary manager primary. 3. Confirm the action when prompted. ![Manual Switchover](./images/redundancy_10.png) 4. The system enters maintenance mode for several minutes. 5. When complete, the proxy or load balancer automatically directs traffic to the new primary. ## Removing Manager Redundancy 1. Go to **Administration > Redundancy > Cluster Configuration**. ![Remove Cluster Configuration](./images/redundancy_11.png) 2. Click **Remove Cluster Configuration**, then confirm - If the Application URL was set to the proxy IP or domain, no changes are sent to the connected appliances - The system enters maintenance mode for a few minutes - Once complete, you will have two independent Spectra Detect Managers. Review [Post De-clustering Actions](#post-de-clustering-actions) after this step to retire the unused manager. ### If the Secondary is unavailable during de-clustering - You can still remove the cluster following the directions in Step 2 above. If the offline secondary manager was permanently removed or destroyed, no additional cleanup is required after Step 2. - If you have access to the server, you can manually run the below command to clean up the de-clustered Spectra Detect Manager that was offline during the initial de-cluster action. ```bash sudo /bin/cluster_manage destroy ``` If the secondary comes back online after de-clustering, it must be manually de-clustered. Contact [ReversingLabs Support](mailto:support@reversinglabs.com) for help. ## Post De-clustering Actions **Important: **After de-clustering, shut down or remove one of the Spectra Detect Managers.**** All connected appliances continue communicating through the proxy or load balancer, which remains configured to route to the active manager. Remove the inactive or retired manager from the proxy or load balancer configuration to prevent connection attempts to a non-existent node. If you plan to re-establish redundancy with a new secondary manager, repeat the steps in [Creating Manager Redundancy](#creating-manager-redundancy). --- ## Spectra Detect YARA Sync — Ruleset Synchronization Across Appliances The YARA Sync page (**Administration ‣ YARA Synchronization**) allows users to easily track the status of YARA ruleset synchronization between connected appliances, and trigger a manual synchronization if rules are not up-to-date. The Manager stores all synchronized rules in a local database and becomes the single source of truth for all connected appliances. When *YARA ruleset synchronization* is enabled, the YARA Sync page displays a table of all appliances connected to the Manager and their YARA ruleset synchronization status. Any connected [Spectra Analyze](/SpectraAnalyze/) appliances must have YARA Synchronization enabled (**Administration ‣ Spectra Detect Manager ‣ General ‣ Synchronization**) to properly display the current status and synchronize rulesets. Appliances can show one of the following statuses: - InSync - OutOfSync - Error - Unavailable - PendingNew - Disabled - NoRules Workers poll the Manager for rule changes every minute. Spectra Analyze appliances push new rules to the Manager as soon as they are created, and pull new rules every 5 minutes. Appliances that are *Not In Sync* can be manually synchronized at any time by clicking the *Start YARA Sync* button in the far right column of the table. Rulesets created on Spectra Analyze appliances before YARA synchronization was enabled will not synchronize to the Manager until the user changes their status or modifies them in any way. Rules present on the Manager, however, will synchronize to newly connected Spectra Analyze appliances regardless of when they were created. Apart from new rulesets, changes in existing rulesets will be synchronized as well. If a ruleset is disabled or deleted on one appliance, its status will be distributed to other appliances. In case of Workers, disabled rulesets will be removed until re-enabled on another appliance. When enabled again, rulesets will be synchronized on the Worker as if they have been newly created. Since all rulesets have owners, their user accounts will be mirrored to other connected appliances, but won’t be able to log into that instance until an administrator enables their account by assigning it a password. YARA Ruleset Restrictions - Naming restrictions: - YARA ruleset names must be between 3 and 48 characters. - The underscore ( _ ) should be used instead of spaces, and any other special characters should be avoided. Ruleset names should only use numbers (0-9) and a-z/A-Z letters. - Ruleset size restrictions: - A ruleset file should not be larger than 4 MB. - A ruleset file should not contain more than 5000 individual rules. - A ruleset larger than 1 MB (1048576 bytes) cannot be saved and run in the [Spectra Intelligence](/SpectraIntelligence/) cloud. - File size restrictions: - YARA rulesets on Spectra Analyze are not applied to files larger than 700 MB. Only rules that have been successfully compiled on Spectra Analyze can be synchronized. --- ## Spectra Detect Configuration — Appliances, Integrations, YARA, and Settings --- ## SpectraDetect AWS EKS Config Reference — Secrets and ConfigMap Values ## Processing ### Configmap configuration secrets | Secret | Type | Description | Used in deployments (Pods) | | :---- | :---- | :---- | :---- | | \-secret-worker-api-token | Optional | Token secret which contains the token that is used to protect all endpoints with /api/ prefix, e.g. file upload. | Auth | | \-secret-worker-api-task-token | Optional | Token secret which contains the token that is used to protect /api/tiscale/v1/task endpoints. If left empty, the mentioned API is protected by \-secret-worker-api-token | Auth | | \-secret-worker-cloud | Required when related feature is enabled | Basic authentication secret which contains username and password for Spectra Intelligence authentication. Required when Spectra Intelligence is enabled (configuration.cloud.enabled). | Processor, Retry Processor, Preprocessor, Postprocessor, Receiver | | \-secret-worker-cloud-proxy | Required when related feature is enabled | Basic authentication secret which contains username and password for Spectra Intelligence Proxy authentication. Required when Spectra Intelligence Proxy is enabled (configuration.cloud.proxy.enabled). | Processor, Retry processor, Preprocessor, Postprocessor, Receiver, Cloud Cache | | \-secret-worker-aws | Required when related feature is enabled | Basic authentication secret which contains username and password for AWS authentication. Required if any type of S3 storage (File, SNS, Report, Unpacked) is enabled (configuration.s3.enabled, configuration.sns.enabled, configuration.reportS3.enabled, configuration.unpackedS3.enabled) | Postprocessor | | \-secret-worker-azure | Required when related feature is enabled | Basic authentication secret which contains username and password for Azure authentication. Required if any type of ADL storage (File, Report, Unpacked) is enabled (configuration.adl.enabled, configuration.reportAdl.enabled, configuration.unpackedAdl.enabled). | Postprocessor | | \-secret-worker-ms-graph | Required when related feature is enabled | Basic authentication secret which contains username and password for Microsoft Cloud Storage authentication. Required if any type of Microsoft Cloud storage (File, Report, Unpacked) is enabled (configuration.msGraph.enabled, configuration.reportMsGraph.enabled, configuration.unpackedMsGraph.enabled). | Postprocessor | | \-secret-unpacked-s3 | Optional | Secret which contains only the password. Used for encryption of the archive file. Relevant only when the configuration.unpackedS3.archiveUnpacked option is set to true. | Postprocessor | | \-secret-report-s3 | Optional | Secret which contains only the password. Used for encryption of the archive file. Relevant only when the configuration.reportS3.archiveSplitReport option is set to true. | Postprocessor | | \-secret-unpacked-adl | Optional | Secret which contains only the password. Used for encryption of the archive file. Relevant only when the configuration.unpackedAdl.archiveUnpacked option is set to true. | Postprocessor | | \-secret-report-adl | Optional | Secret which contains only the password. Used for encryption of the archive file. Relevant only when the configuration.reportAdl.archiveSplitReport option is set to true. | Postprocessor | | \-secret-unpacked-ms-graph | Optional | Secret which contains only the password. Used for encryption of the archive file. Relevant only when the configuration.unpackedMsGraph.archiveUnpacked option is set to true. | Postprocessor | | \-secret-report-ms-graph | Optional | Secret which contains only the password. Used for encryption of the archive file. Relevant only when the configuration.reportMsGraph.archiveSplitReport option is set to true. | Postprocessor | | \-secret-splunk | Optional | Token secret which contains the token for Splunk authentication. Relevant only if Splunk Integration is enabled (configuration.splunk.enabled). | Postprocessor | | \-secret-archive-zip | Optional | Secret which contains only the password. Relevant only when the configuration.archive.fileWrapper value is set to "zip" or "mzip". | Postprocessor | | \-secret-sa-integration-token | Required when related feature is enabled | Token secret which contains the token used in authentication on Spectra Analyze when Spectra Analyze Integration is enabled (configuration.spectraAnalyzeIntegration.enabled). This token should be created in Spectra Analyze. | Postprocessor | ### Configmap Configuration values for Worker pods | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | appliance.configMode | string | `"STANDARD"` | Configuration mode of the appliance. Allowed values: CONFIGMAP (Configuration is provided with configmap), STANDARD (configuration is provided over UI). | | configuration.a1000 | object | \- | Integration with Spectra Analyze appliance. | | configuration.a1000.host | string | `""` | The hostname or IP address of the A1000 appliance associated with the Worker. | | configuration.adl | object | \- | Settings for storing files in an Azure Data Lake container. | | configuration.adl.container | string | `""` | The hostname or IP address of the Azure Data Lake container that will be used for storage. Required when storing files in ADL is enabled. | | configuration.adl.enabled | bool | `false` | Enable or disable the storage of processed files. | | configuration.adl.folder | string | `""` | Specify the name of the folder on the container where files will be stored. | | configuration.apiServer | object | \- | Configures a custom Worker IP address which is included in the response when uploading a file to the Worker for processing. | | configuration.apiServer.host | string | `""` | Configures the hostname or IP address of the Worker. Only necessary if the default IP address or network interface is incorrect. | | configuration.archive | object | \- | After processing, files can be zipped before external storage. Available only for S3 and Azure. | | configuration.archive.fileWrapper | string | `""` | Specify whether the files should be compressed as a ZIP archive before uploading to external storage. Supported values are: zip, mzip. If this parameter is left blank, files will be uploaded in their original format. | | configuration.archive.zipCompress | int | `0` | ZIP compression level to use when storing files in a ZIP file. Allowed range: 0 (no compression) to 9 (maximum compression). | | configuration.archive.zipMaxfiles | int | `0` | Maximum allowed number of files that can be stored in one ZIP archive. Allowed range: 1-65535. 0 represents unlimited. | | configuration.authentication | object | \- | Authentication settings for Detect Worker | | configuration.authentication.enabled | bool | `false` | Enable/disable authentication on Detect Worker ingress APIs | | configuration.authentication.externalAuthUrl | string | `""` | If set, external/custom authentication service will be used for authentication, otherwise simple Token service is deployed which protects paths with tokens defined in the secrets. | | configuration.aws | object | \- | Configuration of integration with AWS or AWS-compatible storage to be used for SNS, and for uploading files and analysis reports to S3. | | configuration.aws.caPath | string | `""` | Path on the file system pointing to the certificate of a custom (self-hosted) S3 server. | | configuration.aws.endpointUrl | string | `""` | Only required in non-AWS setups in order to store files to an S3-compatible server. When this parameter is left blank, the default is `https://aws.amazonaws.com`. Supported pattern(s): `https?://.+"`. | | configuration.aws.maxReattempts | int | `5` | Maximum number of retries when saving a report to an S3-compatible server. | | configuration.aws.payloadSigningEnabled | bool | `false` | Specifies whether to include an SHA-256 checksum with Amazon Signature Version 4 payloads. | | configuration.aws.region | string | `"us-east-1"` | Specify the correct AWS geographical region where the S3 bucket is located. Required parameter, ignored for non-AWS setups. | | configuration.aws.serverSideEncryption | string | `""` | Specify the encryption algorithm used on the target S3 bucket (e.g. aws:kms or AES256). | | configuration.aws.sslVerify | bool | `false` | Enable/disable SSL verification. | | configuration.awsRole | object | \- | Configures the AWS IAM roles used to access S3 buckets without sharing secret keys. The IAM role which will be used to obtain temporary tokens has to be created in the AWS console. | | configuration.awsRole.enableArn | bool | `false` | Enables or disables this entire feature. | | configuration.awsRole.externalRoleId | string | `""` | The external ID of the role that will be assumed. This can be any string. Usually, it’s an ID provided by the entity which uses (but doesn’t own) an S3 bucket. The owner of that bucket takes that external ID and builds an ARN with it. | | configuration.awsRole.refreshBuffer | int | `5` | Number of seconds to fetch a new ARN token before the token timeout is reached. | | configuration.awsRole.roleArn | string | `""` | The role ARN created using the external role ID and an Amazon ID. In other words, the ARN which allows a Worker to obtain a temporary token, which then allows it to save to S3 buckets without a secret access key. | | configuration.awsRole.roleSessionName | string | `""` | Name of the session visible in AWS logs. Can be any string. | | configuration.awsRole.tokenDuration | int | `900` | How long before the authentication token expires and is refreshed. The minimum value is 900 seconds. | | configuration.azure | object | \- | Configures integration with Azure Data Lake Gen2 for the purpose of storing processed files in Azure Data Lake containers. | | configuration.azure.endpointSuffix | string | `"core.windows.net"` | Specify the suffix for the address of your Azure Data Lake container. | | configuration.callback | object | \- | Settings for automatically sending file analysis reports via POST request. | | configuration.callback.advancedFilterEnabled | bool | `false` | Enable/disable the advanced filter. | | configuration.callback.advancedFilterName | string | `""` | Name of the advanced filter. | | configuration.callback.caPath | string | `""` | If the url parameter is configured to use HTTPS, this parameter can be used to set the path to the certificate file. This automatically enables SSL verification. If this parameter is left blank or not configured, SSL verification will be disabled, and the certificate will not be validated. | | configuration.callback.enabled | bool | `false` | Enable/disable connection. | | configuration.callback.maliciousOnly | bool | `false` | When set, the report will only contain malicious and suspicious children. | | configuration.callback.reportType | string | `"medium"` | Specifies which report\_type is returned. By default, or when empty, only the medium (summary) report is provided in the callback response. Set to extended\_small, small, medium or large to view results of filtering the full report. | | configuration.callback.splitReport | bool | `false` | By default, reports contain information on parent files and all extracted children files. If set to true, reports for extracted files will be separated from the full report and saved as standalone files. If any user-defined data was appended to the analyzed parent file, it will be included in every split child report. | | configuration.callback.sslVerify | bool | `false` | Enable/disable SSL verification | | configuration.callback.timeout | int | `5` | Specify the number of seconds to wait before the POST request times out. In case of failure, the Worker will retry the request up to six times, increasing the waiting time between requests after the second retry has failed. With the default timeout set, the total possible waiting time before a request finally fails is 159 seconds. | | configuration.callback.topContainerOnly | bool | `false` | If set to true, the reports will only contain metadata for the top container. Reports for unpacked files will not be generated. | | configuration.callback.url | string | `""` | Specify the full URL that will be used to send the callback POST request. Both HTTP and HTTPS are supported. If this parameter is left blank, reports will not be sent, and the callback feature will be disabled. Supported pattern(s): http?://.+ | | configuration.callback.view | string | `""` | Specifies whether a custom report view should be applied to the report. | | configuration.cef | object | \- | Configures Common Event Format (CEF) settings. CEF is an extensible, text-based logging and auditing format that uses a standard header and a variable extension, formatted as key-value pairs. | | configuration.cef.cefMsgHashType | string | `"md5"` | Specify the type of hash that will be included in CEF messages. Supported values are: md5, sha1, sha256. | | configuration.cef.enableCefMsg | bool | `false` | Enable or disable sending CEF messages to syslog. Defaults to `false` to avoid flooding. | | configuration.classify | object | \- | Configure settings for Worker analysis and classification of files using the Spectra Core static analysis engine. | | configuration.classify.certificates | bool | `true` | Enable checking whether file certificate passes the certificate validation, in addition to checking certificate whitelists and blacklists. | | configuration.classify.documents | bool | `true` | Enable document format threat detection. | | configuration.classify.emails | bool | `true` | Enable detection of phishing and other email threats. | | configuration.classify.hyperlinks | bool | `true` | Enable embedded hyperlinks detection. | | configuration.classify.ignoreAdware | bool | `false` | When set to true, classification results that match adware will be ignored. | | configuration.classify.ignoreHacktool | bool | `false` | When set to true, classification results that match hacktool will be ignored. | | configuration.classify.ignorePacker | bool | `false` | When set to true, classification results that match packer will be ignored. | | configuration.classify.ignoreProtestware | bool | `false` | When set to true, classification results that match protestware will be ignored. | | configuration.classify.ignoreRiskware | bool | `false` | When set to true, classification results that match riskware will be ignored. | | configuration.classify.ignoreSpam | bool | `false` | When set to true, classification results that match spam will be ignored. | | configuration.classify.ignoreSpyware | bool | `false` | When set to true, classification results that match spyware will be ignored. | | configuration.classify.images | bool | `true` | When true, the heuristic image classifier for supported file formats is used. | | configuration.classify.pecoff | bool | `true` | When true, the heuristic Windows executable classifier for supported PE file formats is used. | | configuration.cleanup | object | \- | Configures how often the Worker file system is cleaned up. | | configuration.cleanup.fileAgeLimit | int | `1440` | Time before an unprocessed file present on the appliance is deleted, in minutes. | | configuration.cleanup.taskAgeLimit | int | `90` | Time before analysis reports and records of processed tasks are deleted, in minutes. | | configuration.cleanup.taskUnprocessedLimit | int | `1440` | Time before an incomplete processing task is canceled, in minutes. | | configuration.cloud | object | \- | Configures integration with the Spectra Intelligence service or a T1000 instance to receive additional classification information. | | configuration.cloud.enabled | bool | `false` | Enable/disable connection. | | configuration.cloud.proxy | object | \- | Configure an optional proxy connection. | | configuration.cloud.proxy.enabled | bool | `false` | Enable/disable proxy server. | | configuration.cloud.proxy.port | int | `8080` | Specify the TCP port number if using an HTTP proxy. Allowed range(s): 1 … 65535\. Required only if proxy is used. | | configuration.cloud.proxy.server | string | `""` | Proxy hostname or IP address for routing requests from the appliance to Spectra Intelligence. Required only if proxy is used. | | configuration.cloud.server | string | `"https://appliance-api.reversinglabs.com"` | Hostname or IP address of the Spectra Intelligence server. Required if Spectra Intelligence integration is enabled. Format: `https://`. | | configuration.cloud.timeout | int | `6` | Specify the number of seconds to wait when connecting to Spectra Intelligence before terminating the connection request. | | configuration.cloudAutomation | object | \- | Configures the Worker to automatically submit files to Spectra Intelligence for antivirus scanning (in addition to local static analysis and remote reputation lookup (from previous antivirus scans)). | | configuration.cloudAutomation.dataChangeSubscribe | bool | `false` | Subscribe to the Spectra Intelligence data change notification mechanism. | | configuration.cloudAutomation.spexUpload | object | \- | Scanning settings. | | configuration.cloudAutomation.spexUpload.enabled | bool | `false` | Enable/disable this feature. | | configuration.cloudAutomation.spexUpload.rescanEnabled | bool | `true` | Enable/disable rescan of files upon submission based on the configured interval to include the latest AV results in the reports. | | configuration.cloudAutomation.spexUpload.rescanThresholdInDays | int | `3` | Set the interval in days for triggering an AV rescan. If the last scan is older than the specified value, a rescan will be initiated. A value of 0 means files will be rescanned with each submission. | | configuration.cloudAutomation.spexUpload.scanUnpackedFiles | bool | `false` | Enable/disable sending unpacked files to Deep Cloud Analysis for scanning. Consumes roughly double the processing resources compared to standard analysis. | | configuration.cloudAutomation.waitForAvScansTimeoutInMinutes | int | `240` | Sets the maximum wait time (in minutes) for Deep Cloud Analysis to complete. If the timeout is reached, the report will be generated without the latest AV results. | | configuration.cloudAutomation.waitForAvScansToFinish | bool | `false` | If set to true, delays report generation until Deep Cloud Analysis completes, ensuring the latest AV results are included. | | configuration.cloudCache.cacheMaxSizePercentage | float | `6.25` | Maximum cache size expressed as a percentage of the total allocated RAM on the Worker. Allowed range: 5 \- 15\. | | configuration.cloudCache.cleanupWindow | int | `10` | How often to run the cache cleanup process, in minutes. It is advisable for this value to be lower, or at least equal to the TTL value. Max: 5 \- 60\. | | configuration.cloudCache.enabled | bool | `true` | Enable or disable the caching feature. | | configuration.cloudCache.maxIdleUpstreamConnections | int | `50` | The maximum number of idle upstream connections. Allowed range: 10 \- 50\. | | configuration.cloudCache.ttl | int | `240` | Time to live for cached records, in minutes. Allowed range: 1 \- 7200\. | | configuration.general.maxUploadSizeMb | int | `2048` | The largest file (in MB) that Worker will accept and start processing. Ignored if Spectra Intelligence is connected and file upload limits are set there. | | configuration.general.postprocessingCheckThresholdMins | int | `720` | How often the postprocessing service will be checked for timeouts. If any issues are detected, the process will be restarted. | | configuration.general.tsWorkerCheckThresholdMins | int | `720` | How often the processing service will be checked for timeouts. If any issues are detected, the process will be restarted. | | configuration.general.uploadSizeLimitEnabled | bool | `false` | Whether or not the upload size filter is active. Ignored if Spectra Intelligence is connected and file upload limits are set there. | | configuration.hashes | object | \- | Spectra Core calculates file hashes during analysis and includes them in the analysis report. The following options configure which additional hash types should be calculated and included in the Worker report. SHA1 and SHA256 are always included and therefore aren’t configurable. Selecting additional hash types (especially SHA384 and SHA512) may slow report generation. | | configuration.hashes.enableCrc32 | bool | `false` | Include CRC32 hashes in reports. | | configuration.hashes.enableMd5 | bool | `true` | Include MD5 hashes in reports. | | configuration.hashes.enableSha384 | bool | `false` | Include SHA384 hashes in reports. | | configuration.hashes.enableSha512 | bool | `false` | Include SHA512 hashes in reports. | | configuration.hashes.enableSsdeep | bool | `false` | Include SSDEEP hashes in reports. | | configuration.hashes.enableTlsh | bool | `false` | Include TLSH hashes in reports. | | configuration.health | object | `-` | Configures system health check configuration. | | configuration.health.disk\_high | int | `95` | Threshold for high disk usage | | configuration.health.enabled | bool | `true` | Enable/disable system health check. | | configuration.health.queue\_high | int | `2000` | Specify the maximum number of items allowed in the queue. If it exceeds the configured value, the appliance will start rejecting traffic. Allowed range(s): 10+ | | configuration.logging | object | \- | Configures the severity above which events will be logged or sent to a remote syslog server. Severity can be: INFO, WARNING, or ERROR. | | configuration.logging.tiscaleLogLevel | string | `"INFO"` | Events below this level will not be saved to logs (/var/log/messages and /var/log/tiscale/\*.log). | | configuration.msGraph.enabled | bool | `false` | Turns the Microsoft Cloud Storage file integration on or off. | | configuration.msGraph.folder | string | `""` | Folder where samples will be stored in Microsoft Cloud Storage. | | configuration.msGraphGeneral | object | \- | Configures the general options for the Microsoft Cloud Storage integration. | | configuration.msGraphGeneral.customDomain | string | `""` | Application’s custom domain configured in the Azure portal. | | configuration.msGraphGeneral.siteHostname | string | `""` | Used only if `storageType` is set to SharePoint. This is the SharePoint hostname. | | configuration.msGraphGeneral.siteRelativePath | string | `""` | SharePoint Online site relative path. Only used when `storageType` is set to SharePoint. | | configuration.msGraphGeneral.storageType | string | `"onedrive"` | Specifies the storage type. Supported values are: onedrive or sharepoint. | | configuration.msGraphGeneral.username | string | `""` | Used only if `storageType` is set to OneDrive. Specifies which user’s drive will be used." | | configuration.processing | object | \- | Configure the Worker file processing capabilities to improve performance and load balancing. | | configuration.processing.cacheEnabled | bool | `false` | Enable/disable caching. When enabled, Spectra Core can skip reprocessing the same files (duplicates) if uploaded consecutively in a short period. | | configuration.processing.cacheTimeToLive | int | `0` | If file processing caching is enabled, specify how long (in seconds) the analysis reports should be preserved in the cache before they expire. A value of 0 uses the default. Default: 600\. Maximum: 86400\. | | configuration.processing.depth | int | `0` | Specifies how "deep" a file is unpacked. By default, when set to 0, Workers will unpack files recursively until no more files can be unpacked. Setting a value greater than 0 limits the depth of recursion, which can speed up analyses but provide less detail. | | configuration.processing.largefileThreshold | int | `100` | If advanced mode is enabled, files larger than this threshold (in MB) will be processed individually, one by one. This parameter is ignored in standard mode. | | configuration.processing.mode | int | `2` | Configures the Worker processing mode to improve load balancing. Supported modes are standard (1) and advanced (2). | | configuration.processing.timeout | int | `28800` | Specifies how many seconds the Worker should wait for a file to process before terminating the task. Default: 28800\. Maximum: 259200\. | | configuration.propagation | object | \- | Configure advanced classification propagation options supported by the Spectra Core static analysis engine. When Spectra Core classifies files, the classification of a child file can be applied to the parent file. | | configuration.propagation.enabled | bool | `true` | Enable/disable the classification propagation feature. When propagation is enabled, files can be classified based on the content extracted from them. This means that files containing a malicious or suspicious file will also be considered malicious or suspicious. | | configuration.propagation.goodwareOverridesEnabled | bool | `true` | Enable/disable goodware overrides. When enabled, any files extracted from a parent file and whitelisted by certificate, source or user override can no longer be classified as malicious or suspicious. This is an advanced goodware whitelisting technique that can be used to reduce the amount of false positive detections. | | configuration.propagation.goodwareOverridesFactor | int | `1` | When goodware overrides are enabled, this parameter must be configured to determine the factor to which overrides will be applied. Supported values are 0 to 5, where zero represents the best trust factor (highest confidence that a sample contains goodware). Overrides will apply to files with a trust factor equal to or lower than the value configured here. | | configuration.report | object | \- | Configure the contents of the Spectra Detect file analysis report. | | configuration.report.firstReportOnly | bool | `false` | If disabled, the reports for samples with child files will include relationships for all descendant files. Enabling this setting will only include relationship metadata for the root parent file to reduce redundancy. | | configuration.report.includeStrings | bool | `false` | When enabled, strings are included in the file analysis report. Spectra Core can extract strings from binaries. This can be useful but may result in extensive metadata. To reduce noise, the types of included strings can be customized in the strings section. | | configuration.report.networkReputation | bool | `false` | If enabled, analysis reports include a top-level `network_reputation` object with reputation information for every extracted network resource. For this feature, Spectra Intelligence must be configured on the Worker, and the `ticore.processingMode` option must be set to "best". | | configuration.report.relationships | bool | `false` | Includes sample relationship metadata in the file analysis report. When enabled, the relationships section lists the hashes of files found within the given file. | | configuration.reportAdl | object | \- | Settings to configure how reports saved to Azure Data Lake are formatted. | | configuration.reportAdl.archiveSplitReport | bool | `true` | Enable sending a single, smaller archive of split report files to ADL instead of each file. Relevant only when the 'Split report' option is used. | | configuration.reportAdl.container | string | `""` | Container where reports will be stored. Required when this feature is enabled. | | configuration.reportAdl.enabled | bool | `false` | Enable/disable storing file processing reports to ADL. | | configuration.reportAdl.filenameTimestampFormat | string | `""` | File naming pattern for the report itself. A timestamp is appended to the SHA1 hash of the file. The timestamp format must follow the strftime specification and be enclosed in quotation marks. If not specified, the ISO 8601 format is used. | | configuration.reportAdl.folder | string | `""` | Specify the name of a folder where analysis reports will be stored. If the folder name is not provided, files are stored into the root of the configured container. | | configuration.reportAdl.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analysis reports. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.reportAdl.maliciousOnly | bool | `false` | When set, the report will only contain malicious and suspicious children. | | configuration.reportAdl.reportType | string | `"large"` | Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Report types are stored in the /etc/ts-report/report-types directory. | | configuration.reportAdl.splitReport | bool | `false` | By default, reports contain information on parent files and all extracted children files. When this option is enabled, analysis reports for extracted files are separated from their parent file report, and saved as individual report files. | | configuration.reportAdl.timestampEnabled | bool | `true` | Enable/disable appending a timestamp to the report name. | | configuration.reportAdl.topContainerOnly | bool | `false` | When enabled, the file analysis report will only include metadata for the top container and subreports for unpacked files will not be generated. | | configuration.reportAdl.view | string | `""` | Apply a view for transforming report data to the “large” report type to ensure maximum compatibility. Several existing views are also available as report types, which should be used as a view substitute due to performance gains. Custom views can be defined by placing the scripts in the “/usr/libexec/ts-report-views.d” directory on Spectra Detect Worker. | | configuration.reportApi | object | \- | Configures the settings applied to the file analysis report fetched using the GET endpoint. | | configuration.reportApi.maliciousOnly | bool | `false` | Report contains only malicious and suspicious children. | | configuration.reportApi.reportType | string | `"large"` | Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Report types are stored in the /etc/ts-report/report-types directory. | | configuration.reportApi.topContainerOnly | bool | `false` | When enabled, thefile analysis report will only include metadata for the top container and subreports for unpacked files will not be generated. | | configuration.reportApi.view | string | `""` | Apply a view for transforming report data to the “large” report type to ensure maximum compatibility. Several existing views are also available as report types, which should be used as a view substitute due to performance gains. Custom views can be defined by placing the scripts in the “/usr/libexec/ts-report-views.d” directory on Spectra Detect Worker. | | configuration.reportMsGraph | object | \- | Settings to configure how reports saved to OneDrive or SharePoint are formatted. | | configuration.reportMsGraph.archiveSplitReport | bool | `true` | Enable sending a single, smaller archive of split report files to Microsoft Cloud Storage instead of each file. Relevant only when the "Split Report" option is used. | | configuration.reportMsGraph.enabled | bool | `false` | Enable/disable storing file processing reports. | | configuration.reportMsGraph.filenameTimestampFormat | string | `""` | This refers to the naming of the report file itself. A timestamp is appended to the SHA1 hash of the file. The timestamp format must follow the strftime specification and be enclosed in quotation marks. If not specified, the ISO 8601 format is used. | | configuration.reportMsGraph.folder | string | `""` | Folder where report files will be stored on the Microsoft Cloud Storage. If the folder name is not provided, files are stored into the root of the configured container. | | configuration.reportMsGraph.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analysis reports. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.reportMsGraph.maliciousOnly | bool | `false` | When set, the report will only contain malicious and suspicious children. | | configuration.reportMsGraph.reportType | string | `"large"` | Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Report types are stored in the /etc/ts-report/report-types directory. | | configuration.reportMsGraph.splitReport | bool | `false` | By default, reports contain information on parent files and all extracted children files. When this option is enabled, analysis reports for extracted files are separated from their parent file report, and saved as individual report files. | | configuration.reportMsGraph.topContainerOnly | bool | `false` | When enabled, file analysis report will only include metadata for the top container, and subreports for unpacked files will not be generated. | | configuration.reportMsGraph.view | string | `""` | Apply a view for transforming report data to the “large” report type to ensure maximum compatibility. Several existing views are also available as report types, which should be used as a view substitute due to performance gains. Custom views can be defined by placing the scripts in the “/usr/libexec/ts-report-views.d” directory on Spectra Detect Worker. | | configuration.reportS3 | object | \- | Settings to configure how reports saved to S3 buckets are formatted. | | configuration.reportS3.advancedFilterEnabled | bool | `false` | Enable/disable usage of the advanced filter. | | configuration.reportS3.advancedFilterName | string | `""` | Name of the advanced filter. | | configuration.reportS3.archiveSplitReport | bool | `true` | Enable sending a single, smaller archive of split report files to S3 instead of each file. Relevant only when the 'Split report' option is used. | | configuration.reportS3.bucketName | string | `""` | Name of the S3 bucket where processed files will be stored. Required when this feature is enabled. | | configuration.reportS3.enabled | bool | `false` | Enable/disable storing file processing reports to S3. | | configuration.reportS3.filenameTimestampFormat | string | `""` | This refers to the naming of the report file itself. A timestamp is appended to the SHA1 hash of the file. The timestamp format must follow the strftime specification and be enclosed in quotation marks. If not specified, the ISO 8601 format is used. | | configuration.reportS3.folder | string | `""` | Folder where report files will be stored in the given S3 bucket. The folder can be up to 1024 bytes long when encoded in UTF-8, and can contain letters, numbers and special characters: "\!", "-", "\_", ".", "\*", "'", "(", ")", "/". It must not start or end with a slash or contain leading or trailing spaces. Consecutive slashes ("//") are not allowed. | | configuration.reportS3.folderOption | string | `"date_based"` | Select the naming pattern used when automatically creating subfolders for storing analysis reports. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.reportS3.maliciousOnly | bool | `false` | When set, the report will only contain malicious and suspicious children. | | configuration.reportS3.reportType | string | `"large"` | Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Report types are stored in the /etc/ts-report/report-types directory. | | configuration.reportS3.splitReport | bool | `false` | By default, reports contain information on parent files and all extracted children files. When this option is enabled, analysis reports for extracted files are separated from their parent file report, and saved as individual report files. | | configuration.reportS3.timestampEnabled | bool | `true` | Enable/disable appending a timestamp to the report name. | | configuration.reportS3.topContainerOnly | bool | `false` | When enabled, the file analysis report will only include metadata for the top container and subreports for unpacked files will not be generated. | | configuration.reportS3.view | string | `""` | Apply a view for transforming report data to the “large” report type to ensure maximum compatibility. Several existing views are also available as report types, which should be used as a view substitute due to performance gains. Custom views can be defined by placing the scripts in the “/usr/libexec/ts-report-views.d” directory on Spectra Detect Worker. | | configuration.s3 | object | \- | Settings for storing a copy of all files uploaded for analysis on Worker to an S3 or a third-party, S3-compatible server. | | configuration.s3.advancedFilterEnabled | bool | `false` | Enable/disable usage of the advanced filter. | | configuration.s3.advancedFilterName | string | `""` | Name of the advanced filter. | | configuration.s3.bucketName | string | `""` | Name of the S3 bucket where processed files will be stored. Required when this feature is enabled. | | configuration.s3.enabled | bool | `false` | Enable/disable storing file processed files on S3. | | configuration.s3.folder | string | `""` | Specify the name of a folder where analyzed files will be stored. If the folder name is not provided, files are stored into the root of the configured bucket. | | configuration.s3.storeMetadata | bool | `true` | When true, analysis metadata will be stored to the uploaded S3 object. | | configuration.scaling | object | \- | Configures the number of concurrent processes and the number of files analyzed concurrently. Parameters in this section can be used to optimize the file processing performance on Worker. | | configuration.scaling.postprocessing | int | `1` | Specify how many post-processing instances to run. Post-processing instances will then modify and save reports or upload processed files to external storage. Increasing this value can increase throughput for servers with extra available cores. Maximum: 256\. | | configuration.scaling.preprocessingUnpacker | int | `1` | Specify how many copies of Spectra Core are used to unpack samples for Deep Cloud Analysis. This setting only has effect if Deep Cloud Analysis is enabled with Scan Unpacked Files capability. | | configuration.scaling.processing | int | `1` | Specify how many copies of Spectra Core engine instances to run. Each instance starts threads to process files. Maximum: 256\. | | configuration.sns | object | \- | Configures settings for publishing notifications about file processing status and links the reports to an Amazon SNS (Simple Notification Service) topic. | | configuration.sns.enabled | bool | `false` | Enable/disable publishing notifications to Amazon SNS. | | configuration.sns.topic | string | `""` | Specify the SNS topic ARN that the notifications should be published to. Prerequisite: the AWS account in the AWS settings must be given permission to publish to this topic. Required when this feature is enabled. | | configuration.spectraAnalyzeIntegration | object | \- | Configuration settings to upload processed samples to configured Spectra Analyze. | | configuration.spectraAnalyzeIntegration.address | string | `""` | Spectra Analyze address. Required when this feature is enabled. Has to be in the following format: https://\. | | configuration.spectraAnalyzeIntegration.advancedFilterEnabled | bool | `true` | Enable/disable the advanced filter. | | configuration.spectraAnalyzeIntegration.advancedFilterName | string | `"default_filter"` | Name of the advanced filter. | | configuration.spectraAnalyzeIntegration.enabled | bool | `false` | Enable/disable integration with Spectra Analyze. | | configuration.splunk | object | \- | Configures integration with Splunk, a logging server that can receive Spectra Detect file analysis reports. | | configuration.splunk.caPath | string | `""` | Path to the certificate. | | configuration.splunk.chunkSizeMb | int | `0` | The maximum size (MB) of a single request sent to Splunk. If an analysis report exceeds this size, it will be split into multiple parts. The report is split into its subreports (for child files). A request can contain one or multiple subreports, as long as its total size doesn’t exceed this limit. The report is never split by size alone \- instead, complete subreports are always preserved and sent to Splunk. Default: 0 (disabled) | | configuration.splunk.enabled | bool | `false` | Enable/disable Splunk integration. | | configuration.splunk.host | string | `""` | Specify the hostname or IP address of the Splunk server that should connect to the Worker appliance. | | configuration.splunk.https | bool | `true` | If set to true, HTTPS will be used for sending information to Splunk. If set to false, HTTP is used. | | configuration.splunk.port | int | `8088` | Specify the TCP port of the Splunk server’s HTTP Event Collector. | | configuration.splunk.reportType | string | `"large"` | Specifies which report\_type is returned. By default or when empty, only the medium (summary) report is provided in the callback response. Set to small, medium or large to view results of filtering the full report. | | configuration.splunk.sslVerify | bool | `false` | If HTTPS is enabled, setting this to true will enable certificate verification." | | configuration.splunk.timeout | int | `5` | Specify how many seconds to wait for a response from the Splunk server before the request fails. If the request fails, the report will not be uploaded to the Splunk server, and an error will be logged. The timeout value must be greater than or equal to 1, and not greater than 999\. | | configuration.splunk.topContainerOnly | bool | `false` | Whether or not Splunk should receive the report for the top (parent) file only. If set to true, no subreports will be sent. | | configuration.splunk.view | string | `""` | Specifies whether a custom Report View should be applied to the file analysis report and returned in the response. | | configuration.strings | object | \- | Configure the output of strings extracted from files during Spectra Core static analysis. | | configuration.strings.enableStringExtraction | bool | `false` | If set to true, user-provided criteria for string extraction will be used. | | configuration.strings.maxLength | int | `32768` | Maximum number of characters in strings. | | configuration.strings.minLength | int | `4` | Minimum number of characters in strings. Strings shorter than this value are not extracted. | | configuration.strings.unicodePrintable | bool | `false` | Specify whether strings are Unicode printable or not. | | configuration.strings.utf16be | bool | `true` | Allow/disallow extracting UTF-16BE strings. | | configuration.strings.utf16le | bool | `true` | Allow/disallow extracting UTF-16LE strings. | | configuration.strings.utf32be | bool | `false` | Allow/disallow extracting UTF-32BE strings. | | configuration.strings.utf32le | bool | `false` | Allow/disallow extracting UTF-32LE strings. | | configuration.strings.utf8 | bool | `true` | Allow/disallow extracting UTF-8 strings. | | configuration.ticore | object | \- | Configures options supported by Spectra Core. | | configuration.ticore.maxDecompressionFactor | float | `1.0` | Decimal value between 0 and 999.9. If multiple decimals are given, it will be rounded to one decimal. Used to protect the user from intentional or unintentional archive bombs, terminating decompression if size of unpacked content exceeds a set quota. | | configuration.ticore.mwpExtended | bool | `false` | Enable/disable information from antivirus engines in Spectra Intelligence. Requires Spectra Intelligence to be configured | | configuration.ticore.mwpGoodwareFactor | int | `2` | Determines when a file classified as KNOWN in Spectra Intelligence Cloud is classified as Goodware by Spectra Core. By default, all KNOWN cloud classifications are converted to Goodware. Supported values are 0 \- 5, where zero represents the best trust factor (highest confidence that a sample contains goodware). Lowering the value reduces the number of samples classified as goodware. Samples with a trust factor above the configured value are considered UNKNOWN. Requires Spectra Intelligence to be configured | | configuration.ticore.processingMode | string | `"best"` | Determines which file formats are unpacked by Spectra Core for detailed analysis. "best" fully processes all supported formats; "fast" processes a limited set. | | configuration.ticore.useXref | bool | `false` | Enabling XREF service will enrich analysis reports with cross-reference metadata like AV scanner results. Requires Spectra Intelligence to be configured | | configuration.unpackedAdl | object | \- | Settings for storing extracted files in an Azure Data Lake container. | | configuration.unpackedAdl.archiveUnpacked | bool | `true` | Enable sending a single, smaller archive of unpacked files to ADL instead of each unpacked file. | | configuration.unpackedAdl.container | string | `""` | Specify the name of the Azure Data Lake container where extracted files will be saved. Required when this feature is enabled. | | configuration.unpackedAdl.enabled | bool | `false` | Enable/disable storing extracted files to ADL. | | configuration.unpackedAdl.folder | string | `""` | Specify the name of a folder in the configured Azure container where extracted files will be stored. If the folder name is not provided, files are stored into the root of the configured container. | | configuration.unpackedAdl.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analyzed files. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.unpackedMsGraph | object | \- | Settings for storing extracted files to Microsoft Cloud Storage. | | configuration.unpackedMsGraph.archiveUnpacked | bool | `true` | Enable sending a single, smaller archive of unpacked files to Microsoft Cloud Storage instead of each unpacked file. | | configuration.unpackedMsGraph.enabled | bool | `false` | Enable/disable storing extracted files. | | configuration.unpackedMsGraph.folder | string | `""` | Folder where unpacked files will be stored on the Microsoft Cloud Storage. If the folder name is not provided, files are stored into the root of the configured container. | | configuration.unpackedMsGraph.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analyzed files. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.unpackedS3 | object | \- | Settings for storing extracted files to S3 container. | | configuration.unpackedS3.advancedFilterEnabled | bool | `false` | Enable/disable the use of advanced filters. | | configuration.unpackedS3.advancedFilterName | string | `""` | Name of the advanced filter. | | configuration.unpackedS3.archiveUnpacked | bool | `true` | Enable sending a single, smaller archive of unpacked files to S3 instead of each unpacked file. | | configuration.unpackedS3.bucketName | string | `""` | Specify the name of the S3 container where extracted files will be saved. Required when this feature is enabled. | | configuration.unpackedS3.enabled | bool | `false` | Enable/disable storing extracted files in S3. | | configuration.unpackedS3.folder | string | `""` | The name of a folder in the configured S3 container where extracted files will be stored. If the folder name is not provided, files are stored into the root of the configured container. The folder can be up to 1024 bytes long when encoded in UTF-8, and can contain letters, numbers and special characters: "\!", "-", "\_", ".", "\*", "'", "(", ")", "/". It must not start or end with a slash or contain leading or trailing spaces. Consecutive slashes ("//") are not allowed. | | configuration.unpackedS3.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analyzed files. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.wordlist | list | \- | List of passwords for protected files. | ### Other Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | advancedFilters | object | `{}` | Contains key-value pairs in which keys are filter names and values are the filter definitions. | | auth.image.pullPolicy | string | `"Always"` | | | auth.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-detect-auth"` | | | auth.image.tag | string | `"latest-dev"` | | | auth.resources.limits.cpu | string | `"4000m"` | | | auth.resources.limits.memory | string | `"256Mi"` | | | auth.resources.requests.cpu | string | `"500m"` | | | auth.resources.requests.memory | string | `"128Mi"` | | | auth.serverPort | int | `8080` | | | authReverseProxy.image.pullPolicy | string | `"Always"` | | | authReverseProxy.image.repository | string | `"nginx"` | | | authReverseProxy.image.tag | string | `"stable"` | | | authReverseProxy.resources.limits.cpu | string | `"2000m"` | | | authReverseProxy.resources.limits.memory | string | `"512Mi"` | | | authReverseProxy.resources.requests.cpu | string | `"250m"` | | | authReverseProxy.resources.requests.memory | string | `"128Mi"` | | | authReverseProxy.serverPort | int | `80` | | | cleanup.failedJobsHistoryLimit | int | `1` | Number of failed finished jobs to keep. | | cleanup.image.pullPolicy | string | `"Always"` | | | cleanup.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-detect-utilities"` | | | cleanup.image.tag | string | `"latest-dev"` | | | cleanup.resources.limits.cpu | string | `"2000m"` | | | cleanup.resources.limits.memory | string | `"2Gi"` | | | cleanup.resources.requests.cpu | string | `"1000m"` | | | cleanup.resources.requests.memory | string | `"1Gi"` | | | cleanup.startingDeadlineSeconds | int | `180` | Deadline (in seconds) for starting the Job, if that Job misses its scheduled time for any reason. After missing the deadline, the CronJob skips that instance of the Job. | | cleanup.successfulJobsHistoryLimit | int | `1` | Number of successful finished jobs to keep. | | cloudCache.image.pullPolicy | string | `"Always"` | | | cloudCache.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-cloud-cache"` | | | cloudCache.image.tag | string | `"latest-dev"` | | | cloudCache.resources.limits.cpu | string | `"4000m"` | | | cloudCache.resources.limits.memory | string | `"4Gi"` | | | cloudCache.resources.requests.cpu | string | `"1000m"` | | | cloudCache.resources.requests.memory | string | `"1Gi"` | | | cloudCache.serverPort | int | `8080` | | | global.umbrella | bool | `false` | | | imagePullSecrets\[0\] | string | `"rl-registry-key"` | | | ingress.annotations | object | `{}` | | | ingress.className | string | `"nginx"` | | | ingress.enabled | bool | `true` | | | ingress.host | string | `""` | | | ingress.paths\[0\].path | string | `"/"` | | | ingress.paths\[0\].pathType | string | `"Prefix"` | | | ingress.tls.certificateArn | string | `""` | | | ingress.tls.issuer | string | `""` | | | ingress.tls.issuerKind | string | `"Issuer"` | | | ingress.tls.secretName | string | `"tls-tiscale-worker"` | | | monitoring.enabled | bool | `false` | Enable/disable monitoring with Prometheus | | monitoring.prometheusReleaseName | string | `"kube-prometheus-stack"` | Prometheus release name | | persistence | object | \- | Persistence values configure options for `PersistentVolumeClaim` used for storing samples and reports | | persistence.accessModes | list | `["ReadWriteMany"]` | Access mode. When autoscaling or multiple worker is used should be set to `[ "ReadWriteMany" ]` | | persistence.requestStorage | string | `"10Gi"` | Request Storage | | persistence.storageClassName | string | `nil` | Storage class name. When autoscaling or multiple worker is used storage class should support "ReadWriteMany" | | postgres.releaseName | string | `""` | Postgres release name, required when deployment is not done with the umbrella Helm chart. | | postprocessor.autoscaling.enabled | bool | `false` | Enable/disable autoscaling | | postprocessor.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled | | postprocessor.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed | | postprocessor.autoscaling.pollingInterval | int | `10` | Interval to check each trigger. In seconds. | | postprocessor.autoscaling.scaleDown | object | `{"stabilizationWindow":180}` | ScaleDown configuration values | | postprocessor.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | postprocessor.autoscaling.scaleUp | object | `{"numberOfPods":1,"period":30,"stabilizationWindow":15}` | ScaleUp configuration values | | postprocessor.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period | | postprocessor.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied | | postprocessor.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | postprocessor.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. | | postprocessor.image.pullPolicy | string | `"Always"` | | | postprocessor.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-detect-postprocessor"` | | | postprocessor.image.tag | string | `"latest-dev"` | | | postprocessor.resources.limits.cpu | string | `"8000m"` | | | postprocessor.resources.limits.memory | string | `"16Gi"` | | | postprocessor.resources.requests.cpu | string | `"2500m"` | | | postprocessor.resources.requests.memory | string | `"2Gi"` | | | preprocessor.autoscaling.enabled | bool | `false` | Enable/disable autoscaling | | preprocessor.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled | | preprocessor.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed | | preprocessor.autoscaling.pollingInterval | int | `10` | Interval to check each trigger. In seconds. | | preprocessor.autoscaling.scaleDown | object | `{"stabilizationWindow":180}` | ScaleDown configuration values | | preprocessor.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | preprocessor.autoscaling.scaleUp | object | `{"numberOfPods":1,"period":30,"stabilizationWindow":15}` | ScaleUp configuration values | | preprocessor.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period | | preprocessor.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied | | preprocessor.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | preprocessor.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. | | preprocessor.image.pullPolicy | string | `"Always"` | | | preprocessor.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-detect-preprocessor"` | | | preprocessor.image.tag | string | `"latest-dev"` | | | preprocessor.replicaCount | int | `1` | | | preprocessor.resources.limits.cpu | string | `"4000m"` | | | preprocessor.resources.limits.memory | string | `"4Gi"` | | | preprocessor.resources.requests.cpu | string | `"1000m"` | | | preprocessor.resources.requests.memory | string | `"1Gi"` | | | preprocessorUnpacker.autoscaling.enabled | bool | `false` | Enable/disable autoscaling | | preprocessorUnpacker.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled | | preprocessorUnpacker.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed | | preprocessorUnpacker.autoscaling.pollingInterval | int | `10` | Interval to check each trigger. In seconds. | | preprocessorUnpacker.autoscaling.scaleDown | object | `{"stabilizationWindow":180}` | ScaleDown configuration values | | preprocessorUnpacker.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | preprocessorUnpacker.autoscaling.scaleUp | object | `{"numberOfPods":1,"period":30,"stabilizationWindow":15}` | ScaleUp configuration values | | preprocessorUnpacker.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period | | preprocessorUnpacker.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied | | preprocessorUnpacker.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | preprocessorUnpacker.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. | | preprocessorUnpacker.replicaCount | int | `1` | | | preprocessorUnpacker.resources.limits.cpu | string | `"16000m"` | | | preprocessorUnpacker.resources.limits.memory | string | `"16Gi"` | | | preprocessorUnpacker.resources.requests.cpu | string | `"4000m"` | | | preprocessorUnpacker.resources.requests.memory | string | `"4Gi"` | | | preprocessorUnpacker.scaling.prefetchCount | int | `4` | Defines the maximum number of individual files that can simultaneously be processed by a single instance of Spectra Core. When one file is processed, another from the queue enters the processing state. Value 0 sets the maximum number of files to be processed to the number of CPU cores on the system. | | processor.autoscaling.enabled | bool | `false` | Enable/disable autoscaling | | processor.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled | | processor.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed | | processor.autoscaling.pollingInterval | int | `10` | Interval to check each trigger. In seconds. | | processor.autoscaling.scaleDown | object | `{"stabilizationWindow":180}` | ScaleDown configuration values | | processor.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | processor.autoscaling.scaleUp | object | `{"numberOfPods":1,"period":30,"stabilizationWindow":15}` | ScaleUp configuration values | | processor.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period | | processor.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied | | processor.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | processor.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. | | processor.image.pullPolicy | string | `"Always"` | | | processor.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-processor"` | | | processor.image.tag | string | `"latest-dev"` | | | processor.replicaCount | int | `1` | | | processor.resources.limits.cpu | string | `"16000m"` | | | processor.resources.limits.memory | string | `"32Gi"` | | | processor.resources.requests.cpu | string | `"4000m"` | | | processor.resources.requests.memory | string | `"4Gi"` | | | processor.scaling.prefetchCount | int | `8` | Defines the maximum number of individual files that can simultaneously be processed by a single instance of Spectra Core. When one file is processed, another from the queue enters the processing state. Value 0 sets the maximum number of files to be processed to the number of CPU cores on the system. | | processorRetry.autoscaling.enabled | bool | `false` | Enable/disable autoscaling | | processorRetry.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled | | processorRetry.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed | | processorRetry.autoscaling.pollingInterval | int | `10` | Interval to check each trigger. In seconds. | | processorRetry.autoscaling.scaleDown | object | `{"stabilizationWindow":180}` | ScaleDown configuration values | | processorRetry.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | processorRetry.autoscaling.scaleUp | object | `{"numberOfPods":1,"period":30,"stabilizationWindow":15}` | ScaleUp configuration values | | processorRetry.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period | | processorRetry.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied | | processorRetry.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | processorRetry.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. | | processorRetry.replicaCount | int | `1` | | | processorRetry.resources.limits.cpu | string | `"16000m"` | | | processorRetry.resources.limits.memory | string | `"64Gi"` | | | processorRetry.resources.requests.cpu | string | `"4000m"` | | | processorRetry.resources.requests.memory | string | `"8Gi"` | | | processorRetry.scaling.prefetchCount | int | `1` | Defines the maximum number of individual files that can simultaneously be processed by a single instance of Spectra Core. When one file is processed, another from the queue enters the processing state. Value 0 sets the maximum number of files to be processed to the number of CPU cores on the system. Recommended value for this type of processor is 1\. | | rabbitmq.releaseName | string | `""` | Rabbitmq release name, required when deployment is not done using the umbrella Helm chart. | | receiver.autoscaling.enabled | bool | `false` | Enable/disable autoscaling | | receiver.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled | | receiver.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed | | receiver.autoscaling.pollingInterval | int | `10` | Interval to check each trigger. In seconds. | | receiver.autoscaling.scaleDown | object | `{"stabilizationWindow":180}` | ScaleDown configuration values | | receiver.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | receiver.autoscaling.scaleUp | object | `{"numberOfPods":1,"period":30,"stabilizationWindow":30}` | ScaleUp configuration values | | receiver.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period | | receiver.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied | | receiver.autoscaling.scaleUp.stabilizationWindow | int | `30` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | receiver.autoscaling.triggerCPUValue | int | `75` | CPU value (in percentage), which will cause scaling when reached. The percentage is taken from the resource.limits.cpu value. Limits have to be set up. | | receiver.image.pullPolicy | string | `"Always"` | | | receiver.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-detect-receiver"` | | | receiver.image.tag | string | `"latest-dev"` | | | receiver.initImage.pullPolicy | string | `"Always"` | | | receiver.initImage.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-detect-utilities"` | | | receiver.initImage.tag | string | `"latest-dev"` | | | receiver.replicaCount | int | `1` | | | receiver.resources.limits.cpu | string | `"4000m"` | | | receiver.resources.limits.memory | string | `"8Gi"` | | | receiver.resources.requests.cpu | string | `"1500m"` | | | receiver.resources.requests.memory | string | `"1Gi"` | | | report.autoscaling.enabled | bool | `false` | Enable/disable autoscaling | | report.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled | | report.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed | | report.autoscaling.pollingInterval | int | `10` | Interval to check each trigger. In seconds. | | report.autoscaling.scaleDown | object | `{"stabilizationWindow":180}` | ScaleDown configuration values | | report.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | report.autoscaling.scaleUp | object | `{"numberOfPods":1,"period":30,"stabilizationWindow":30}` | ScaleUp configuration values | | report.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period | | report.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied | | report.autoscaling.scaleUp.stabilizationWindow | int | `30` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | report.autoscaling.triggerCPUValue | int | `75` | CPU value (in percentage), which will cause scaling when reached. The percentage is taken from the resource.limits.cpu value. Limits have to be set. | | report.image.pullPolicy | string | `"Always"` | | | report.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-report"` | | | report.image.tag | string | `"latest-dev"` | | | report.resources.limits.cpu | string | `"8000m"` | | | report.resources.limits.memory | string | `"8Gi"` | | | report.resources.requests.cpu | string | `"2000m"` | | | report.resources.requests.memory | string | `"2Gi"` | | | reportTypes | object | `{}` | Contains key-value pairs where keys are the report type names and values are the report type definitions. | | securityContext.privileged | bool | `false` | | | tcScratch | object | \- | tcScratch values configure generic ephemeral volume options for the Spectra Core `/tc-scratch` directory. | | tcScratch.accessModes | list | `["ReadWriteOnce"]` | Access modes. | | tcScratch.requestStorage | string | `"100Gi"` | Requested storage size for the ephemeral volume. | | tcScratch.storageClassName | string | `nil` | Sets the storage class for the ephemeral volume. If not set, `emptyDir` is used instead of an ephemeral volume. | | tclibs.autoscaling.enabled | bool | `false` | Enable/disable autoscaling | | tclibs.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled | | tclibs.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed | | tclibs.autoscaling.pollingInterval | int | `10` | Interval to check each trigger. In seconds. | | tclibs.autoscaling.scaleDown | object | `{"stabilizationWindow":180}` | ScaleDown configuration values | | tclibs.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | tclibs.autoscaling.scaleUp | object | `{"numberOfPods":1,"period":30,"stabilizationWindow":30}` | ScaleUp configuration values | | tclibs.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period | | tclibs.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied | | tclibs.autoscaling.scaleUp.stabilizationWindow | int | `30` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | tclibs.autoscaling.triggerCPUValue | int | `75` | CPU value (in percentage), which will cause scaling when reached. The percentage is taken from the resource.limits.cpu value. Limits have to be set. | | tclibs.image.pullPolicy | string | `"Always"` | | | tclibs.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-tclibs"` | | | tclibs.image.tag | string | `"latest-dev"` | | | tclibs.replicaCount | int | `1` | | | tclibs.resources.limits.cpu | string | `"2000m"` | | | tclibs.resources.limits.memory | string | `"2Gi"` | | | tclibs.resources.requests.cpu | string | `"1000m"` | | | tclibs.resources.requests.memory | string | `"1Gi"` | | | yaraSync.enabled | bool | `false` | | | yaraSync.image.pullPolicy | string | `"Always"` | | | yaraSync.image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-detect-yara-sync"` | | | yaraSync.image.tag | string | `"latest-dev"` | | | yaraSync.replicaCount | int | `1` | | | yaraSync.resources.limits.cpu | string | `"2000m"` | | | yaraSync.resources.limits.memory | string | `"2Gi"` | | | yaraSync.resources.requests.cpu | string | `"1000m"` | | | yaraSync.resources.requests.memory | string | `"1Gi"` | | | yaraSync.serverPort | int | `8080` | | ## Connector S3 ### Secrets | Secret (fullNameOverride is set) | Secret (deployment with umbrella) | Secret (deployment without umbrella) | Type | Description | | :---- | :---- | :---- | :---- | :---- | | \-secret-\ | \-connector-s3-secret-\ | \-secret-\ | required | Authentication secret used to connect to AWS S3 or any S3-compatible storage system. | ### Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | configuration.dbCleanupPollInterval | int | `7200` | Specifies time in seconds, in which the database cleanup will be run. | | configuration.dbCleanupSampleThresholdInDays | int | `21` | Number of previous days that the data will be preserved. | | configuration.diskHighPercent | int | `0` | disk high percent | | configuration.inputs | list | `[]` | Configuration for S3 File Storage Input. [S3 input](#configmap-configuration-values-for-s3-connector---configuration-for-s3-file-storage-input). | | configuration.maxFileSize | int | `0` | The maximum sample size in bytes that will be transmitted from the connector to the appliance for analysis. Setting it to 0 will disable the option. | | configuration.maxUploadDelayTime | int | `10000` | Delay in milliseconds. In case the Worker cluster is under high load, this parameter is used to delay any new upload to the Worker cluster. The delay parameter will be multiplied by the internal factor determined by the load on the Worker cluster. | | configuration.maxUploadRetries | int | `100` | Number of times the connector will attempt to upload the file to the processing appliance. Upon reaching the number of retries, it will be saved in the error\_files/ destination or be discarded. | | configuration.uploadTimeout | int | `10000` | Period (in milliseconds) between upload attempts of the sample being re-uploaded. | | configuration.uploadTimeoutAlgorithm | string | `"exponential"` | Enum: "exponential" "linear". The algorithm used for managing delays between re-uploading the samples into the processing appliance. In exponential, the delay is defined by multiplying the max upload timeout parameter by 2, until max value of 5 minutes. Linear backoff will always use the Max upload timeout value for the timeout period between re-uploads. | #### Configmap System Info values for Connector S3 | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | configuration.systemInfo | object | \- | Configuration for S3 System Info. [S3 System Info](#configmap-system-info-values-for-connector-s3). | | configuration.systemInfo.centralLogging | bool | `false` | Enable central logging | | configuration.systemInfo.diskHighPercent | int | `0` | Dish high percent | | configuration.systemInfo.fetchChannelSize | int | `40` | Fetch channel size | | configuration.systemInfo.hostUUID | string | `""` | Host UUID | | configuration.systemInfo.maxConnections | int | `10` | Max number of connections | | configuration.systemInfo.maxSlowFetches | int | `12` | Max slow fetches | | configuration.systemInfo.numberOfRetries | int | `300` | Number of retries | | configuration.systemInfo.requestTimeout | int | `43200` | Timeout for requests | | configuration.systemInfo.slowFetchChannelSize | int | `100` | Slow fetch channel size. | | configuration.systemInfo.slowFetchPause | int | `5` | Slow fetch pause | | configuration.systemInfo.type | string | `"tiscale"` | Type | | configuration.systemInfo.verifyCert | bool | `false` | Verify SSL certificate | | configuration.systemInfo.version | string | `"5.6.0"` | Version | | configuration.systemInfo.waitTimeout | int | `1000` | Wait timeout | #### Configmap Configuration values for S3 connector \- Configuration for S3 File Storage Input | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | inputs\[n\] | list | \- | Configmap Configuration for S3 File Storage Input. | | inputs\[n\].awsEnableArn | bool | `false` | Enable/disable the usage of AWS IAM roles to access S3 buckets without sharing secret keys. | | inputs\[n\].awsExternalRoleId | string | `""` | The external ID of the role that will be assumed. This can be any string. | | inputs\[n\].awsRoleArn | string | `""` | The role ARN created using the external role ID and an Amazon ID. In other words, the ARN which allows a Worker to obtain a temporary token, which then allows it to save to S3 buckets without a secret access key. | | inputs\[n\].awsRoleSessionName | string | `"ARNRoleSession"` | Name of the session visible in AWS logs. Can be any string. | | inputs\[n\].bucket | string | `""` | Name of an existing S3 bucket which contains the samples to process. | | inputs\[n\].deleteSourceFile | bool | `false` | Selecting the checkbox will allow the connector to delete source files on S3 storage after they have been processed. Required if 'require\_analyze' or 'post\_actions\_enabled' is true | | inputs\[n\].endpoint | string | `""` | Custom S3 endpoint URL. Leave empty if using standard AWS S3. | | inputs\[n\].folder | string | `""` | The input folder inside the specified bucket which contains the samples to process. All other samples will be ignored. | | inputs\[n\].identifier | string | `""` | Unique name of S3 connection. Must contain only lowercase alphanumeric characters or hyphen (-). Must start and end with an alphanumeric character. Identifier length must be between 3 and 49 characters. | | inputs\[n\].knownBucket | string | `""` | Specify the bucket into which the connector will store files classified as "Malicious". If empty, the input bucket will be used. | | inputs\[n\].knownDestination | string | `"goodware"` | The folder into which the connector will store files classified as 'Goodware'. The folder is contained within the specified bucket field. | | inputs\[n\].maliciousBucket | string | `""` | Specify the bucket into which the connector will store files classified as 'Malicious'. If empty, the input bucket will be used. | | inputs\[n\].maliciousDestination | string | `"malware"` | The folder into which the connector will store files classified as 'Malicious'. The folder is contained within the specified bucket field. | | inputs\[n\].paused | bool | `false` | Temporarily pause the continuous scanning of this Storage Input. This setting must be set to true to enable retro hunting. | | inputs\[n\].postActionsEnabled | bool | `false` | Disable/enable post actions for S3 connectors | | inputs\[n\].priority | int | `5` | A higher Priority makes it more likely that files from this bucket will be processed first. The supported range is from 1 (highest) to 5 (lowest). Values outside of those minimum and maximum values will be replaced by the minimum or maximum, respectively. Multiple buckets may share the same priority. | | inputs\[n\].requireAnalyze | bool | `false` | Disable/enable the requirement for analysis of data processed by connector | | inputs\[n\].serverSideEncryptionCustomerAlgorithm | string | `""` | Customer provided encryption algorithm. | | inputs\[n\].serverSideEncryptionCustomerKey | string | `""` | Customer provided encryption key. | | inputs\[n\].suspiciousBucket | string | `""` | Specify the bucket into which the connector will store files classified as 'Suspicious'. If empty, the input bucket will be used. | | inputs\[n\].suspiciousDestination | string | `"suspicious"` | The folder into which the connector will store files classified as 'Suspicious'. The folder is contained within the specified bucket field. | | inputs\[n\].unknownBucket | string | `""` | Specify the bucket into which the connector will store files classified as "Unknown". If empty, the input bucket will be used. | | inputs\[n\].unknownDestination | string | `"unknown"` | The folder into which the connector will store files classified as 'Unknown'. The folder is contained within the specified bucket field. | | inputs\[n\].verifySslCertificate | bool | `true` | Connect securely to the custom S3 instance. Deselect this to accept untrusted certificates. Applicable only when using a custom S3 endpoint. | | inputs\[n\].zone | string | `"us-east-1"` | AWS S3 region | #### Other Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | boltdb.claimName | string | `nil` | | | enabled | bool | `false` | | | fullNameOverride | string | `""` | overrides connector-s3 chart full name | | image.imagePullPolicy | string | `"Always"` | | | image.repository | string | `"alt-artifactory-prod.rl.lan/appliances-docker-test/detect/images/components/rl-integration-s3"` | | | image.tag | string | `"latest-dev"` | | | imagePullSecrets\[0\] | string | `"rl-registry-key"` | | | nameOverride | string | `""` | overrides connector-s3 chart name | | persistence.accessModes | list | `["ReadWriteOnce"]` | Access mode | | persistence.requestStorage | string | `"10Gi"` | Request Storage | | persistence.storageClassName | string | `"encrypted-gp2"` | Storage class name | | receiver.baseUrl | string | `nil` | | | receiver.service.httpPort | int | `80` | | | tmp | object | \- | tmp values configure generic ephemeral volume options for the connectors `/data/connectors/connector-s3/tmp` directory. | | tmp.accessModes | list | `["ReadWriteOnce"]` | Access modes. | | tmp.requestStorage | string | `"100Gi"` | Requested storage size for the ephemeral volume. | | tmp.storageClassName | string | `nil` | Sets the storage class for the ephemeral volume. If not set, `emptyDir` is used instead of an ephemeral volume. | | worker.releaseName | string | `nil` | Set Spectra Detect Worker release for connector to connect to. It is required if not using the umbrella Helm chart. | ## RabbitMQ | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | affinity | object | `{}` | | | createManagementAdminSecret | bool | `true` | A user management admin secret will be created automatically with given admin username and admin password; otherwise, the secret must already exist. | | createUserSecret | bool | `true` | A user secret will be created automatically with the given username and password; otherwise, the secret must already exist. | | global.umbrella | bool | `false` | | | host | string | `""` | Host for external RabbitMQ. When configured, the Detect RabbitMQ cluster won't be created. | | managementAdminPassword | string | `""` | Management admin password. If left empty, defaults to `password`. | | managementAdminUrl | string | `""` | Management Admin URL. If empty, defaults to "http://:15672". | | managementAdminUsername | string | `""` | Management admin username. If left empty, defaults to `username`. | | password | string | `"guest_11223"` | Password | | persistence.requestStorage | string | `"5Gi"` | | | persistence.storageClassName | string | `nil` | | | port | int | `5672` | RabbitMQ port | | replicas | int | `1` | Number of replicas | | resources.limits.cpu | string | `"2"` | | | resources.limits.memory | string | `"2Gi"` | | | resources.requests.cpu | string | `"1"` | | | resources.requests.memory | string | `"2Gi"` | | | useQuorumQueues | bool | `false` | Setting this to true defines queues as quorum type (recommended for multi-replica/HA setups); otherwise, queues are classic. | | useSecureProtocol | bool | `false` | Setting this to true enables the secure AMQPS protocol for the RabbitMQ connection. | | username | string | `"guest"` | Username | | vhost | string | `""` | Vhost. When empty, the default `rl-detect` vhost is used. | ## PostgreSQL | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | affinity | object | `{}` | | | createUserSecret | bool | `true` | A user secret will be created automatically with the given username and password; otherwise, the secret must already exist. | | database | string | `"tiscale"` | Database name | | global.umbrella | bool | `false` | | | host | string | `""` | Host for external PostgreSQL. When configured, the Detect PostgreSQL cluster won't be created. | | image.repository | string | `"ghcr.io/cloudnative-pg/postgresql"` | | | image.tag | string | `"17.6"` | | | password | string | `"tiscale_11223"` | Password | | persistence.accessModes\[0\] | string | `"ReadWriteOnce"` | | | persistence.requestStorage | string | `"5Gi"` | | | persistence.storageClassName | string | `nil` | | | port | int | `5432` | PostgreSQL port. | | replicas | int | `1` | Number of replicas. | | resources.limits.cpu | string | `nil` | | | resources.limits.memory | string | `nil` | | | resources.requests.cpu | string | `"500m"` | | | resources.requests.memory | string | `"1Gi"` | | | username | string | `"tiscale"` | Username. Required if `host` is not set, because the Detect PostgreSQL cluster will be created and this user will be set as the database owner. | --- ## SpectraDetect AWS EKS Microservices Deployment — Helm, KEDA, and RabbitMQ ## Introduction **Warning: **PREVIEW**** This deployment and its interfaces are under active development and subject to change. Compatibility is not guaranteed across minor updates during the beta period. Scope: **Non‑production use** This document describes how the Spectra Detect platform is deployed and operated on Kubernetes, providing a high-volume, high-speed file processing and analysis that seamlessly integrates into existing infrastructure and effectively scales with business needs. The platform is packaged as container images and deployment is managed with Helm charts, which package Kubernetes manifests and configuration into versioned releases for consistent installs, upgrades, and rollbacks. Configuration is externalized via [ConfigMaps](./config-reference.md) and [Secrets](./config-reference.md) so behavior can be adjusted without rebuilding images, and sensitive data is stored separately with controlled access. Horizontal Pod Autoscaling may adjust replica counts based on metrics such as CPU utilization or queue size. ## Requirements - EKS version 1.34, Amazon Linux 2023 with cgroupsv2 (tested) - `Persistent Volume` provisioner supporting `ReadWriteMany` (e.g. Amazon EFS CSI). - Helm 3 or above ## Operators and Tools ### Keda (autoscaling - optional) For Spectra Detect to autoscale Workers, [Keda](https://keda.sh/) needs to be installed on the cluster. Keda can be deployed following the official [Deploying Keda documentation](https://keda.sh/docs/2.17/deploy/). It is not required to have Keda installed to run Spectra Detect on K8s, but it is required to utilize Worker autoscaling features. ### Prometheus Operator All pods have metrics exposed in Prometheus format. Prometheus can be used by setting: ```yaml # Can be enabled with the following worker: monitoring: # -- Enable/disable monitoring with Prometheus enabled: true # -- Use actual release name prometheusReleaseName: "${PROMETHEUS RELEASE NAME}" ``` ### RabbitMQ Broker Spectra Detect Helm charts support using external RabbitMQ Brokers (like AmazonMQ), as well as deploying and using RabbitMQ cluster resources as part of a Detect deployment installed in the same namespace. Choose which option to use based on the business requirements. - **External RabbitMQ Broker** (deployed and managed outside of Spectra Detect Helmcharts) External/existing RabbitMQ Broker needs to be set up as per the broker installation guides. As an example, please check the [Amazon MQ instructions](https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/getting-started-rabbitmq.html). - **RabbitMQ Operator** (deployed and managed by Spectra Detect Helm charts) Cloud native brokers can be deployed and managed by Spectra Detect Helm charts. RabbitMQ Operator needs to be installed in the K8s cluster. ```bash kubectl apply --wait -f \ https://github.com/rabbitmq/cluster-operator/releases/download/v2.6.0/cluster-operator.yml ``` | Secret | Type | Description | |----------------------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `-rabbitmq-secret` | required | Basic authentication secret which contains the RabbitMQ username and password. Secret is either created manually (rabbitmq chart) or already exists. | | `-rabbitmq-secret-admin` | optional | Basic authentication secret which contains the RabbitMQ Admin username and password. Secret is either created manually (rabbitmq chart) or already exists. If missing, credentials from `-rabbitmq-secret`. | ### PostgreSQL Server Spectra Detect Helm charts support using external PostgreSQL Clusters (like Amazon RDS), as well as deploying and using PostgreSQL cluster resources as part of a Detect deployment. - **External PostgreSQL Server** (deployed and managed outside of Spectra Detect Helm charts) External/existing PostgreSQL server needs to be set up as per the PostgreSQL server guide. As an example, please check the Amazon RDS instructions - **CloudNativePG Operator** (deployed and managed by Spectra Detect Helm charts) CloudNativePG Operator needs to be installed in the K8s cluster. ```bash # PostgreSQL Operator - CloudNativePG (CNPG) kubectl apply --wait -f \ https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.21/releases/cnpg-1.21.1.yaml ``` | Secret (deployment with Detect chart) | Type | Description | |---------------------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------| | `-postgres-secret` | required | Basic authentication secret which contains the database username and password. Secret is either created manually (postgres chart) or already exists. | ## Storage ### Persistent volume The `/scratch` folder is implemented as a persistent volume. Multiple services have access to the volume: - Cleanup - Preprocessor - Processor - Postprocessor - Receiver - Rabbit (if deployed) - Postgres (if deployed) - Connector S3 (if deployed) Since multiple services (pods) are accessing the volume, the access mode of that volume has to be `ReadWriteMany`. In AWS, it is recommended to use EFS storage since it supports the requested access for pods. #### Amazon EFS Remote Storage If you are running Kubernetes on Amazon EKS, you can use Amazon EFS storage for the shared storage. You will need to: 1. Install Amazon [EFS CSI Driver](https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html) on the cluster to use EFS 2. Create EFS file system via Amazon EFS console or [command line](https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/docs/efs-create-filesystem.md) 3. Set Throughput mode to Elastic or Provisioned for higher throughput levels 4. Add [mount targets](https://docs.aws.amazon.com/efs/latest/ug/accessing-fs.html) for the node's subnets 5. Create a [storage class](https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/examples/kubernetes/dynamic_provisioning/README.md#dynamic-provisioning) for Amazon EFS ### Ephemeral volume The `/tc-scratch` folder is used as an ephemeral volume for temporary processing files. - Processor The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). The storage class name is empty by default (value `tcScratch.storageClassName`). If not overridden, `emptyDir` will be used for storage. ## Getting Started ReversingLabs Spectra Detect Helm charts and container images are available at **registry.reversinglabs.com**. In order to connect to the registry, you need to use the ReversingLabs Spectra Intelligence account. ```bash helm registry login registry.reversinglabs.com -u "${RL_SPECTRA_INTELLIGENCE_USERNAME}" ``` If you want to see which versions of the charts are available in the registry, you can use a tool like [Skopeo](https://github.com/containers/skopeo) to log in to the registry and list the versions: ```bash skopeo login registry.reversinglabs.com -u "${RL_SPECTRA_INTELLIGENCE_USERNAME}" skopeo list-tags docker://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform eg. { "Repository": "registry.reversinglabs.com/detect/charts/detect-suite/detect-platform", "Tags": [ "5.7.0-0.beta.3" ] } ``` ### List of Detect Images | Image | Mandatory/Optional | Scaling * (see [Appendix](#appendix)) | |---------------------------|--------------------|---------------------------------------| | `rl-detect-auth` | Optional | N/A | | `rl-detect-receiver` | Mandatory | CPU | | `rl-detect-preprocessor` | Optional | QUEUE | | `rl-processor` | Mandatory | QUEUE | | `rl-tclibs` | Mandatory | CPU | | `rl-detect-postprocessor` | Optional | QUEUE | | `rl-integration-s3` | Optional | N/A | | `rl-report` | Optional | CPU | | `rl-cloud-cache` | Optional | N/A | | `rl-detect-utilities` | Mandatory | N/A | ### Deploying Detect using Helm Charts 1. Create a namespace or use an existing one ```bash kubectl create namespace detect # Namespace name is arbitrary ``` 2. Set up the Registry Secret to allow Kubernetes to pull container images. The Kubernetes secret `rl-registry-key` containing the user's Spectra Intelligence credentials needs to be created in the namespace where Detect will be installed. 2.1. The secret can either be created via Detect Helm chart - `registry.createRegistrySecret`: needs to be set to true (default) - `registry.authSecretName`: value needs to be the Spectra Intelligence account username. - `registry.authSecretPassword`: value needs to be the Spectra Intelligence account password. 2.2. Can be managed outside the Helm release - `registry.createRegistrySecret`: value should be set to false. You can create the secret manually by using the following command ```bash kubectl apply -n "detect" -f - <<EOF apiVersion: v1 kind: Secret metadata: name: "rl-registry-key" type: kubernetes.io/dockerconfigjson data: .dockerconfigjson: $(echo -n '{"auths": {"registry.reversinglabs.com": {"auth": "'$(echo -n "${SPECTRA_INTELLIGENCE_USERNAME}:${SPECTRA_INTELLIGENCE_PASSWORD}" | base64)'"}}}' | base64 | tr -d '\n') EOF ``` 3. Install a Spectra Detect Chart Choose any deploy name and run the following command. For more details regarding values, please reference the [Appendix](#appendix). ```bash helm install "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \ --version "${DETECT_HELM_CHART_VERSION}" --namespace "${NAMESPACE}" -f values.yaml ``` 4. Configure Ingress Controller In order to be able to access Spectra Detect endpoints from outside the cluster, and in order for Worker pod to be able to connect to the Spectra Detect Manager, an Ingress Controller (like AWS ALB or Nginx Controller) must be configured on the K8s cluster. Follow the official installation guides for the controllers: - [AWS Load Balancer Controller](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.13/deploy/installation/) - [Nginx Controller](https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/) This example shows how to configure Ingress using [AWS ALB Controller](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/) and how to use [External DNS](https://kubernetes-sigs.github.io/external-dns/v0.15.0/) to automatically create DNS records in AWS Route53. > Ingress values configuration example using AWS ALB Controller ```yaml worker: ingress: annotations: alb.ingress.kubernetes.io/backend-protocol: HTTP alb.ingress.kubernetes.io/certificate-arn: <> alb.ingress.kubernetes.io/group.name: detect alb.ingress.kubernetes.io/healthcheck-interval-seconds: "15" alb.ingress.kubernetes.io/healthcheck-path: /readyz alb.ingress.kubernetes.io/healthcheck-port: "80" alb.ingress.kubernetes.io/healthcheck-protocol: HTTP alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "5" alb.ingress.kubernetes.io/healthy-threshold-count: "2" alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=600 alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true" alb.ingress.kubernetes.io/scheme: internal alb.ingress.kubernetes.io/security-groups: <> alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10 alb.ingress.kubernetes.io/success-codes: 200,301,302,404 alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=1200 alb.ingress.kubernetes.io/target-type: ip alb.ingress.kubernetes.io/unhealthy-threshold-count: "2" external-dns.alpha.kubernetes.io/hostname: detect-platform.example.com external-dns.alpha.kubernetes.io/ingress-hostname-source: annotation-only className: alb enabled: true host: detect-platform.example.com ``` ### Updating Detect using Helm Charts Before running the upgrade, you can use the helm `diff upgrade` command to see the changes that will occur in the Kubernetes manifest files. [Helm Diff Plugin](https://github.com/helmfile/helmfile) must be installed to utilize the `diff` feature. You can install Helm Diff using the following command: ```bash # Install plugin helm plugin install https://github.com/databus23/helm-diff # Run diff command helm diff upgrade "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \ --version "${DETECT_CHART_VERSION}" \ --namespace "${NAMESPACE}" \ -f values.yaml # Check the Helm chart readme beforehand if you want helm show readme oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform --version "${DETECT_CHART_VERSION}" # Run upgrade helm upgrade "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \ --version "${DETECT_CHART_VERSION}" \ --namespace "${NAMESPACE}" \ -f values.yaml ``` ### Uninstalling Detect ```bash helm uninstall detect -n "${NAMESPACE}" ``` ## Authentication Authentication is achieved by leveraging [Authentication based on SubRequest Result](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-subrequest-authentication/). This is natively supported by the Nginx Ingress Controller. In case a different ingress controller is used (e.g. ALB on AWS), additional Nginx Reverse Proxy is deployed in order to support the authentication mechanism. Authentication can be configured in the following ways: 1. Using external authentication service by specifying the `externalAuhtUrl` in the configuration If an external authentication service is enabled, all header values from the incoming request will be forwarded to the external authentication service. The external authentication service needs to return following responses in order to support this authentication mechanism: - `HTTP 200`: authentication successful (ingress will forward the traffic to the backend service) - `HTTP 401` or `HTTP 403`: authentication failed 2. Using a simple authentication service deployed in the cluster Authentication service supports a simple Token check based on API path. The token needs to be included in the "Authorization" header with the "Token" prefix/scheme. ```bash curl -H "Authorization: Token " ``` The tokens are configured as secrets with the following behavior: | Secret | Type | Description | Used in deployments (Pods) | |:----------------------------------------------|:---------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------| | `-secret-worker-api-token` | Optional | Token secret containing a token that is used to protect all endpoints with /api/ prefix, e.g. file upload. | Auth | | `-secret-worker-api-task-token` | Optional | Token secret containing a token that is used to protect `/api/tiscale/v1/task` endpoints. If left empty, the mentioned API is protected by `-secret-worker-api-token`. | Auth | The authentication service won’t be deployed in the cluster if `externalAuhtUrl` is defined: ```yaml # Example for enabling authentication worker: configuration: authentication: enabled: true externalAuthUrl: "" ``` ## Appendix ### Set Report Types - Report types can be added with the `--set-file` option by providing the name of the report type (added as a new key to `reportTypes`) and path to the file in which the filter is defined in JSON format - Report type name must match the one defined in the given file - Report types can be deleted by defining the name of the `report.type` and setting it to the value "" instead of adding the filepath - **Limitation**: max size of the report type file is 3MiB ### Set Report Types Example ```bash # Example of adding the 2 report types helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.reportTypes.some_report_type=/some_report-type.json --set-file worker.reportTypes.extendedNoTags=/extendedNoTags.json # Example of adding the new report type and removing the existing report type helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.reportTypes.some_report_type=/some_report-type.json \ --set-file worker.reportTypes.extendedNoTags="" \ ############### Example of the report type file content ############### { "name": "exampleReportType", "fields": { "info" : { "statistics" : true, "unpacking" : true, "package" : true, "file" : true, "identification" : true }, "story": true, "tags" : true, "indicators" : true, "classification" : true, "relationships" : true, "metadata" : true } } ``` ### Use Report Type Example Uploading a report type using the method above only makes the report type available to the system. To actually use the custom report type, you must configure it on the appropriate egress integration. For example, to use a custom report type with S3 storage: ```yaml # Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. worker: configuration: reportS3: reportType: "custom_report_type" ``` ### Set YARA Rules - YARA rules can be added with the `--set-file` option by providing the name of the rule file (added as a new key to `yaraRules`) and path to the file in which the rule is defined - The rule file name must follow camelCase format - YARA rules can be deleted by defining the rule file name and setting it to the value "" instead of adding the filepath - **Limitation**: max size of YARA ruleset file is 45MiB ### Set YARA Rules Example ```bash # Example of adding yara rules helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-platform -f custom_values.yaml \ --set-file worker.yaraRules.rule1=/someYaraRule.yara \ --set-file worker.yaraRules.rule2=/other_yara_rule.yara # Example of adding the new yara rule and removing the existing yara rule helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.yaraRules.rule1=/someYaraRule.yara \ --set-file worker.yaraRules.rule2="" ############### Example of the yara rule file content ############### rule ExampleRule : tc_detection malicious { meta: tc_detection_type = "Adware" tc_detection_name = "EXAMPLEYARA" tc_detection_factor = 5 strings: $1 = "example" $2 = "eeeeeee" condition: $1 or $2 } ``` ### Set Advanced Filter - Advanced filters can be added with the `--set-file` option by providing the name of the filter (added as a new key to `advancedFilters`) and path to the file in which the filter is defined in YAML format. Filter name must match the one defined in the given file - Advanced filters can be deleted by defining the name of the filter and setting it to the value "" instead of adding the filepath ```bash # Example of adding Advance Filter helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.advancedFilters.filter1=/some_filter.yaml \ --set-file worker.advancedFilters.filter2=/other_filter.yaml # Example of adding Advance Filter helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.advancedFilters.filter1=/some_filter.yaml \ --set-file worker.advancedFilters.filter2="" ``` Example of the filter file content: ```yaml name: some_filter description: Custom filter for Spectra Analyze integration scope: container type: filter_in condition: and: - range: info.file.size: gt: 50 lt: 20000 - one_of: classification.classification: - 3 - 2 ``` ### [Configuration Reference](config-reference.md) **Info: View the [Configuration Reference](config-reference.md) for more information about the configuration options.** ## Scaling Scaling of the services is done in one of the following ways: - Scaling based on CPU usage - Scaling based on the number of messages waiting in the queue ### Scaling based on CPU usage Scaling based on CPU usage is implemented in the following way: - Users can provide `triggerCPUValue` which represents the percentage of the given CPU resources. Service will be scaled up when this threshold is reached and scaled down when CPU usage drops below the threshold. - CPU resources are defined with `resources.limits.cpu` value, which represents the maximum CPU value that can be given to the pod - **Default values**: - scaling disabled by default - when CPU usage reaches 75%, the scaling is triggered - scaling delay is 30 seconds (scaling is triggered when scaling condition is met for 30 seconds) - one pod in every 30 seconds is created - number of maximum replicas is 8 - every 10 seconds the average CPU status is checked - **Services with CPU scaling**: - receiver - report - tclibs ### Scaling based on the number of messages waiting in the queue Scaling based on the number of messages waiting in the queue is implemented in the following way: - Scaling disabled by default - User can provide `targetInputQueueSize` which represents the number of messages in the queue for which the scaling will start - Unacknowledged messages are excluded from this calculation - Each scaled service has a different queue that is observed and scaled on - **Default values**: - when 10 or more messages are waiting in the queue for a longer period of time the scaling will be triggered - scaling delay is 15 seconds (scaling is triggered when scaling condition is met for 15 seconds) - one pod in every 30 seconds is created - number of maximum replicas is 8 - every 10 seconds status of the queue is checked - **Services with Queue based scaling**: - processor: number of messages in the `tiscale.hagent_input` is used for scaling - processor-retry: number of messages in the `tiscale.hagent_retry` is used for scaling - postprocessor: number of messages in the `tiscale.hagent_result` is used for scaling - preprocessor: number of messages in the `tiscale.preprocessing` is used for scaling - preprocessor-unpacker: number of messages in the `tiscale.preprocessing_unpacker` is used for scaling ### Scaling configuration | Value | Description | |:-----------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------| | `enabled` | enables/disables auto-scaling | | `maxReplicas` | maximum number of replicas that can be deployed when scaling in enabled | | `minReplicas` | minimum number of replicas that need to be deployed | | `pollingInterval` | interval to check each trigger (in seconds) | | `scaleUp` | configuration values for scaling up | | `stabilizationWindow` | number of continuous seconds in which the scaling condition is met (when reached, scale up is started) | | `numberOfPods` | number of pods that can be scaled in the defined period | | `period` | interval in which the numberOfPods value is applied | | `scaleDown` | configuration values for scaling down | | `stabilizationWindow` | number of continuous seconds in which the scaling condition is not met (when reached, scale down is started) | | **CPU scaling** | | | `triggerCPUValue` | CPU value (in percentage), which will cause scaling when reached; percentage is taken from the resource.limits.cpu value; limits have to be set up | | **Queue scaling** | | | `targetInputQueueSize` | Number of waiting messages to trigger scaling on; unacknowledged messages are excluded | --- ## SpectraDetect Central SDM — Federated Manager Monitoring and YARA Policy Sync The Central Spectra Detect Manager (CSDM) is a management overlay that operates on top of deployed SDM instances. It provides a high-level, aggregated operational dashboard across all deployed SDM environments and ensures consistent security policies are maintained across the entire federated SDM deployment topology. ## Overview The current Spectra Detect Manager (SDM) implementation supports both Open Virtual Appliance (OVA) and monolith microservice deployment models, but cannot manage both simultaneously within a single SDM instance. This limitation arises because the configuration switches that govern the instance's deployment behavior are implemented as global parameters. The CSDM addresses this limitation by providing: - **Centralized monitoring**: Aggregated operational dashboard across all deployed SDM environments - **Policy synchronization**: Consistent YARA rule distribution across the entire federated SDM deployment topology - **Multi-deployment support**: Ability to manage both OVA and Kubernetes deployments from a single interface, supporting migration from legacy deployment models ### What Central SDM does not do The CSDM operates as a monitoring and YARA rule management plane. It does not replace the Spectra Detect Manager for day-to-day configuration tasks. The following operations remain on each local SDM: - Connector configuration (ingress and egress) - User management and access control - Quota settings - Classification change notifications - Central Configuration settings Local SDMs continue to serve as the only integration point for Worker and Spectra Analyze appliances. The CSDM is not a direct integration endpoint. ## Requirements - All local SDMs registered with the Central SDM must run version 5.7.2 or later - OpenID Connect (OIDC) authentication is strongly recommended for Single Sign-On (SSO) functionality ## Architecture The CSDM uses a centralized approach where a single SDM instance manages all deployments, providing a unified view over the entire system while respecting the specifics of different deployment types. The CSDM transparently aggregates groups from different environments. ### Design approach The CSDM architecture is based on an API-driven model where the CSDM acts as a lightweight orchestrator rather than a complete data repository. It relies on APIs to communicate with local SDMs, sending requests to relevant instances when information is needed or changes must be made. Each local SDM remains the true source of its own data. All communication from the CSDM to local SDMs is initiated by the CSDM. This design simplifies network configuration and prevents the need for complex, direct connections between components. ### Local Spectra Detect Manager registration Local Spectra Detect Manager (SDM) instances are registered with the Central SDM through configuration. This registration specifies the necessary parameters for the SDM's integration into the broader architecture. ### Data ingestion and M2M communication The Central SDM collects operational data and telemetry from the registered local SDMs using the standard SDM API. Authentication for this Machine-to-Machine (M2M) communication is secured through a token-based mechanism. The CSDM operates solely as a centralized monitoring and management plane only. ### Communication patterns #### API requests The CSDM sends API requests to local SDMs when configuration changes or information retrieval is needed. For Kubernetes deployments, a Manager component serves as a proxy between the CSDM and individual appliances. The appliance ID is included as part of the request, and the Manager forwards the request to the specific hub or worker. #### Appliance events Events from appliances are collected by local SDMs and stored in persistent storage. The CSDM pulls events from local SDMs on a regular polling schedule, reversing the typical communication direction for event handling. ### YARA rule synchronization #### Centralized rule authority The CSDM maintains the master copy and serves as the single source of truth for all YARA rules. The CSDM is responsible for aggregating and resolving rule conflicts from all connected Detect and Analyze appliances. #### Distribution mechanism The CSDM pushes YARA rules to local SDMs. For Kubernetes deployments, local SDMs serve as repositories from which workers pull rules. This maintains the standard worker behavior while ensuring centralized rule management. #### Upgrade path and initial data seeding An existing cluster or set of clusters can be switched to the CSDM. During this transition, the existing single SDM is temporarily designated as the initial source, and its active YARA rules are imported into the CSDM repository. After import, all local SDMs are required to use the YARA rules managed by the CSDM. #### Request forwarding Spectra Analyze appliances continue to direct requests to their respective local SDMs. However, the local SDMs are configured to forward all YARA-related requests from both Spectra Analyze appliances and Workers to the CSDM for centralized processing. Sample-specific operations like egress configuration and retro hunting remain local. Worker synchronization is also forwarded to CSDM. Synchronization status is resolved by local SDM by fetching the rules versions from CSDM and comparing them with Workers' statuses. ## Configuration ### Central SDM The Central SDM (CSDM) pulls operational telemetry from local SDM instances. Local SDM discovery information is configured in the `multiRegion` section of the CSDM's ConfigMap. Local SDMs are enumerated here, and the overall multi-region polling feature is activated by setting the `enabled` flag to true. ```yaml multiRegion: enabled: true initialYaraSyncPullId: vm-1 sdmList: - id: vm-1 name: "OVA First" url: https://10.200.1.1 type: "ova" - id: vm-2 name: "OVA Second" url: https://10.200.2.2/ type: "ova" - id: k8s-2 name: "AWS Kubernetes Third" url: "https://third.detect-dev.reversinglabs.com/" type: "kubernetes" - id: k8s-1 name: "AWS Kubernetes Fourth" url: "https://fourth.detect-dev.reversinglabs.com/" type: "kubernetes" ``` #### Configuration parameters - **enabled**: Activates or deactivates Central SDM multi-region polling - **initialYaraSyncPullId**: Specifies which local SDM environment serves as the initial source of YARA rules. This must match one of the `id` values in the `sdmList` - **sdmList**: Defines all local SDMs that the Central SDM monitors - **id**: Internal identifier for the SDM (used by APIs and UI routing) - **name**: Display name shown in the Central SDM UI - **url**: Base URL of the local SDM - **type**: Deployment type (`ova` or `kubernetes`) #### Initial YARA rule synchronization When configuring Central SDM, exactly one environment is designated as the initial YARA rule source via the `initialYaraSyncPullId` parameter. This environment's Spectra Analyze appliances should contain the YARA rulesets that will be distributed to all other environments. All other Spectra Analyze appliances must start without existing rulesets. They will receive their initial rulesets through synchronization from the designated source environment. After the initial synchronization is complete, the Central SDM maintains a consistent ruleset across all environments. ### Local SDM The local Spectra Detect Manager (SDM) generally operates independently of the CSDM. The exception to this independence is the YARA rule synchronization process. The local SDM assumes a simple consumer role, exclusively fetching the master YARA rule set from the central CSDM. This behavior is configured exclusively on the CSDM side via an API operation. Refer to the Central SDM API documentation for endpoint details. This design decision ensures compatibility with deployment environments, such as OVA-based clusters, where configuration by a ConfigMap is not available. Local SDMs are registered at the Central SDM. No separate configuration on each local SDM is required beyond standard YARA synchronization with Spectra Analyze and workers. ## Using Central SDM ### Authentication Authentication for the Central SDM is configured to be consistent with the approach used for the local Spectra Detect Manager (SDM) instances. It is strongly recommended that OpenID Connect (OIDC) be deployed as the primary authentication mechanism. Using OIDC enables Single Sign-On (SSO), allowing users to seamlessly transition between the CSDM interface and local SDM management consoles without requiring re-authentication. For machine-to-machine communication, local SDMs are authenticated with the CSDM using token-based authentication. Workers are authenticated with their local SDM instances as needed. ### Landing page The UI of the CSDM is similar to the regular SDM UI, but with reduced functionality. Most of it is located on the single, initial screen. On the left side, it shows a list of the configured SDMs together with their type icon (VM/Kubernetes), name, and status. ![image](./images/CSDM.png) SDM statuses are: - **Unknown**: Before status is known - **Online**: Everything is operating normally - **Degraded**: Some hubs or workers are down - **Error**: Some component reported an error - **Offline**: Secondary unavailable - **Maintenance**: Maintenance or restart in progress - **Unauthorized**: Invalid primary authentication configuration ### Appliances tab The Appliances tab on the SDM-CM interface largely mirrors the core status presentation found in a standalone local SDM. However, it incorporates CSDM specific controls which act as a centralized gateway to manage individual instances. Added controls: - **Manage button**: Launches the UI of the specific local SDM in a new browser tab. This is an authenticated launch (preferably via SSO) for direct configuration and troubleshooting. ### YARA synchronization The overview screen provides a granular status display, detailing the current sync state for each appliance. Possible synchronization states include: - **InSync**: The local SDM has successfully fetched and applied the latest rule set - **OutOfSync**: The local SDM is using an older version of the rules - **Error**: The last attempted synchronization failed - **Unavailable**: The CSDM cannot establish communication with the local SDM endpoint - **PendingNew**: A new rule set is available on the CSDM-CM, awaiting local SDM retrieval - **Disabled**: Synchronization for this local SDM has been administratively deactivated - **NoRules**: The local SDM has not yet received any rules from the CSDM ## Upgrading to Central SDM Upgrading from a single existing local SDM to Central SDM is the supported migration path. The process involves designating an existing SDM as the initial YARA rule source and importing its active rules into the Central SDM. ### Migration considerations - All SDMs in a Central SDM deployment share the same YARA rules - The upgrade requires a maintenance window during which YARA synchronization is paused and rules are migrated - Environments with multiple pre-existing SDMs that have independent rulesets require manual alignment before migration ### Migration procedure 1. Prepare the existing SDM that will serve as the initial rules source: - Place the SDM into maintenance mode - Stop YARA synchronization between Spectra Analyze and the SDM - Export active YARA rules from this SDM 2. Import rules into the Central SDM 3. Configure the local SDM to consume rules from the Central SDM 4. Re-enable synchronization (Central SDM to local SDM to workers) 5. Add additional SDMs to the Central SDM configuration. These will automatically adopt the centralized ruleset ## Limitations and considerations ### Functional scope The Central SDM UI does not include the following features, which remain available only on each local Spectra Detect Manager instance: - Central Configuration page - Connector settings (ingress and egress configuration) - Classification change notifications - Local user and access management ### Operational considerations - The Central SDM does not push configuration changes to local SDMs. It serves as a monitoring and YARA rule authority only - If you need to decommission the Central SDM or remove an instance from it, coordinate this change with support to avoid interruptions in rule management - All connected SDMs must share a single, unified YARA ruleset managed by the Central SDM --- ## SpectraDetect Deployment — OVA, AMI, Kubernetes EKS, and Multi-Region Spectra Detect appliances are available as OVA, AMI, or container images. Use your platform’s standard deployment process, and refer to the guides below for detailed instructions. Additionally, you can deploy Spectra Detect using Kubernetes. Kubernetes is an open source platform installed on Linux servers according to the manufacturer's instructions, and supports several different Linux distributions, including current versions of Ubuntu 24.04 LTS and Debian 12. ## Deployment Guides --- ## SpectraDetect Multi-Region Deployment — High Availability and Data Residency A multi-region deployment distributes Spectra Detect across two or more geographic locations to achieve high availability, disaster recovery, and data residency compliance. Files are analyzed in the region where they are submitted, and a centralized SDM cluster provides unified management across all regions. ## Architecture ![Spectra Detect multi-region deployment](../images/spectra-detect-multi-region.png) A single load balancer sits in front of both regions and receives all incoming traffic from file sources (email gateways, web proxies, endpoint agents, file storage, and cloud buckets). It routes each request to the active region based on health, geography, or routing policy. Within each region, an SDM node and a Hub node run as an active/standby pair. Region A runs the active SDM and the active Hub; Region B runs the standby SDM and standby Hub, kept in sync via state replication. Workers run as a cluster in each region and process files submitted by the local Hub. A worker in Region B handles overflow or failover from Region A when needed. Analysis results flow from the Worker clusters through the Hubs and out to the configured output destinations — SOAR, SIEM, EDR, sandbox, and threat intelligence platforms — regardless of which region performed the analysis. ## When to use multi-region deployment Use a multi-region deployment when you need: - **Data residency** — files must be analyzed in a specific jurisdiction and must not leave that region. - **Low latency** — file sources in different geographies connect to the nearest Hub rather than routing across continents. - **High availability** — a regional outage does not stop file analysis in other regions. - **Disaster recovery** — the SDM failover ensures management continuity even if one data center is lost. ## Components | Component | Role in multi-region | |---|---| | SDM load balancer | Routes SDM API/UI traffic to the active SDM node; fails over to standby across regions | | SDM cluster (active + standby) | Central control plane; state-synced between nodes | | Hub load balancer | Distributes file submission traffic to regional Hub clusters by geography or policy | | Regional Hub cluster (active + standby) | Ingests files and distributes them to Workers in the same region | | Regional Worker cluster | Performs file analysis locally; scales independently per region | ## Prerequisites Before deploying across multiple regions: - An SDM instance must be reachable from all Hub clusters (consider network peering or a transit gateway between regions). - Each regional Hub and Worker cluster must be deployed and registered with SDM before enabling cross-region routing. - TLS certificates must cover all regional endpoints, or a wildcard/SAN certificate must be issued for the load balancer and regional hostnames. **Load balancer requirements by platform:** | Platform | SDM load balancer | Hub load balancer | |---|---|---| | On-premises data center | Customer-provided hardware or software load balancer (for example, F5 BIG-IP, HAProxy, or NGINX) with health-check and failover support | Customer-provided load balancer with geographic or policy-based routing between data center sites | | AWS | AWS Global Accelerator or Application Load Balancer (ALB) | AWS Global Accelerator with endpoint groups per region, or Route 53 latency-based or geolocation routing | | Microsoft Azure | Azure Front Door or Azure Load Balancer with Traffic Manager | Azure Front Door with origin groups per region, or Azure Traffic Manager with geographic routing | | Google Cloud | Google Cloud Load Balancing (global external HTTP(S) load balancer) | Google Cloud Load Balancing with backend services per region, or Cloud DNS with geolocation routing policies | ## Deployment steps ### 1. Deploy the SDM cluster Deploy SDM in your primary region following the appropriate [deployment guide](./index.md) for your platform (OVA, AMI, or Kubernetes). Configure the standby SDM node in a second region and enable state synchronization between them. Place a load balancer (or DNS failover) in front of both SDM nodes so that API and UI clients use a single hostname regardless of which node is active. ### 2. Deploy regional Hub and Worker clusters For each region: 1. Deploy a Hub cluster (active + standby) following the appropriate [deployment guide](./index.md) for your platform. 2. Deploy a Worker pool in the same region and register it with the local Hub. 3. Register the Hub cluster with SDM using the SDM management API or UI. Repeat for each region. Each region should have its own Hub cluster and Worker pool that operate independently. ### 3. Configure the Hub load balancer Configure your global load balancer to route incoming file submission traffic to the correct regional Hub cluster. Common routing strategies: - **Geographic routing** — route requests to the nearest region based on the client's location. - **Latency-based routing** — route to the region with the lowest measured latency from the client. - **Failover routing** — designate a primary region and fail over to a secondary if the primary becomes unhealthy. ### 4. Verify cross-region operation After deployment: 1. Submit a file from each region and confirm it is analyzed by Workers in the correct regional cluster. 2. Check the SDM dashboard and confirm all regional Hubs and Workers appear as connected and healthy. 3. Test SDM failover by stopping the active SDM node and confirming the standby takes over without interrupting Hub connectivity. 4. Confirm YARA rules deployed through SDM propagate to Workers in all regions. ## Considerations **YARA rule synchronization** — SDM pushes YARA rule updates to all registered Workers across all regions. Ensure network connectivity between SDM and every regional Worker cluster. **License** — each Worker node consumes a license seat regardless of region. Contact your ReversingLabs account team to confirm your license covers the total Worker count across all regions. **Analysis results** — by default, each regional Hub stores results locally. Configure S3 output or webhook forwarding if you need results aggregated centrally. **SDM availability** — if the SDM cluster is unreachable, Hubs and Workers continue processing files using their last-known configuration. YARA rule updates and license checks are paused until SDM connectivity is restored. --- ## Spectra Detect File Analysis — Worker API, Reports, and Spectra Core Results Spectra Detect Worker analyzes files submitted via the [Worker API](../API/UsageAPI.md) and produces a detailed analysis report for every file using the built-in Spectra Core static analysis engine. Analysis reports can be retrieved in several ways, depending on the Worker configuration. It is also possible to control the contents of the report to an extent. ## Retrieving Analysis Reports There are two ways to get the file analysis report(s): 1. The [Get information about a processing task](../API/UsageAPI.md#get-information-about-a-processing-task-taskinfo) endpoint. Sending a GET request to the endpoint with the task ID returns the analysis report in the response. 2. Saved on one of the configured integrations. - **S3** - for hosted deployments, this is the only supported integration - Microsoft services (Azure Storage, SharePoint, OneDrive) - file shares (NFS, SMB) - Splunk - callback server ## Adding Custom Data to the Report Users can also save any custom data in the analysis report by submitting it in the file upload request. The `custom_data` field accepts any user-defined data as a JSON-encoded payload. This data is included in all file analysis reports (Worker API, Callback, AWS S3, Azure Data Lake and Splunk, if enabled) in its original, unchanged form. The `custom_data` field will not be returned in the [Get information about a processing task](../API/UsageAPI.md#get-information-about-a-processing-task-taskinfo) endpoint response if the file has not been processed yet. Users should avoid using `request_id` as a key in their `custom_data`, as that value is used internally by the appliance. **Example - Submitting a file with the custom_data parameter to store user-defined information in the report** ```bash curl https://tiscale-worker-01/api/tiscale/v1/upload -H 'Authorization: Token 94a269285acbcc4b37a0ad335d221fab804a1d26' -F file=@Classification.pdf -F 'custom_data={"file_source":{"uploader":"malware_analyst", "origin":"example"}}' ``` ## Customizing Analysis Reports There are several different ways of customizing an analysis report: 1. through report configuration 2. through report **types** 3. through report **views** These methods are not mutually exclusive and are applied in the order above (configuration first, then report type, then report view). For example, to even be present for later filtering/transforming, **strings** found in a file must be included in the report. **Report types** are results of *filtering* the **full** report. In other words, fields can be included or excluded as required. On the other hand, **report views** are results of *transforming* parts of the report, such as field names or the structure of the report. Historically, views could also be used to just filter out certain fields without any transformations, and this functionality has been maintained for backward compatibility. However, filtering-only views should be replaced by their equivalent report types as they are much faster. As previously mentioned, filtering and transforming actions are not mutually exclusive. You can filter out some fields (using a report type), and then perform a transformation on what remains (using a report view). However, not all report views are compatible with all report types. This is because some report views expect certain fields to be present. ### Report Types Report types are JSON configuration files with the following format: ```json5 { "name": "string", "exclude_fields": true, "fields": { "example_field": false, "another_example": { "example_subfield": false, "another_subfield": false } } } ``` Some default options: - `small` - Contains only the classification of the file, and some information about the file. - `extended_small` - Contains information about file classification, information about the file, the `story` field, `tags` and `interesting_strings`. - `medium` - This is the **default report** that’s served when there are no additional query parameters (in other words, it’s not necessary to specifically request this report, as it’s sent by default). - It is equivalent to the previous "summary" report with some small differences: - each subreport contains an `index` and `parent` field - if `metadata.application.capabilities` is 0, then this field is not present in the report - Changes in this report: - excludes the entire `relationships` section - excludes certain fields under the `info` section, such as `warnings` and `errors` - many `metadata` fields are not present such as those related to certificates - there are no strings, no story and no tags - `large` - Includes every single field present in the analysis report. It is equivalent to the previous "full" report (`?full=true`). **Note: You can upload custom report types to the appliance through **Central Configuration > Egress Integrations**, where you can also manage other settings for each integration. Custom report types are visible for all integrations, regardless of which section was used to upload it.** ------ Report types that replace report views with the same name: - `classification` - This report returns only the classification of the file, story and info. It has no metadata except the `attack` field. - `classification_tags` - Same as the classification view, with the addition of Spectra Core tags. - `extended` - Compared to the default (medium) type, contains:all metadata, relationships, tagsthe `story` fieldunder `info`, contains statistics and unpacking information - `mobile_detections` - Contains mobile-related metadata, as well as classification and story. - `mobile_detections_v2` - Contains more narrowly defined mobile metadata, with exclusive focus on Android. Also contains classification and story. - `short_cert` - Contains certificate and signature-related metadata, as well as indicators and some classification info. The `name` of the report type is the string you’ll refer to when calling the [Get information about a processing task](../API/UsageAPI.md#get-information-about-a-processing-task-taskinfo) endpoint (or the one passed to the relevant configuration command). For example, if the name of your report type is `my-custom-report-type`, you would include it in the query parameters as follows: `?report_type=my-custom-report-type`. The `exclude_fields` field defines the behavior of report filtering. This is an optional field and is `false` by default. This means that, by default, the report fields under `fields` will be **included** (you explicitly say which fields you **want**). Conversely, if this value is set to `true`, then the report fields under `fields` will be **excluded** (you explicitly say which fields you **don’t want**). The `fields` nested dictionary contains the fields that are either included or excluded (depending on the value of `exclude_fields`). If a subfield is set to a boolean value (`true`/`false`), then the inclusion/exclusion applies to **that section and all sections under it**. For example, `small.json`: ``` { "name": "small", "fields": { "info" : { "file": true, "identification": true }, "classification" : true } } ``` In this configuration, we’re explicitly **including** fields (`exclude_fields` was not set, so it’s `false` by default). Setting individual fields to `true` will make them (and their subfields) appear in the final report. In other words, the only sections that will be in the final report are the entire `classification` section and the `file` and `identification` fields from the `info` section. Everything else will not be present. Or, `exclude-example.json`: ``` { "name": "exclude-example", "exclude_fields": true, "fields": { "relationships": false, "info": { "statistics": false, "binary_layer": false, } } } ``` In this configuration, the entire `relationships` section is excluded, as well as `statistics` and `binary_layer` from the `info` section. Everything else will be present in the report. #### Limitations - `info.file` cannot be excluded and is always present. - **Lists cannot be partially filtered.** If you include a list field (such as `indicators` or `interesting_strings`), all items in that list are included with all their primitive fields. You cannot selectively include or exclude individual fields within list items. For example, if you want to include `indicators` but only show the `priority` field: ```json { "name": "custom-report", "fields": { "indicators": { "priority": true } } } ``` This will **not** work as expected. The entire `indicators` list will be included with all fields (`priority`, `category`, `description`, `reasons`, etc.). This limitation applies to all list types in the report schema, including: - `indicators` (struct-list) - `interesting_strings` (struct-list) - `strings` (struct-list) - `scan_results` (struct-list) - `tags` (list) - `story` (struct-list) You can include or exclude these lists as a whole, but you cannot selectively filter individual fields within list items. This applies regardless of whether you use include mode (`exclude_fields: false`) or exclude mode (`exclude_fields: true`). - **Primitive fields cannot be excluded if they share a level with an included field**. Primitive types (*string*, *number*, *boolean*, *null*) on the same level as an included field, or above it, will always be included. Only complex objects (structs) and lists can be excluded at those levels. Example structure: ```json { "classification": { "propagated": false, "classification": 3, "factor": 2, "result": "Win32.Trojan.Generic", "rca_factor": 7, "scan_results": [ { "name": "TitaniumCore RHA1", "classification": 3 } ], "propagation_source": { "sha1": "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3" } } } ``` If you include only `result` (the threat name): - `propagated`, `classification`, `factor`, `rca_factor` are included (primitives on the same level as `result`) - `scan_results` and `propagation_source` are excluded (complex types not explicitly included) Filtering result: ```json { "classification": { "propagated": false, "classification": 3, "factor": 2, "result": "Win32.Trojan.Generic", "rca_factor": 7 } } ``` ### Report Views Views are transformations of the JSON analysis output produced by the Worker. For example, views can be used to change the names of some sections in the analysis report. There are also deprecated views that allow filtering fields in or out, but this functionality is covered by report types (see above). The following views are present by default (deprecated views are excluded): - `classification_top_container_only` - Returns a report view equivalent to the `classification` report **type** (see above), but for the top-level container (parent file). - `flat` - "Flattens" the JSON structure. Without flattening: ```json "tc_report": [ { "info": { "file": { "file_type": "Binary", "file_subtype": "Archive", "file_name": "archive.zip", "file_path": "archive.zip", "size": 20324816, "entropy": 7.9999789976332245, ``` With flattening: ```json "tc_report": [ { "info_file_entropy": 7.9999789976, "info_file_file_name": "archive.zip", "info_file_file_path": "archive.zip", "info_file_file_subtype": "Archive", "info_file_file_type": "Binary", ``` - `flat-one` - Returns the `flat` report, but only for the parent file. - `no_goodware` - Returns a short version of the report for the top-level container, and any children files that are suspicious or malicious (goodware files are filtered out). This view is not compatible with split reports. - `no_email_indicator_reasons` - Strips potential PII (personally identifiable information) from some fields in analysis reports for email messages, and replaces it with a placeholder string. - `splunk-mod-v1` - Transforms the report so that it’s better suited for indexing by Splunk. The changes are as follows: - if `classification` is 0 or 1, `factor` becomes `confidence` - if `classification` is 2 or 3, `factor` becomes `severity` - a `string_status` field is added with the overall classification (UNKNOWN, GOODWARE, SUSPICIOUS, MALICIOUS) - scanner `name` becomes `reason` - scanner `result` becomes `threat` Views can generally be applied to both split (available in self-hosted deployments) and non-split reports. If none of these views satisfy your use case, contact [ReversingLabs Support](mailto:support@reversinglabs.com) to get help with building a new custom view. ## Interpreting the report After sending files to be processed, you will receive a link to a JSON report. It contains a `tc_report` field, which looks something like this: ```json "tc_report": [ { "info": { "file": { "file_type": "Text", "file_subtype": "Shell", "file_name": "test.sh", "file_path": "test.sh", "size": 35, "entropy": 3.7287244452691413, "hashes": [] } }, "classification": { "propagated": false, "classification": 0, "factor": 0, "scan_results": [ {} ], "rca_factor": 0 } } ] ``` ### High-level overview The key information here is the classification value (`tc_report[].classification.classification`), which will be a number from 0 to 3: | `classification` | description | |------------------|------------------| | 0 | no threats found | | 1 | goodware | | 2 | suspicious | | 3 | malicious | ### More information For more information, use the `tc_report[].classification.rca_factor` field. The higher its value, the more dangerous the threat, except for files that weren’t classified (their `classification` is 0). In that case, `rca_factor` will be 0 and will not signal trustworthiness. For even more information on why a file was given a certain classification, look at the `scan_results`. This field contains the individual scanners which processed the file (`name`), as well as their reason for classifying a file a given way (`result`). The following table maps the `classification` value to the old "trust factor" and "threat level" values, and the new "RCA factor" value which replaces them. It also provides a mapping to a color-coded severity value, and provides general commentary with examples regarding the origin of any given classification. ### Risk tolerance profiles Based on your risk tolerance, you can set up the downstream systems (for example, a SIEM/SOAR) to act on different values in the classification field. Here are two example risk tolerance profiles - adapt them to your own use case: #### Low risk tolerance - receive alerts on all possible threats (maximum number of matches) - act on both suspicious and malicious verdicts ```python classification = report['tc_report'][0]['classification']['classification'] if classification >= 2: alert_SOC() ``` #### High risk tolerance - receive alerts only on highly impactful threats ```python classification = report['tc_report'][0]['classification']['classification'] risk_score = report['tc_report'][0]['classification']['rca_factor'] if classification == 3 and risk_score >= 7: alert_SOC() ``` ## Spectra Core Spectra Detect Worker relies on the built-in Spectra Core static analysis engine to classify files and produce the analysis report. The file classification system can produce the following classification states: *goodware*, *suspicious*, *malicious*. With this classification system, any file found to contain a malicious file will be considered malicious itself if classification propagation has been enabled in the configuration. In the default configuration, propagation is enabled. Multiple technologies are used for file classification, such as: format identification (malware packers), signatures (byte pattern matches), file structure validation (format exploits), extracted file hierarchy, file similarity (RHA1), certificates, machine learning (for Windows executables), heuristics (scripts and fileless malware) and YARA rules. These are shipped with the static analysis engine, and their coverage varies based on threat and file format type. ## Classifying Files with Cloud-Enabled Spectra Core Spectra Core can be connected to Spectra Intelligence to use file reputation data. This data is not based solely on antivirus scanning results, but on the interpretation of the accuracy of those results by ReversingLabs, as well as on the analyst-provided (manual) classification overrides. Note that the only information Spectra Core submits to the Spectra Intelligence cloud is the file hash. Connecting Spectra Core to the cloud will add threat reputation to the `scan_results` in the report, for example: ```json { "ignored": false, "type": "av", "classification": 0, "factor": 0, "name": "", "rca_factor": 0 } ``` To connect Spectra Core to Spectra Intelligence, in the Manager, go to *Central Configuration > Spectra Intelligence*. ### How Spectra Intelligence Enhances Spectra Core Classification **Threat Naming Accuracy** When classifying files, Spectra Core takes all engines listed in the analysis report (Spectra Intelligence included) into consideration. Based on their responses, it selects the technology that provides the most accurate threat naming. More specific methods that identify the malware family more accurately are given precedence. Generic and heuristic classification are picked last, and only if there is no better-named classification response. Spectra Intelligence generally returns specific threat names, and it will be selected as authoritative if a better option is not available. It can also enhance Spectra Core classification results. For instance, Spectra Core machine learning can classify malware only with heuristic, non-named classification. If Spectra Core finds ransomware via machine learning, the threat name will appear as *Win32.Ransomware.Heuristic*. However, if Spectra Core is connected to Spectra Intelligence, the cloud response can change the threat name to something better-defined, such as *Win32.Ransomware.GandCrab*. This helps users understand exactly which malware family they are dealing with, as opposed to just the threat type (ransomware). **Whitelisting and Goodware Overrides** When not connected to Spectra Intelligence, Spectra Core can classify files as goodware either based on digital certificates that were used to sign the files, or via graylisting - a system that can declare certain file types as goodware based on the lack of code detected within them. When connected to Spectra Intelligence, whitelisting can be expanded based on file reputation and origin information. As a result, the number of unknown files (files without classification) will be significantly reduced. Users also get an insight into the trustworthiness of whitelisted files measured through trust factor values. If classification propagation is enabled on Spectra Core, a whitelisted file can still be classified as suspicious or malicious if any of its extracted files is classified as suspicious or malicious. Goodware overrides is a feature designed to prevent this. When enabled, it ensures that any files extracted from a parent and whitelisted by certificate, source, or user override can no longer be classified as malicious or suspicious. Spectra Core automatically calculates the trust factor of the certificates before applying goodware overrides, and does not use certificates with the trust factor lower than the user-configured goodware override factor. With goodware overrides enabled, classification is suppressed on any files extracted from whitelisted containers. In this case, whitelisting done by either Spectra Intelligence or certificates will be the final classification outcome. Spectra Core will still report all malicious files it finds, but they won’t be displayed as the final file classification. This feature allows for more advanced heuristic classifications that have a chance of catching supply chain attacks. As those rules tend to be noisy, they can be suppressed by using this feature. The user can still see all engine classification results, and can use them to proactively hunt for possible supply chain attacks. Note that the goodware overrides will not apply if the extracted file is found by itself (for example, if an extracted file is rescanned without its container). ### Mapping Spectra Intelligence and Spectra Core Classification States When Spectra Core is connected to Spectra Intelligence, it uses a combination of two classification systems - Spectra Core file classification and Malware Presence (MWP). While their *malicious* and *suspicious* classification states translate well from one to another, MWP *known* to Spectra Core *goodware* does not. By default, Spectra Core converts all MWP *known* states to *goodware*. This can be problematic when false negative scans happen in the Spectra Intelligence cloud, as the cloud would declare a file as KNOWN, but in reality, the file would be undiscovered malware. Such files generally have a low trust factor value. To resolve those issues, users can rescan files classified as non-malicious to confirm whether they are false negatives, or configure Spectra Core to map Spectra Intelligence *known* to *goodware* based on the trust factor. Users can configure the MWP Goodware Factor value, which defines the threshold for automatically converting MWP *known* to Spectra Core *goodware*. When this value is configured, instead of converting *known* to *goodware* in all cases, Spectra Core will only convert it when a file has a trust factor value equal to or lower than the configured one. By default, the value is configured to 5 (convert all), and the recommended value is 2. In this case, if a file classified as *known* in the cloud has a trust factor greater than 2, Spectra Core will not consider that file as *goodware*. It will be considered unclassified (with no threats found), and its cloud classification will not be present in the list of scanners in the Spectra Core analysis report. ## Spectra Detect Decision Matrix The following section explains how to interpret the classification data provided by Spectra Detect. The intent is to maximize the effectiveness of malicious classifications, while reducing the negative impact false positive detections might have. Start the decision-making process by looking at the top-level file, found first in the Spectra Detect report. Perform the following checks in order. 1. **No Threats Found classification** If the value of `tc_report.classification.classification` is "0" (no threats found), Spectra Detect detected no threats and the file is not present in ReversingLabs Cloud (or in the T1000 database). The sample could receive a classification at a later date, or it can be uploaded for analysis to the Spectra Intelligence cloud. After cloud classification, the sample may be marked as 3 (Malicious), 2 (Suspicious) or 1 (Known/Goodware). ReversingLabs reserves the "Known/Goodware" classification only for samples that the classification algorithm deems as trustworthy. If a sample remains unclassified, analysis did not find malicious intent at that time, but the sample and its metadata are not trustworthy enough to be declared "Known/Goodware." 2. **Known classification** If the value of `tc_report.classification.classification` is 1 (known), the file has been analyzed and the analysis did not find any known threats. To determine if the file is goodware, trusted and clean, perform the following checks: 1. If the value of `tc_report.classification.factor` is 0 or 1, the file is goodware and it comes from a highly reputable source. 2. If the value of `tc_report.classification.factor` is 2 or 3, the file is clean and it comes from known public sources usually unrelated to malware infections. 3. If the value of `tc_report.classification.factor` is 4 or 5, the file is likely clean and it comes from known public sources, some of which have been known to carry malware in the past. Files with this factor can change to other classifications over time, or their factor can improve when they are found in better sources. 3. **Suspicious classification** If the value of `tc_report.classification.classification` is 2 (suspicious), the file has been analyzed and the analysis found a possibility that the file is a new type of threat. This classification category is reserved for static analysis and cloud reputation heuristics, and it can lead to false positives. Depending on the risk aversion profile, two approaches are advised: 1. High risk tolerance - suspicious classifications are allowed as heuristics trigger often on files. This most commonly happens in the case where the files are collected in a corrupted or truncated state before analysis. 2. Low risk tolerance - suspicious classifications are not allowed. However, filtering on specific reasons for suspicious classifications is still available via the first element in the `tc_report.classification.scan_results[0].name` list. 4. **Malicious classification** If the value of `tc_report.classification.classification` is 3 (malicious), the file has been analyzed and recognized as a known malware threat. Depending on the risk aversion profile, two approaches are advised: 1. High risk tolerance - malicious files are not allowed, but PUA (potentially unwanted applications) are. Lower risk malware, PUA, have `tc_report.classification.factor` set to 1. In case PUA are allowed, an additional filter that blocks files with a factor greater than 1 is advised. 2. Low risk tolerance - malicious files are not allowed regardless of classification reason. ## Classification Propagation Spectra Detect unpacks files during analysis, so it is possible to have a file that is classified based on its content. Any file that contains a malicious or suspicious file is also considered malicious or suspicious because of its content. ### Propagated Classification Suppression In some cases, unpacked files might contain files that are misclassified. These false positives are propagated to the top, and it may appear that the entire archive is malicious. Suppression of these classifications is possible, and is safe, under the following conditions: 1. The classification is caused by propagation. When this happens, the optional field `tc_report.classification.propagation_source` exists. 2. Either of the following or all of the following: - The top-level file has been found in a trusted source. Find the scanner named `av` in the `tc_report.classification.scan_results` scanner list. If it exists, and its classification is 1 (known), check its factor value. If the factor value is 0 or 1, the file is goodware and it comes from a highly reputable source. - The top-level file is signed by a trusted party. Find the scanner named "TitaniumCore Certificate Lists" in the `tc_report.classification.scan_results` scanner list. If it exists and its classification is 1 (known), the file is goodware regardless of its factor value. ### Propagation Suppression Example The following false positive scenario illustrates how the suppression logic is applied: 1. A file is considered whitelisted because it is signed by a trusted digital certificate. 2. It has been classified as known and highly trusted in ReversingLabs Cloud, and has no positive antivirus detections in ReversingLabs Cloud. 3. However, during extraction, one or more malicious files were detected inside the file, and one or more malicious detections have been declared as a false positive. The classification of the file that has been submitted to Spectra Detect is indicated in the top-level section. Since it contains the `propagation_source` field, that indicates the top-level file is considered malicious because it contains at least one malicious file. The SHA1 value points to this file, which is the origin of the final top-level classification. The scanners in the `scan_results` list enumerate all the factors that contributed to the final classification. For example, they might include: 1. Resulting classification from propagation: Classification is malicious ( `classification: 3` ) with `factor: 2` 2. Certificate whitelisting for the top-level file: Classification is goodware ( `classification: 1` ) with `factor: 0` 3. Cloud response for the top-level file: Classification is goodware ( `classification: 1` ) with `factor: 0` The suppression algorithm can be applied to this top-level file, as it is not only signed with a whitelisted certificate, but is also considered known and highly trusted in ReversingLabs Cloud. ## Deep Cloud Analysis Pipeline Diagram When Deep Cloud Analysis is enabled on Spectra Detect, the appliance uses the following pipeline to analyze file submissions. ```mermaid flowchart TD %% Compact layout with all-rectangle nodes; only YES, NO, SKIP, SCAN uppercase S[START] --> A[Is the hash in user_data hash block list?] A -->|YES| X1[SKIP] A -->|NO| B[Is the hash in the global hash block list?] B -->|YES| X2[SKIP] B -->|NO| C[Is full scan enabled?] C -->|YES| Y1[SCAN] C -->|NO| D[Is this an unpacked file?] D -->|YES| E[Is the file type blocked?] E -->|YES| X3[SKIP] E -->|NO| F[CONTINUE] D -->|NO| F F[Was the file previously scanned?] F -->|NO| Y2[SCAN] F -->|YES| G[Is rescanning enabled?] G -->|NO| X4[SKIP] G -->|YES| H[Is the previous scan result older than the configured rescan interval?] H -->|YES| Y2 H -->|NO| X5[SKIP] ``` --- ## Spectra Detect Dashboard — Manager UI, Navigation, and Status Indicators This short introductory section is intended to help with understanding the basic layout of the user interface, terminology and visual indicators that are used on the Manager and in the rest of this User Guide. ## Global Header Bar At the top of the Manager interface is the global header bar, containing the most commonly used options and the main appliance menu used to access all sections of the Manager. - **Quota**: Usage-based quota insights for all Spectra Intelligence accounts used by the connected appliances. Clicking the triangle icon expands the header and displays the limit and the license renewal dates for each account. Quota limit statuses are color indicated. Additionally, this menu contains the option to contact ReversingLabs Support. - **Dashboard**: The dashboard displays statistics related to the amount and type of files that have been submitted and processed on the appliance within a specified time range. - **Central Configuration**: The [Central Configuration Manager](../Config/ApplianceConfiguration.md) allows users to modify configuration settings on Spectra Detect appliances directly from the Manager interface. The Central Configuration feature makes it easier to configure appliances remotely, and to ensure that the settings are consistent and correct across multiple appliances. The appliances must first be connected and authorized on the Manager instance. **Spectra Analyze appliances can be configured using the Spectra Detect Manager APIs.** - **Administration**: Allows users to access and configure [Spectra Detect Manager Settings](../Admin/ManagerSettings.md), [YARA Synchronization](../Config/YARASync.md), [Redundancy](../Config/Redundancy.md), and [Email Alerting](../Admin/EmailAlerting.md). - **Help**: Contains an option to access Manager API Documentation and the online product documentation. - **Notifications**: Contains a bell icon that shows unread notifications for cloud classification changes. Clicking the `See all notifications` redirects users to the [Notifications](../Config/Notifications.md) page, where they can view and manage all notifications. - **User menu**: Shows the username of the current user, contains link to user details, and the option to Log Out from the appliance. - **Integrations Status Indicators**: Contains arrow labels providing information on which file input sources and file/report output sources are currently configured on the connected Hub groups. ## Integrations Indicators The **Integrations Indicators** at the top-right of the interface contain `File Ingress Connectors` and `File Egress Connectors` providing information on which file input sources ([Connectors](../Config/AnalysisInput.md)) and file/report output sources ([Egress Integrations](../Config/ApplianceConfiguration.md#egress-integrations)) are currently configured on the connected Hub groups. Green icons indicate that the item has an existing configuration, not necessarily that the configuration is correct, and the service is connected, whereas grey icons indicate that the item does not have an existing configuration. Hub groups with configured Connectors are listed below the Connector, and outside of parentheses. ## Central Logging The Manager dashboard has two modes: central logging *enabled*/*disabled*. If central logging is enabled (*Administration > Spectra Detect Manager > Central Logging*), users can access the **Analytics** tab which shows various statistics for the processed files before showing appliance status. If it's not enabled, the default tab is **Appliance Management** showing the status of the Manager and all connected appliances. When central logging is enabled, the Analytics tab on the dashboard shows a detailed breakdown of analyzed files according to classification state, file type, malware family, the total size of all processed files, and more. This classification data and error logs can be exported to CSV by clicking the `Export Classification Data` button at the top of the page. Note that some processed files in these .csv files appear with slightly changed names, e.g. `-` become `[-]`. This is a measure to ensure sanitized input. ![](../images/c1000-dashboard-logging.png) Exported classification data CSV files contain the following information: sample (container), classification, rca_factor, malware_platform, malware_type, malware_family, filename, file_size (bytes), file_type_group, file_type, file_subtype, identification, processed_at, hostname. ## Detections Overview The Detections Overview table is a list of files analyzed on connected Workers. It updates in 15 second intervals, and shows sample classification, file name, size, file type, threat name, scan date, AV detections, SHA1 and SHA256 hashes, source, source tag, and detailed analysis. All columns, apart from File, can be enabled or disabled using the gear icon menu in the top right of the table section. The table can be sorted by clicking the column headers, and filtered using the `Show Table Filters` button which reveals text field filters above the table columns. Results can additionally be filtered by the time they were analyzed using the drop-down menu at the top of the table. Clicking the `Showing LIVE Results` button stops the table from automatically updating, which can be useful while inspecting a specific set of results. Automatic updating is also paused when the user navigates from the first page of results. Cloud analysis results do not affect the dashboard classification. However, if a newer cloud classification differs from the existing dashboard classification, a red indicator is displayed in the *Classification* column to signal that reprocessing is recommended. The *AV Detections* column in the Detections Overview table displays results from the Deep Cloud Analysis (Multi-Scanning) Service, combining Worker Static Analysis and Spectra Intelligence analysis. It can be expanded to show the names of AV scanners and any detected threats, reflecting the analysis conducted by the Spectra Intelligence, where files are sent to AV scanners in the Cloud, with the results then exposed on the Manager dashboard. To enable the Deep Cloud Analysis (Multi-Scanning) Service, navigate to *Administration > Spectra Detect Manager > Dashboard Configuration*, and select the `Enable Multi-Scanning` checkbox. When the Multi-Scanning is enabled, Workers upload samples to the Cloud only if the sample does not already exist in the Cloud, and passes the filtering criteria: up to 2GB in size. If the sample already exists in the Cloud, the Manager will monitor for any changes in the data and update the Manager dashboard accordingly. Read more details in the Multi-Scanning section of the [Spectra Detect Manager Settings](../Admin/ManagerSettings.md) chapter. The `RESCAN` button sends a request to the Cloud to check for any updates associated with the submitted samples, and updates the information accordingly. The `See All Scans` button opens a new page with all available Spectra Intelligence and AV Detections information. ### Product integration with Spectra Analyze The `Detailed Analysis` column in the Detections Overview table allows the users to import the sample analyzed on the Spectra Detect platform into Spectra Analyze for a deeper insight into the analyzed file. For easier filtering, files imported to the Spectra Analyze appliance using this feature will automatically be tagged with the `spectra_detect` tag. For this integration to be enabled, the Manager must be connected to at least one instance of Spectra Analyze 8.1 or higher, and central logging (*Administration > Spectra Detect Manager > Dashboard Configuration > Central Logging*) must be enabled. In addition, the sample must be stored either in an S3 bucket or on the Manager itself (Central Storage). If both of these features are enabled, Manager central storage takes priority (Spectra Analyze will download files from the Manager). When Spectra Analyze integration is enabled and files are uploaded, the Sample Details link is available in the `Detailed Analysis` column, allowing users to directly open the file analysis page on the configured Spectra Analyze instance. This link remains available for the same files even after disabling the integration and takes priority over the Central File Storage and AWS S3 import links when multiple options are enabled. ## Processing Timeline The Processing Timeline section of the dashboard shows a graph of uploaded, processed and post-processed samples, and also the number of samples that failed to analyze. To retrieve a list of files that failed to analyze on the connected Spectra Detect appliances in the last 90 days, use the `Export Errors / Hashes` button. The exported CSV file contains the following information: - `host_uuid` - `hostname` - `time` - `event_type` - `task_id` - `sample` - `container` The `host_uuid` value is set automatically when the Worker connects to the Manager, and obtainable using the `conf_centralmanager` command on the Worker. Note that exported error logs might contain double entries for some errors. For example, if a file processing task fails, causing a failed report upload to S3, this is counted as two errors, despite being one event. ## Malware Types / Malware Family Count The Malware Types and Malware Family Count charts show the analyzed samples categorized by Malware Type and Malware Family Count, respectively. Malware Type is presented as a percentage in a pie chart while Malware Families are displayed as a bar chart indicating the sample count per family. ## Appliance Management ![](../images/c1000-dashboard.png) System information about the Manager instance and connected appliances can be found on the **Appliance Management** tab and is updated every 5 minutes. The *Status* column indicates whether the appliance is online, offline, unlicensed, in error state, or unauthorized. If YARA ruleset synchronization is enabled (*Administration > Spectra Detect Manager > General*) and on at least one connected appliance that supports it, a *YARA* column will show the current YARA ruleset synchronization status for each appliance. Possible YARA synchronization statuses are *In Sync*, *Not In Sync*, *Unknown*, *Please Update* and *Please Set To HTTPS*. See the [YARA Synchronization](../Config/YARASync.md) section for more details. The Appliance Management page can be configured to display up to 100 appliances per page, and also filtered using the `Show Table Filters` button at the top of the list. This displays filter input fields in a row above the appliances table. Each table column has its own filter input field, allowing simultaneous filtering by multiple criteria. Keep in mind that some actions (like configuration changes) will result in a system restart. Depending on the type of appliance, the process of restarting and reloading the configuration might take some time. Spectra Detect Worker appliances generally take longer to restart. The following table describes common management actions and their consequences. | ACTION | APPLIANCE RESTART | MANAGER RESTART | |------------------------------------------------|-------------------|------------------| | Update the Manager instance | NO | YES | | Modify settings on the Manager instance | NO | YES | | Connect an appliance to the Manager | NO | NO | | Authorize an appliance | NO | NO | | Update a connected appliance | YES | NO | | Modify settings on the Appliance Status page | YES | NO | | Disconnect an appliance from the Manager | NO | NO | ## Connecting Appliances to the Manager **Note: Adding the same appliance to multiple Managers is not supported. It can lead to misconfigurations and conflicts. Always remove the appliance from one Manager before adding it to another.** To add an appliance, click `Add new appliance` on the Appliance Management tab on the Dashboard. In the *Add new appliance* dialog, choose the appliance type, then enter a name and an URL for the new appliance. **All appliance URLs must use the HTTPS protocol.** Note that the Manager does not validate SSL certificates automatically, so users must ensure their certificates are valid prior to connecting appliances. The *SNMP community* field is required for the appliance to properly communicate with the Manager, and for the Manager to display accurate status information on the dashboard page. Note that the SNMP community string set here must match the string previously configured on the appliance itself. If the selected appliance type is *TiScale Hub*, an additional field called *Load Balancer password* becomes available in the dialog. If the password is provided here, the appliance status page will display a tab with Load Balancer (HAProxy) statistics. Note that the password must be previously configured directly on the Hub, and on all Workers connected to that Hub. Clicking `Add` for Spectra Analyze appliances will redirect to the appliance login page, where the appliance must be authorized in order to successfully connect to the Manager. If authorization does not complete successfully, the appliances will be added to the Manager and displayed on the dashboard with *Unauthorized* status. They can be authorized at any moment from the [Appliance Status](#appliance-status-page) page. Workers and Hubs are immediately added to the Manager after clicking `Add`, without the authorization step. ## Appliance Status Page Apart from allowing access to *Update*, *Edit*, *Authorize*, and *Remove* options, the appliance status page provides detailed information about the system resources and health of the appliance. If there is a problem with retrieving the status information from the appliance, a warning message is displayed at the top of the appliance status page. In this case, users should check whether the SNMP community string is properly configured for the appliance on the Manager, and on the appliance itself. ![](../images/c1000-tabs.png) The information on the appliance status page is divided into tabs and refreshed every 5 minutes. The appliance type - Spectra Analyze, Spectra Detect Worker, Spectra Detect Hub - determines which tabs will be visible. | TAB | SUPPORTED ON | DESCRIPTION | |------------------|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **System** | All appliance types | Displays the appliance type, status, version, name, description, and uptime. | | **CPU** | All appliance types | Displays the number of cores on the appliance, the average load over the last minute for each individual core and for all cores. | | **Storage** | All appliance types | Shows the total storage size (in GB), the amount of used storage (in GB and percentage) and allocation units for each partition on the appliance. | | **Network** | All appliance types | Provides an overview of network devices on the appliance, with information about their type, operational and administrative status (up/down), physical address, bandwidth (in Mb/s or Gb/s), and the total amount of traffic received and sent. | | **Queues** | All appliance types except Spectra Detect Hub | If supported by the appliance type, displays the state of queues on the appliance. Queues are used to communicate between various background services. The state of all queues should be “running”, and each queue should have at least one consumer (a service connected to the queue). | | **Processing** | Worker only | If supported by the appliance type, provides statistics on the amount and size of files submitted, processed, and unpacked on the selected Worker in each of the predefined time intervals. The *Queue AVG* column indicates the average number of files in the incoming queue for each of the predefined time intervals. | | **Load Balancer**| Hub only | If supported by the appliance type, provides real-time information on the status of the HAProxy service used for load balancing on the Hub. The data is updated every 10 seconds. Hovering over column names in the tables provides tooltips that clarify what each column refers to. This tab is visible only if the load balancer (HAProxy) password is provided in the appliance configuration dialog. | | **Metrics** | Worker only | If supported by the appliance type, displays additional file processing and post-processing statistics. The statistics track the following events: files successfully uploaded to Worker and sent for analysis; success and failure events for file processing on the Worker; success and failure events for uploading parent files to S3; success and failure events for uploading analysis reports to Splunk and Callback server (if those report uploading options are configured on Worker). The statistics are collected for the same predefined time intervals as in the *Processing* tab, and preserved for the maximum duration of 1 day (24 hours). The statistics are collected using HTTP calls to the Worker service API, but the SNMP community string must be set for this tab to be visible. Counts in each interval automatically adjust to indicate only the events that occurred within the exact interval, while all events exceeding it are removed from the count. Note that the count precision may be impacted if the system is under a heavier load, but it should improve within 2-3 minutes. Additionally, extracted files are not individually counted in S3 upload events - only their parent files are. This may cause discrepancies between the count of files processed on Worker versus files uploaded to S3. | ### RabbitMQ Queues Based on the appliance type and configuration, the following RabbitMQ queues may be available. #### Spectra Detect Worker | Queue | Description | | --- | ------------- | | `tiscale.preprocessing_dispatcher` | Waits for AV analyses to complete, then routes samples to `hagent_input`, retry, or preprocessing based on thresholds and Deep Cloud Analysis settings. | | `tiscale.hagent_result` | Stores successful analysis results from `ts-worker` for postprocessing. | | `tiscale.hagent_url` | Handles submit-url and submit-docker-image requests: downloads samples and routes them to `hagent_input`, retry, or preprocessing based on thresholds and Deep Cloud Analysis settings. | | `tiscale.hagent_error` | Stores failed analysis results from `ts-worker` for postprocessing. | | `tiscale.preprocessing` | Manages subscription and upload of samples for AV scanning when Deep Cloud Analysis is enabled. | | `tiscale.preprocessing_unpacker` | Unpacks samples before preprocessing when Deep Cloud Analysis is configured to scan child files. | | `tiscale.hagent_input` | Receives samples for analysis. Routes to `hagent_result` on success or `hagent_retry` on failure. Preprocessing occurs first if Deep Cloud Analysis is enabled. | | `tiscale.hagent_retry` | Handles retry attempts for failed analyses. Routes to `hagent_result` on success or `hagent_error` on final failure. | ### Additional Options The "Upload SSL Certificate" button allows the users to apply new HTTPS certificates to Workers, Hub, or Manager. The Actions > Download logs button downloads a support archive containing relevant system logs from all connected appliances. Appliance logs can also be downloaded using the [Download support logs endpoint](../API/ServiceAPI.md#download-support-logs). #### Spectra Detect Hub For Hubs that are configured as a Hub group (redundancy cluster), the appliance status page contains additional information above the tabs and a button to promote the secondary Hub into primary. The link *Redundant with other Hub instance* allows users to quickly access the other Hub instance in the cluster, and view its status page. ## Editing and Removing Appliances To edit an appliance, click on its row in the Dashboard to access the appliance status page. Click `Edit` to open the *Configure appliance* dialog. Here the name, host name, URL, and the SNMP (Simple Network Management Protocol) community string of the appliance can be modified. The HTTPS protocol is mandatory for the appliance URL. For Spectra Detect Hub appliances, it's also possible to provide the HAProxy password to enable HAProxy monitoring. Click `Save` to save changes, or `Cancel` to return to the appliance status page without saving. To remove an appliance, click `Remove` on the appliance status page. Confirm the removal in the popup dialog that appears at the top of the page. To safely remove an appliance, always use the *Remove* option on the appliance status page when the appliance is online. Attempting to remove or replace an appliance by changing its URL, or removing it while it is offline will result in an error. Appliances periodically check whether they are still connected to the Manager. If an appliance is removed improperly, the Manager will detect it, and the appliance will be automatically removed from the Manager. ## Authorizing Spectra Analyze While Workers and Hubs are authorized automatically, Spectra Analyze appliances are not. In that case, the *Status* column in the dashboard shows an "Unauthorized" message. The `Authorization` button is visible on the status page of an unauthorized appliance, and it redirects to the authorization page on Spectra Analyze. ### Enabling Appliances Search on Spectra Analyze Spectra Analyze appliances connected and authorized to the same Manager instance can be used to perform an Appliances Search for samples. The Appliances Search feature looks for samples on all connected and authorized Spectra Analyze appliances, and provides links to the results on each appliance from Spectra Analyze Uploads page. Users can search for samples by file name and sample hash. Multi-hash search is supported, and different types of hashes (for example, MD5 and SHA1) can be submitted in one query. A notification message will appear if an appliance is not reachable or if the search cannot be performed on an appliance. To enable the Appliances Search feature on a Spectra Analyze appliance, access the *Administration > Configuration > Spectra Detect Manager* dialog and select the *Enable Appliances Search* checkbox. --- ## Spectra Detect YARA Hunting — Custom Rules, Modules, and Worker Sync ## Classifying Files with YARA Rules ## Using YARA with Spectra Detect Worker Default YARA rulesets on the appliance are automatically installed with the Spectra Core static analysis engine. With every engine update, these rulesets are updated as well. The rulesets cannot be saved to the Spectra Intelligence cloud or modified (edited, disabled, or deleted) in any way by any type of user. Additionally, ReversingLabs publishes open source YARA rules in [a public GitHub repository](https://github.com/reversinglabs/reversinglabs-yara-rules). These rules can be freely downloaded and imported into any Worker. In addition to default YARA rulesets, the Worker can use custom rulesets created by users. This is available by pulling rulesets from other Spectra Analyze and Worker appliances using [the YARA Sync feature](../Config/YARASync.md) on Spectra Detect Manager. ### Rulesets and restrictions ReversingLabs products support the following YARA modules: - PE - ELF - Math - Hash - Time - Dotnet **Note: "Import" and "include" statements are not supported.** Save custom YARA rulesets as files with the *.yara* extension. Naming restrictions: - YARA ruleset names must be between 3 and 48 characters. - The underscore ( _ ) should be used instead of spaces, and any other special characters should be avoided. Ruleset names should only use numbers (0-9) and a-z/A-Z letters. **Tip: For more information on writing YARA rulesets, consult one of the following sources:** - ReversingLabs publishes guidance for using YARA on the official blog. See the blog posts ["Level up your YARA game"](https://blog.reversinglabs.com/blog/level-up-your-yara-game) , ["Writing detailed YARA rules for malware detection"](https://www.reversinglabs.com/blog/writing-detailed-yara-rules-for-malware-detection) and ["Five Uses of YARA"](https://blog.reversinglabs.com/blog/five-uses-of-yara) to learn more. - The official YARA documentation offers detailed advice on [how to write YARA rules](https://yara.readthedocs.io/en/stable/writingrules.html). - Use Spectra Core rulesets present on the Spectra Analyze appliance as reference. ## Synchronizing YARA Rulesets via Spectra Detect Manager In order to synchronize YARA rulesets, the Worker appliance must be connected to a Manager, and YARA syncing must be enabled on that Manager. The worker-c1000 section briefly explains how to connect a Worker to a Manager. If the Worker is connected to a Manager which has YARA synchronization enabled, rulesets from the Worker will be automatically synchronized with other appliances, and vice-versa. Likewise, when YARA synchronization is disabled on a Manager that Worker is connected to, it will be automatically disabled on the Worker as well. When YARA synchronization is enabled on a Manager, the Worker will poll it for new and updated rulesets once per minute. The YARA Sync page on the Manager will display a table of all appliances connected to the Manager and their YARA ruleset synchronization status. Appliances can show one of the following statuses: - **In Sync** - The rulesets on the connected appliance match the rulesets on the Manager. - **Not In Sync** - The connected appliance doesn’t have the newest YARA rulesets. - **Unknown** - The connected appliance doesn’t have YARA synchronization enabled, or is unreachable. - **Please Update** - The connected appliance needs to be updated to a newer version before it can show the YARA synchronization status. - **Please Set To HTTPS** - The appliance is connected to the Manager using the unsupported HTTP protocol. The appliance URL must be updated to `https://` in the *Configure appliance* dialog on the Manager. Workers will poll the Manager for rule changes every minute. Spectra Analyze appliances will push new rules to the Manager as soon as they are created, and pull new rules every 5 minutes. **Example case: One Spectra Analyze appliance and one Worker attached to a Manager with YARA Synchronization enabled** Any YARA ruleset change made on Spectra Analyze will take up to 5 minutes to be synchronized to the Manager. Once the change reaches the Manager, it will take up to 1 minute for the change to be synchronized to the Worker. In total, it will take up to 6 minutes for a change to be synchronized from a Spectra Analyze appliance to a Worker. Appliances that are *Not In Sync* can be manually synchronized at any time by clicking the *Start YARA Sync* button in the far right column of the table. Rulesets created on Spectra Analyze appliances before YARA synchronization was enabled won’t synchronize to the Manager until the user changes their status or modifies them in any way. Rules present on the Manager, however, will synchronize to newly connected Spectra Analyze appliances regardless of when they were created. Apart from new rulesets, changes in existing rulesets will be synchronized as well. If a ruleset is disabled or deleted on one appliance, its status will be distributed to other appliances. In case of Workers, disabled rulesets will be removed until they are re-enabled on another appliance. When enabled again, those rulesets will be synchronized to the Worker as if they have been newly created. This means that the Worker only contains enabled (synchronizable) rulesets at all times. ## Troubleshooting YARA Issues on the Worker 1. From the Worker appliance status page on the Manager, disconnect the Worker by clicking the *Remove* button. 2. Access the dashboard page and click the *Add new appliance* button. 3. In the *Add new appliance* dialog that opens, select *Spectra Detect Worker* as the appliance type, and fill in the configuration fields with the data of the previously disconnected Worker instance. 4. Click *Submit* to connect the Worker instance to the Manager again. If the process completes successfully, the YARA Sync page on the Manager should display the status of the Worker instance. --- ## Spectra Detect Usage — Analysis, Dashboards, and YARA Management --- ## Spectra Detect Troubleshooting — Workers, Queues, YARA Sync, and Updates # Troubleshooting This guide covers common issues with [Spectra Detect](./index.md) deployments and the steps to resolve them. --- ## Worker node not appearing in SDM dashboard **Symptom** A newly deployed or restarted Worker node does not appear in the Spectra Detect Manager (SDM) appliance list, or its status shows as disconnected. **Cause** - The Worker is not configured with the correct SDM address or port. - A firewall is blocking the communication channel between the Worker and SDM. - The Worker has not been authorized in SDM. - TLS certificate mismatch between the Worker and SDM. **Solution** 1. On the Worker appliance, verify that the SDM address is correctly configured under [Appliance Configuration](./Config/ApplianceConfiguration.md). 2. Test network connectivity from the Worker to the SDM: ```bash curl -k https://:/api/v1/health-check/ ``` If this fails, check firewall rules between the Worker and SDM host. The required ports are documented in the deployment prerequisites. 3. In the SDM interface, navigate to **Appliances** and check whether the Worker appears as pending authorization. Newly connected Workers must be explicitly authorized before they appear as active. See [Manager Settings](./Admin/ManagerSettings.md). 4. If TLS certificates are custom, verify that the Worker trusts the SDM certificate. See [Certificate Management](./Config/CertificateManagement.md) for certificate configuration. 5. Check SDM logs for connection rejection messages to identify the specific failure. --- ## File analysis queue is backing up **Symptom** The [Dashboard](./Usage/Dashboard.md) shows the analysis queue growing continuously. Files are taking much longer than normal to receive a classification result. The queue count in the SDM overview is increasing or not decreasing. **Cause** - The cluster does not have enough Worker nodes or CPU resources to keep up with the current ingestion rate. - A subset of Worker nodes is unhealthy or offline, reducing effective throughput. - Very large or deeply nested archives are occupying Worker instances for extended periods. - The input connector (S3, folder watch, or API) is ingesting files faster than Workers can process them. **Solution** 1. Check the status of all Worker nodes in the SDM dashboard. Confirm that all expected Workers are online and healthy. 2. Review Worker CPU and memory usage. Sustained CPU at 100% indicates the Workers are capacity-constrained. 3. For K8s deployments, scale the Worker deployment horizontally by increasing the replica count: ```bash kubectl scale deployment spectra-detect-worker \ --replicas= -n ``` --- ## YARA rules not syncing to Workers **Symptom** YARA rules uploaded to an appliance or to SDM do not appear on Worker nodes. Samples that should match a YARA rule do not produce expected results. The SDM YARA management page shows rules as uploaded but Workers do not reflect the changes. **Cause** - The YARA sync service on a Worker node is stopped or misconfigured. - A network connectivity issue is preventing SDM from pushing rules to the Worker. - A syntax error in a newly uploaded YARA rule is causing the entire ruleset to be rejected by the Worker. **Solution** 1. Verify that YARA sync is enabled and configured correctly on the Worker. See [YARA Sync](./Config/YARASync.md) for the required configuration parameters. 2. Check YARA sync logs on the Worker node for error messages: ```bash # For OVA/AMI deployments journalctl -u spectradetect-yarasync -n 100 # For K8s deployments kubectl logs -c yara-sync -n ``` 3. If the logs show rule validation errors, identify the specific rule causing the failure. Invalid rules are typically logged with the rule name and a parse error message: ``` ERROR: YARA rule parse failed: rule "my_rule" at line 14 - unexpected token ``` Remove or correct the invalid rule and re-upload the ruleset. 4. Test SDM-to-Worker connectivity on the YARA sync port. Consult [Certificate Management](./Config/CertificateManagement.md) if TLS is involved. 5. After resolving connectivity or rule issues, trigger a manual YARA sync from the SDM interface or via the [Management API](./API/ManagementAPI.md). --- ## S3 connector not picking up files **Symptom** Files placed in the configured S3 bucket are not being submitted for analysis. The analysis queue remains empty even though new files are present in the bucket. **Cause** - The S3 connector credentials (AWS access key or IAM role) are incorrect or lack the required permissions. - The bucket name or prefix path in the connector configuration does not match the actual bucket. - The S3 connector service is stopped or has encountered an error during startup. - The bucket is in a different AWS region than expected. **Solution** 1. Review the S3 connector configuration under [Analysis Input](./Config/AnalysisInput.md). Confirm the bucket name, region, and prefix are correct. 2. Verify that the IAM role or access keys have the following minimum permissions on the bucket: - `s3:GetObject` - `s3:ListBucket` - `s3:DeleteObject` (if files should be removed after processing) ```bash # Test access from the connector host aws s3 ls s3://// --region ``` 3. Check the connector service logs for authentication or connectivity errors: ```bash # For OVA/AMI deployments journalctl -u spectradetect-connector -n 100 # For K8s deployments kubectl logs -n ``` 4. Confirm the connector service is running. Restart it if it has crashed: ```bash # For OVA/AMI deployments sudo systemctl restart spectradetect-connector ``` 5. For K8s deployments, note that S3 connector support is provided through the Hub component. Refer to your deployment documentation for Hub configuration details. --- ## SDM shows appliance as unreachable **Symptom** In the SDM overview, one or more connected appliances (Workers or Spectra Analyze instances) display an "Unreachable" status. Alerts may be generated for the affected appliances. **Cause** - The appliance is powered off or has lost network connectivity. - The SDM heartbeat check is failing due to a temporary network disruption. - The appliance's management interface has changed IP address. - A TLS certificate on the appliance has expired, causing the SDM connection to fail. **Solution** 1. Attempt to access the appliance web interface or SSH directly to confirm it is online: ```bash ssh admin@ ping -c 4 ``` 2. In the SDM interface, navigate to **Manager Settings** and verify the registered IP address or hostname for the appliance. Update it if the IP has changed. See [Manager Settings](./Admin/ManagerSettings.md). 3. Check whether the appliance TLS certificate has expired. Certificate issues are logged in the appliance system logs and also visible on the Certificate Management page. See [Certificate Management](./Config/CertificateManagement.md) for certificate renewal steps. 4. If the appliance is accessible but SDM still shows it as unreachable, restart the SDM communication service on the appliance: ```bash sudo systemctl restart spectradetect-sdm-agent ``` 5. Review SDM logs for specific error messages related to the unreachable appliance to identify the root cause. --- ## Analysis results not appearing in dashboard **Symptom** Files are being submitted and Workers show activity (CPU usage, queue movement), but analysis results do not appear in the SDM [Dashboard](./Usage/Dashboard.md) or in the analysis results view. **Cause** - The results reporting service on the Worker is misconfigured and is not sending results back to the Hub or SDM. - A downstream results consumer (webhook, SIEM integration, or S3 output bucket) is misconfigured, causing result delivery to fail silently. - The SDM database is not receiving result records due to a connectivity issue between the Hub and SDM. **Solution** 1. Check Worker logs for result delivery errors: ```bash kubectl logs -n | grep -i "result\|output\|error" ``` 2. Verify the output configuration under [Analysis Input](./Config/AnalysisInput.md). Confirm that the output destination (Hub address, S3 bucket, or webhook URL) is correct and reachable. 3. Test connectivity from the Worker to the Hub or SDM results endpoint. 4. For notification-based integrations (email alerts, webhooks), check the notification configuration. See [Notifications](./Config/Notifications.md) for configuration options. 5. If results are being generated locally but not forwarded, check for disk space issues on the Worker node that might be causing result queuing to back up locally rather than forwarding to SDM. --- ## Update fails or gets stuck **Symptom** A software update initiated from the SDM **Updating** page (or via the update CLI) does not complete. The update status page shows the process as "In Progress" for an extended period, or the update fails with an error message. **Cause** - Network connectivity between the appliance and the ReversingLabs update server is disrupted during the download. - The appliance does not have sufficient disk space to stage the update package. - An SDM-managed appliance was rebooted or lost connectivity to SDM mid-update. - An incompatible update sequence (for example, skipping a required intermediate version). **Solution** 1. Navigate to the [Updating](./Admin/Updating.md) page in SDM and review the update log for specific error messages. 2. Check available disk space on the appliance before retrying: ```bash df -h ``` If disk space is insufficient, free space by purging old analysis results or logs before reattempting the update. 3. Test network connectivity to the ReversingLabs update infrastructure from the appliance: ```bash curl -I https://updates.reversinglabs.com ``` 4. Do not reboot or restart the appliance while an update is in progress unless instructed to do so in the error message. 5. If the update process is hung (no log activity for more than 30 minutes), contact [ReversingLabs Support](mailto:support@reversinglabs.com) before attempting to cancel or retry the update, as incomplete updates may leave the system in a partially upgraded state. 6. For K8s deployments, follow the upgrade procedure in your deployment documentation rather than using the SDM update mechanism. --- ## High memory usage on Worker nodes **Symptom** Kubernetes Worker pods are being OOM-killed, or node memory usage is consistently above 90%. The SDM dashboard or cluster monitoring (Prometheus, CloudWatch) shows frequent memory pressure on Worker nodes. **Cause** - The number of concurrent analyses (`concurrency-limit`) is set too high relative to the available memory per Worker pod. - Large files or archives with high decompression ratios are consuming more memory than typical workloads. - Memory limits set in the Helm chart are too low for the configured number of Spectra Core instances. **Solution** 1. Check if OOM events are occurring: ```bash kubectl describe pod -n | grep -i "OOMKilled\|Reason" dmesg | grep -i "oom\|killed" ``` 2. Review the Worker's Helm chart values and reduce the concurrency limit or the number of Spectra Core instances (`number-of-regular-cores`) to reduce peak memory usage. 3. Increase the memory limits and requests for Worker pods in the Helm values if available node memory allows it. Refer to your deployment's Helm values reference documentation. 4. Consider configuring a dedicated large-file Worker group with higher memory limits and a separate concurrency limit, leaving the regular pool for smaller files. Refer to your deployment documentation for Worker group customization options. 5. Review the [platform requirements](/General/DeploymentAndIntegration/PlatformRequirements) to confirm that the cluster nodes meet the recommended memory specifications for the configured number of Spectra Core instances. --- ## Cluster certificate errors **Symptom** Connections between cluster components fail with TLS errors. SDM reports appliances as unreachable. Browser access to the SDM web interface shows a certificate warning. Logs contain messages such as: ``` TLS handshake failed: x509: certificate has expired or is not yet valid TLS handshake failed: x509: certificate signed by unknown authority ``` **Cause** - Internal cluster certificates have expired and have not been renewed. - A custom CA certificate used to sign internal certificates is not trusted by all components. - The system clock on one or more nodes is incorrect, causing certificate validity window checks to fail. **Solution** 1. Identify which certificate is causing the failure by examining the TLS error in detail: ```bash openssl s_client -connect : -showcerts 2>&1 | openssl x509 -noout -dates ``` 2. Check certificate expiration across cluster components. See [Certificate Management](./Config/CertificateManagement.md) for the full list of certificates used and the renewal procedure. 3. Renew expired certificates following the procedure documented in [Certificate Management](./Config/CertificateManagement.md). After renewal, restart the affected services. 4. Verify that the system clock is synchronized on all nodes: ```bash timedatectl status chronyc tracking ``` If clock drift is detected, synchronize with NTP and verify that `ntpd` or `chronyd` is running and configured. 5. If an "unknown authority" error is present, ensure that the custom CA certificate is distributed to and trusted by all cluster components and the SDM host. See [Certificate Management](./Config/CertificateManagement.md) for CA distribution steps. --- ## Spectra Intelligence Automation API — Upload, Download & Monitor The Automation APIs enable programmatic management of samples in the Spectra Intelligence repository, including uploading, downloading, deleting, and monitoring files for changes. ## Common Use Cases ### Upload and manage samples 1. **[File upload (TCA-0202-0203)](tca-0202-0203.md)** - Upload samples and metadata to Spectra Intelligence for analysis 2. **[Reanalyze file (TCA-0205)](tca-0205.md)** - Trigger rescanning of existing samples with updated AV signatures 3. **[Delete file (TCA-0204)](tca-0204.md)** - Remove samples you own from the repository ### Download samples - **[File download (TCA-0201)](tca-0201.md)** - Download samples from the repository by hash (MD5, SHA1, SHA256) ### Monitor for changes - **[Reputation and metadata alerts (TCA-0206)](tca-0206.md)** - Subscribe to samples and receive notifications when their classification or metadata changes ## All Automation APIs --- ## File Download API (tca-0201) — Spectra Intelligence # File download (TCA-0201) This service allows a user to download a sample from the Spectra Intelligence repository. The download request requires the MD5/SHA1/SHA256 value of the sample to download, and receives a byte stream containing the requested sample. Faster downloads are supported through multithreading. It is suggested to use at most 4 parallel threads. If there is a need for additional threads, contact your sales representative. **This API is rate limited to 100 requests per minute.** ## Sample Download This query returns the contents of a sample matching the requested hash. The contents are returned as a byte stream.